Image processing device, image processing method and program

Abstract

To increase the convenience for an examiner to make a diagnosis.SOLUTION: A first analysis map is the result of analysis of three-dimensional data obtained by photographing an area including an optic papilla section in a first period of time. A second analysis map is the result of analysis of three-dimensional data obtained by photographing an area including a macula part in the first period of time. A third analysis map is related to the optic papilla section which is the result of analysis of three-dimensional data obtained by photographing an area including the optic papilla section and the macula part in a second period of time after the first period of time. A fourth analysis map is related to the macula part. An image processing device arranges the first analysis map and the third analysis map in chronological order to display them in a first display area, and arranges the second analysis map and the fourth analysis map in chronological order to display them in a second display area different from the first display area.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The disclosed technology relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Optical coherence tomography is known as an ophthalmic imaging device for photographing the subject's eye. Optical coherence tomography uses optical coherence tomography (OCT) to obtain cross-sectional and three-dimensional images of the fundus and anterior segment of the eye. Furthermore, the data obtained by the optical coherence tomography is used for analysis processing to understand the condition of the subject's eye.

[0003] The analysis process includes an analysis of the thickness of the fundus layer (fundus layer thickness analysis). In fundus layer thickness analysis, a thickness distribution of a specific layer is generated and compared with a normal eye database. The target layers include the retinal nerve fiber layer (RNFL), ganglion cell layer (GCL), a composite layer of the ganglion cell layer and inner plexiform layer (IPL), and a composite layer of the ganglion cell layer, inner plexiform layer, and retinal nerve fiber layer.

[0004] There are cases where it is desirable to perform analysis processing over a wide range. In fundus thickness analysis for glaucoma, it is desirable to target both the region including the optic disc and the region including the macula. In the conventional technology, data collection for the region including the optic disc and data collection for the region including the macula are performed separately, and the collected data are analyzed separately.

[0005] Here, Patent Document 1 discloses a technique for photographing an area including the optic disc and the macular region as a wide area of ​​the fundus using optical coherence tomography (OCT). Patent Document 1 also discloses a technique for displaying an analysis map related to the optic disc and an analysis map related to the macular region obtained by separately analyzing an area including the optic disc and an area including the macular region, superimposed on a front image of the subject's eye, in order to easily grasp the state of the wide area of ​​the fundus. At this time, Patent Document 1 discloses that the analysis map related to the optic disc and the analysis map related to the macular region are displayed partially superimposed on each other. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2017-074325 A Summary of the Invention [Problem to be solved by the invention]

[0007] Here, in the diagnosis of glaucoma, in order to observe the analysis map related to the optic disc and the analysis map related to the macular region over time, it may be necessary to compare the past data and the current data obtained from the same examined eye. In this case, depending on the state of the past data, for example, it may be necessary to compare the analysis map obtained by analyzing the three-dimensional data obtained by separately photographing the area including the optic disc and the area including the macular region in the past with the analysis map obtained by analyzing the three-dimensional data obtained by photographing the area including the optic disc and the macular region in the present.

[0008] One of the objectives of the disclosed technology is to improve convenience for examiners when making diagnoses.

[0009] In addition to the above-mentioned objective, another objective of the present invention is to achieve an effect that is derived from each configuration shown in the description of the embodiment of the invention described below, and that cannot be obtained by conventional techniques. [Means for solving the problem]

[0010] One of the disclosed image processing devices is A region including the optic disc of the fundus of the subject eye is imaged as a first image using optical coherence tomography. A first analysis is an analysis result of three-dimensional data on the optic disc obtained by photographing the optic disc at and a region including the macula of the fundus is imaged using optical coherence tomography. A second analysis map is an analysis result of the three-dimensional data on the macular region obtained by photographing the macular region at the time of the second analysis map. and an area including the optic disc and the macula using optical coherence tomography. The result of analysis of the three-dimensional data obtained by photographing the subject at a second time after the first time is shown in FIG. A third analysis map relating to the optic disc and a fourth analysis map relating to the macula are An acquisition unit that acquires the information; The first analysis map and the third analysis map are displayed in a time series manner in a first display area. and controlling a display unit to display the second analysis map and the fourth analysis map in a time series manner. The display unit displays the images in a second display area different from the first display area in a row. A display control unit that controls an adjustment unit that adjusts the resolution of the optic disc or the macula in the three-dimensional data in order to compare with statistical information of a normal eye; Equipped with. Effect of the Invention

[0011] According to one of the disclosed techniques, it is possible to improve convenience for examiners when making diagnoses. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing the configuration of an image processing system. [Diagram 2] 2A to 2C are diagrams for explaining the structure of an eye, a tomographic image, and a fundus image. [Diagram 3] 4 is a flowchart showing a processing flow in the image processing system. [Figure 4] FIG. 4 is a diagram for explaining a shooting area. [Diagram 5] FIG. 4 is a diagram for explaining a screen for displaying an image. [Figure 6] FIG. 4 is a diagram for explaining a screen for displaying an image. [Figure 7] FIG. 1 is a diagram showing the configuration of an image processing system. [Figure 8] FIG. 4 is a diagram for explaining a screen for displaying an image. [Figure 9] 11 is a flowchart showing a flow of selecting time-series data. [Figure 10] FIG. 13 is a diagram for explaining time-series data selection. [Figure 11] FIG. 4 is a diagram for explaining a screen for displaying an image. [Figure 12] 4 is a flowchart showing a processing flow in the image processing system. [Figure 13] FIG. 13 is a diagram for explaining motion contrast data. [Figure 14] FIG. 4 is a diagram for explaining a screen for displaying an image. [Figure 15] FIG. 1 is a diagram showing the configuration of an image processing system. [Figure 16] A diagram to explain a machine learning model. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] First Embodiment Hereinafter, a first embodiment will be described with reference to the drawings. According to one of the present embodiments, an analysis of a region including the optic disc and an analysis of a region including the macula (fovea) are performed on a single three-dimensional data of the fundus obtained by one scan pattern (for example, raster scan) using optical coherence tomography (OCT), and the results of these analyses (analysis maps) are displayed side by side on the display unit. That is, according to one of the present embodiments, an analysis map of the optic disc and an analysis map of the macula are obtained as the analysis results of three-dimensional data obtained by photographing a region including the optic disc and the macula of the fundus of the subject eye using optical coherence tomography, and these analysis maps are displayed side by side on the display unit. At this time, according to one of the present embodiments, these analysis maps and a two-dimensional image of the region including the optic disc and the macula are displayed side by side on the display unit. This can improve the convenience of the examiner in making a diagnosis. For example, this is because it eliminates the situation that occurs in the prior art, where an analysis map relating to the optic disc and an analysis map relating to the macula partially overlap each other, causing part of one analysis map to become invisible.

[0014] Here, the analysis map of the optic disc may be any two-dimensional image showing the analysis result of the region including the optic disc. The analysis map of the macular region may be any two-dimensional image showing the analysis result of the region including the macular region. The two-dimensional image showing the analysis result is, for example, a layer thickness map or a blood vessel density map described later, a map showing the comparison result between the analysis information (for example, layer thickness information) obtained by the analysis and the statistical information (normative data) of a normal eye. The two-dimensional image of the region including the optic disc and the macular region is, for example, an SLO image (or a color fundus image or a monochrome fundus image) obtained by photographing the region including the optic disc and the macular region. The two-dimensional image of the region including the optic disc and the macular region is, for example, three-dimensional data obtained by using OCT, such as a front image (enface image) obtained by using at least a part of the three-dimensional data in the depth direction of the subject's eye. In addition, the two-dimensional image of the area including the optic disc and the macula is, for example, a layer thickness map obtained by superimposing layer thickness information obtained by analyzing at least a portion of the three-dimensional data in the depth direction of the test eye as color information (with a predetermined transparency set) on the above-mentioned front image.

[0015] According to another embodiment of the present invention, a two-dimensional image of a region including the optic disc and the macular region and one of the analysis maps of the optic disc and the macular region may be displayed on the display unit in an overlapping manner. At this time, according to another embodiment of the present invention, the other analysis map and the two-dimensional image may be displayed side by side on the display unit. As a result, for example, the analysis map of the optic disc and the analysis map of the macular region can be displayed so as not to overlap with each other partially. Therefore, the convenience of the examiner in making a diagnosis can be improved. The analysis map displayed on the two-dimensional image may be displayed on the corresponding position on the two-dimensional image in an overlapping manner. The other analysis map may be displayed on the display unit in an overlapping manner on another two-dimensional image. At this time, the two two-dimensional images on which the analysis maps are displayed on top of each other may be the same type of two-dimensional images or different types of two-dimensional images.

[0016] An image processing system including an image processing device according to this embodiment will be described in detail below.

[0017] Fig. 1 is a diagram showing the configuration of an image processing system 100 including an image processing device 300 according to this embodiment. As shown in Fig. 1, the image processing system 100 is configured by connecting the image processing device 300 to a tomographic image capturing device 200, a fundus image capturing device 400, an external storage unit 500, a display unit 600, and an input unit 700 via interfaces.

[0018] The tomographic imaging device 200 is a device that captures a tomographic image of an eye. The tomographic imaging device 200 is configured with an optical coherence tomography (OCT) such as SD-OCT (Spectral domain OCT) or SS-OCT (Swept Source OCT). Here, SD-OCT is an OCT that uses a broadband light source and acquires an interferogram with a spectroscope. Also, SS-OCT is an OCT that uses a high-speed wavelength swept light source as a light source and measures spectral interference with a single channel photodetector. In this case, the tomographic imaging device 200 has a detection unit (not shown) that detects interference light between return light from the fundus irradiated with measurement light and reference light, and is an example of an ophthalmic imaging device including OCT. Note that the tomographic imaging device 200 is a known device, so a detailed description is omitted, and here, the imaging of a tomographic image performed in response to an instruction from the image processing device 300 and analysis processing are described.

[0019] In Fig. 1, a galvanometer mirror 201 is used to scan the fundus with measurement light, and defines the fundus imaging range by OCT. A drive control unit 202 controls the driving range and speed of the galvanometer mirror 201 to define the imaging range in the planar direction of the fundus and the number of scanning lines (scanning speed in the planar direction). Here, the galvanometer mirror is shown as one unit for simplicity, but in reality it is composed of a mirror for X-scan and a mirror for Y-scan, and a desired range on the fundus can be scanned with the measurement light.

[0020] The focus 203 is for focusing on the retinal layer of the fundus through the anterior segment of the eye, which is the subject. The measurement light is focused on the retinal layer of the fundus by a focus lens (not shown) through the anterior segment of the eye, which is the subject. The measurement light that irradiates the fundus is reflected and scattered by each retinal layer and returns.

[0021] The internal fixation light 204 is composed of a display unit 241 and a lens 242. The display unit 241 uses a plurality of light-emitting diodes (LD) arranged in a matrix. The lighting position of the light-emitting diodes is changed according to the part to be photographed under the control of the drive control unit 202. The position of the fixation light may be a macular fixation light position for photographing the periphery of the macular region, an optic disc fixation light position for photographing the periphery of the optic disc, or a posterior pole central fixation light position for photographing both the macular region and the optic disc. The light from the display unit 241 is guided to the subject's eye via the lens 242. The light emitted from the display unit 241 is 520 nm, and a desired pattern is displayed by the drive control unit 202.

[0022] The coherence gate stage 205 is controlled by the drive control unit 202 to accommodate differences in axial length of the subject's eye. The coherence gate refers to the position where the optical distance between the measurement light and the reference light in OCT is equal. Furthermore, as an imaging method, the position of the coherence gate is controlled to control imaging on the retinal layer side or deeper than the retinal layer. Here, the structure of the eye and the images acquired by the image processing system will be described with reference to FIG. 2.

[0023] Fig. 2(a) shows a schematic diagram of an eyeball. In Fig. 2(a), C represents the cornea, CL represents the crystalline lens, V represents the vitreous body, M represents the macular region (the central portion of the macula represents the fovea), and D represents the optic disc. The tomographic imaging apparatus 200 according to this embodiment will be described mainly in terms of imaging the posterior pole of the retina including the vitreous body, the macular region, and the optic disc. Note that the tomographic imaging apparatus 200 can also image the anterior segments of the eye, such as the cornea and the crystalline lens.

[0024] FIG. 2(b) shows an example of a tomographic image obtained by the tomographic imaging device 200 when the retina is photographed. In FIG. 2(b), AS represents an A-scan, which is a unit of image acquisition in an OCT tomographic image. A number of A-scans constitute one B-scan. This B-scan is called a tomographic image (or a tomographic image). In FIG. 2(b), V represents the vitreous body, M represents the macula, D represents the optic disc, and La represents the lamina cribrosa. In addition, L1 represents the boundary between the internal limiting membrane (ILM) and the nerve fiber layer (NFL), L2 represents the boundary between the nerve fiber layer and the ganglion cell layer (GCL), L3 represents the inner-outer segment junction (ISOS), L4 represents the retinal pigment epithelium layer (RPE), L5 represents Bruch's membrane (BM), and L6 represents the choroid. In the tomographic image, the horizontal axis (main scanning direction of OCT) is the x-axis, and the vertical axis (depth direction) is the z-axis.

[0025] FIG. 2(c) shows an example of a fundus image acquired by the fundus image capturing device 400. The fundus image capturing device 400 is a device for capturing a fundus image of an eye, and examples of such devices include a fundus camera and a scanning laser ophothalmoscope (SLO). In FIG. 2(c), M represents the macula, D represents the optic disc, and the thick curve represents the blood vessels of the retina. In the fundus image, the horizontal axis (main scanning direction of OCT) is the x-axis, and the vertical axis (sub-scanning direction of OCT) is the y-axis. The tomographic image capturing device 200 and the fundus image capturing device 400 may be configured as an integrated device or as separate devices.

[0026] The image processing device 300 includes an image acquisition unit 301, a storage unit 302, an image processing unit 303, an instruction unit 304, and a display control unit 305. The image acquisition unit 301 includes a tomographic image generation unit 311, which acquires signal data of a tomographic image captured by the tomographic image capturing device 200 and generates a tomographic image by performing signal processing. It also acquires fundus image data captured by the fundus image capturing device 400. Then, the generated tomographic image and fundus image are stored in the storage unit 302. The image processing unit 303 includes a layer detection unit 331, an analysis unit 332, a feature detection unit 333, a region adjustment comparison unit 334, and a two-dimensional data generation unit 335.

[0027] The layer detection unit 331 detects the boundary line of each layer from the tomographic image. The analysis unit 332 calculates the thickness of each layer based on the boundary line information detected by the layer detection unit 331. The feature detection unit 333 detects the optic disc and the fovea of ​​the macular region using at least one of the tomographic image and the fundus image. The region adjustment comparison unit 334 adjusts the position and size of the region for comparison with normative data (a statistical value calculated from multiple data) described later based on the position of the part detected by the feature detection unit 333, and then compares the data. The two-dimensional data generation unit 336 generates two-dimensional data based on the analysis result of the analysis unit 332 and the comparison result of the region adjustment comparison unit 334.

[0028] The external storage unit 500 stores information about the subject's eye (such as the patient's name, age, and sex), captured image data, capture parameters, image analysis parameters, and parameters set by the operator, each associated with the other. In addition, normative data generated from information about a plurality of normal eyes is also stored. For example, the normative data is the thickness of the NFL around the optic disc, the thickness of the composite layer of the NFL, GCL, and IPL around the fovea of ​​the macular region, and the like.

[0029] The input unit 700 is, for example, a mouse, a keyboard, a touch screen, or the like, and the operator issues instructions to the image processing device 300, the tomographic image capturing device 200, and the fundus image capturing device 400 via the input unit 700.

[0030] Next, the processing procedure of the image processing device 300 of this embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the operation processing of the entire system in this embodiment.

[0031] <Step S301> In step S301, the subject's eye information acquiring unit (not shown) acquires a subject's identification number from the outside as information for identifying the subject's eye. Then, based on the subject's identification number, the information on the subject's eye stored in the external storage unit 500 is acquired and stored in the storage unit 302.

[0032] <Step S302> In step S302, the subject's eye is scanned and photographed. When the operator selects to start scanning (not shown), the tomographic imaging apparatus 200 controls the drive control unit 202 to operate the galvanometer mirror 201 to scan the tomographic image. The galvanometer mirror 201 is composed of an X scanner for the horizontal direction and a Y scanner for the vertical direction. Therefore, by changing the orientation of each of these scanners, it is possible to scan in the horizontal direction (X) and the vertical direction (Y) in the apparatus coordinate system. By simultaneously changing the orientation of these scanners, it is possible to scan in a direction that combines the horizontal and vertical directions, making it possible to scan in any direction on the fundus plane.

[0033] When performing imaging, various imaging parameters are adjusted. Specifically, at least the position of the internal fixation light, the scan range, the scan pattern, the coherence gate position, and the focus are set. The drive control unit 202 controls the light-emitting diode of the display unit 241 to control the position of the internal fixation light 204 so that imaging is performed at the center of the posterior pole, the center of the macular region, and the center of the optic disc. The scan pattern is set to a raster scan, radial scan, cross scan, or other scan pattern for imaging a three-dimensional volume. In this embodiment, the position of the fixation light is the center of the posterior pole, the scan pattern is raster scan, and the scan range is a wide range (wide area) that covers everything from the optic disc to the macular region. For example, in this embodiment, the range is 13×10 mm, and the main scanning direction is a vertical raster scan. Furthermore, the center of the imaging range is imaged by horizontal scanning. The horizontal scan is performed before or after the three-dimensional raster scan. The horizontal scan is performed by taking one or more images (for example, about 10 images). After these adjustments of the imaging parameters are completed, the operator selects start of imaging (not shown) to perform imaging. The tomographic imaging device 200 can also perform tracking of the subject's eye. This reduces the influence of fixational eye movement, and when data is lost due to blinking or the like, good quality 3D data can be obtained by rescanning only the area where data is lost.

[0034] <Step S303> In step S303, a tomographic image is generated. The tomographic image generating unit 311 generates a tomographic image by performing a general reconstruction process on each of the interference signals.

[0035] First, the tomographic image generating unit 311 removes fixed pattern noise from the interference signal. Fixed pattern noise removal is performed by extracting fixed pattern noise by averaging multiple detected A-scan signals and subtracting this from the input interference signal. Next, the tomographic image generating unit 311 performs a desired window function process to optimize the depth resolution and dynamic range, which are in a trade-off relationship when Fourier transform is performed in a finite section. Next, a tomographic signal is generated by performing FFT processing.

[0036] <Step S304> In step S304, the layer detection unit 331 performs layer detection. The layer detection unit 331 performs noise removal and edge enhancement processing on the tomographic image to be processed. For example, a median filter or a Gaussian filter is applied as the noise removal processing. For edge enhancement processing, a Sobel filter or a Hessian filter is applied. Here, edge enhancement processing on a two-dimensional tomographic image is described using a two-dimensional Hessian filter. The Hessian filter can enhance the secondary local structure of the two-dimensional gray distribution based on the relationship between two eigenvalues ​​(λ1, λ2) of the Hessian matrix. The two-dimensional line structure is enhanced using the relationship between the eigenvalues ​​and eigenvectors (e1, e2) of the Hessian matrix. Since the line structure in the two-dimensional tomographic image corresponds to the retinal layer, the structure of the retinal layer is enhanced. In order to detect retinal layers with different thicknesses, it is sufficient to change the resolution of the smoothing by the Gaussian function performed when calculating the Hessian matrix. In addition, when applying a two-dimensional Hessian filter, it is desirable to apply it after transforming the data so as to match the physical size of the XZ of the image. In the case of general OCT, the physical size in the XY direction is different from that in the Z direction. Therefore, a filter is applied by adjusting the physical size of the retinal layer for each pixel. Alternatively, if the physical size is not normalized, it can be approximately handled by changing the resolution of smoothing by a Gaussian function. The above describes processing with a two-dimensional tomographic image, but this is not limited to this. If the data structure when a tomographic image is captured is a three-dimensional tomographic image by raster scanning, it is also possible to apply a three-dimensional Hessian filter. In this case, after performing XZ alignment processing in an alignment unit (not shown) between adjacent tomographic images, the secondary local structure of the three-dimensional gray distribution can be emphasized based on the relationship between the three eigenvalues ​​(λ1, λ2, λ3) of the Hessian matrix. It is also possible to emphasize edges three-dimensionally by emphasizing the three-dimensional layer structure using the relationship between the eigenvalues ​​and eigenvectors (e1, e2, e3) of the Hessian matrix.

[0037] The layer detection unit 331 detects a boundary line from the tomographic image 703 that has been subjected to edge enhancement processing. In this embodiment, the boundary line between the ILM and the NFL and the RPE are first obtained, and then the boundaries between the ISOS, the NFL and the GCL are detected. Although not shown, other boundary lines may be detected, such as the boundary line between the outer plexiform layer (OPL) and the outer nuclear layer (ONL), the boundary line between the inner plexiform layer (IPL) and the inner nuclear layer (INL), the boundary line between the INL and the OPL, the boundary line between the GCL and the IPL, and the boundary line between the choroid and the sclera. As a method of detecting the boundary line, a process is performed in which a plurality of points with strong edge strength are detected as boundary candidates in each A-scan, and points are connected as lines based on the continuity between the boundary candidates in adjacent A-scans. When the points are connected as lines, the smoothness of the lines is evaluated to remove outliers. Then, the boundary line is determined based on the distance and positional relationship between the upper and lower boundaries. If no boundary line is detected as a result of removing the outliers in each A-scan, the boundary line may be obtained by interpolation from the surrounding boundaries. Alternatively, a boundary candidate may be searched for in the horizontal direction based on edges from the surrounding boundaries, and the boundary may be determined again based on the boundary candidate searched for from the surrounding boundaries.

[0038] Then, a process for correcting the shape of the detected boundary line to make it smooth is executed. For example, the shape of the boundary line may be smoothed using image features and shape features by Snakes or the like. Alternatively, the coordinate values ​​of the boundary line shape may be regarded as time-series data of a signal, and the shape may be smoothed by a smoothing process such as a Savitzky-Golay filter, a simple moving average, a weighted moving average, or an exponential moving average. Alternatively, instead of detecting the boundary line for all A-scans, the boundary line may be detected discretely (for example, at intervals of 5 A-scans), and the boundary line position of the A-scan where the boundary line is not detected may be obtained by interpolation from the detected boundary line.

[0039] <Step S305> In step S305, the analysis unit 331 calculates the thickness of any layer based on the detection result of the layer detection unit 331. For example, in the case of the thickness of the NFL, the difference in the depth direction of the region defined by the boundary lines L1 and L2 in FIG. 2(b) is the thickness of the layer. It is possible to calculate the thickness of a layer that combines the NFL, GCL, and IPL, as well as the thickness of all layers including all layers from the NFL (or ILM) to the BM (or RPE). Since the calculation result is a value of one point per 1 A scan, the thickness is calculated in all A scans and B scans to create a two-dimensional layer thickness map. If there are multiple types of layers to be calculated, multiple two-dimensional layer thickness maps will be output.

[0040] <Step S306> In step S306, the feature detection unit 333 detects the optic disc and the central fovea of ​​the macular region. First, an example of detection related to the optic disc is shown. The feature detection unit 333 adjusts the contrast of the SLO image or the Enface image (frontal retinal image) generated from multiple OCT tomographic images to make the image brightness uniform to a certain standard. Then, the image is smoothed using a smoothing filter such as a Gaussian filter or a median filter. Furthermore, in order to emphasize the image features, the contrast may be enhanced by performing processing such as CLAHE (Contrast Limited Adaptive Histogram Equalization). These preprocessing processes make it possible to extract a circular (elliptical) area with low brightness value from the image, and the area is set as the optic disc. Furthermore, not only the frontal retinal image such as SLO or Enface, but also the layer boundary line result in the OCT tomographic image may be used. For example, as shown in D in FIG. 2(b), the optic disc is a large concave area. Therefore, the optic disc may be extracted from the shape features obtained by the detection result of the layer boundary line.

[0041] Next, an example of detection for the macular fovea is shown. The macular fovea is a small depression region, as shown by M in Figure 2(b). Furthermore, the distance between the ILM and IS / OS is closer than other locations. Therefore, the depression region of the macular fovea can be obtained by using the distance between the ILM and IS / OS. Note that although it is a depression region, the IS / OS does not exist in the optic disc, so the feature using the distance between the ILM and IS / OS in the optic disc becomes an invalid region.

[0042] In steps S304 and S306, examples of detecting layer boundaries and features using image processing have been described, but the present invention is not limited to this. Any layer or feature may be detected from an input image by learning images and correct labels using a machine learning method such as deep learning.

[0043] <Step S307> In step S307, the area adjustment comparison unit 334 adjusts the area to be compared with the normative data based on the positions of the optic disc and the macular region extracted by the feature extraction unit 333. This will be described with reference to Fig. 4. Note that the normative data of this embodiment will be described assuming that the data around the optic disc and the data around the macular region are stored separately.

[0044] A rectangular area 401 shown by a solid line in FIG. 4(a) shows an example of a wide-range imaging area that covers everything from the optic disc to the macula in this embodiment. For example, in this embodiment, the area is 13×10 mm. A rectangular area 402 shown by a dashed line shows the range of normative data in the optic disc, and is, for example, 6×6 mm. A rectangular area 403 shown by a dotted line shows the range of normative data in the macula, and is, for example, 10×10 mm. FIG. 4(b) shows the overlapping areas in FIG. 4(a) separately. FIGS. 4(a) and 4(b) are examples showing the area by size, but in the case of OCT, the resolution differs depending on how many A-scans and how many B-scans are used to capture a certain area. This is shown in FIG. 4(c). In FIG. 4(c), for the sake of explanation, the characteristics of the optic disc, the macula, and blood vessels are omitted, and only the ratio of A-scans to B-scans is shown. In Fig. 4(c), rectangular area 411 is a size indicating the ratio of A scans and B scans corresponding to rectangular area 401, and arrow 421 indicates the main scanning direction. In this embodiment, rectangular area 411 is an example indicating a size of 500 A scans and 250 B scans. Similarly, rectangular area 412 corresponds to rectangular area 402, and rectangular area 413 corresponds to rectangular area 403, and each indicates a size indicating the ratio of A scans and B scans. Rectangular area 412 indicates a size of 500 A scans and 500 B scans, and rectangular area 413 is an example indicating a size of 1000 A scans and 125 B scans.

[0045] As shown in FIG. 4, even if the imaging range during wide-range imaging covers the imaging range during normative data acquisition, the resolution during imaging may differ from the resolution during normative data acquisition. Therefore, the region adjustment comparison unit 334 adjusts the layer thickness map region obtained in step S305. Here, an example of region adjustment is shown. The region adjustment comparison unit 334 enlarges or reduces the layer thickness map corresponding to the rectangular region 411 at a ratio of 13:10 so as to correspond to the imaging region. As a result, a layer thickness map corresponding to the imaging region 401 as shown in FIG. 4(b) is obtained. Since the layer thickness map is two-dimensional, the method of enlargement or reduction may be general image processing such as bicubic or bilinear, or super-resolution processing previously learned by machine learning may be applied. From the enlarged or reduced layer thickness map, a range of 6:6 and 10:10 is specified, respectively, centered on the positions corresponding to the positions of the optic disc and macula extracted by the feature extraction unit 333 in step S306. Normative data corresponding to the optic disc and macula are also scaled to 6:6 and 10:10, respectively, and compared with the layer thickness map.

[0046] In the above, an example of simple enlargement / reduction is described regardless of the number of A scans and B scans during shooting, but the present invention is not limited to this. For example, enlargement / reduction may be performed after thinning out data with a high resolution when data is acquired. For example, the rectangular area 412 is an example in which data is acquired in a range of 6×6 mm at 500×500 and normative data is generated. This may be enlarged / reduction after thinning out the data to match the rectangular area 411 of this embodiment. For example, by thinning out 500×500 data to 115×300 and then executing the enlargement / reduction process, the scale conversion is almost the same as the enlargement / reduction of the rectangular area 411, and the interpolation process by the enlargement / reduction is also almost the same. Therefore, even if the resolution during shooting and the resolution during normative data acquisition are different, the resolution during data comparison can be made almost the same.

[0047] Here, the area adjustment comparison unit 334 is an example of an adjustment unit that adjusts the number of A scans and B scans in the three-dimensional data by adjusting the position and size of a portion of the three-dimensional data for comparison with statistical information of normal eyes, based on the position of the optic disc or macula in the three-dimensional data.

[0048] In this embodiment, the layer thickness map is created in step S305, and the comparison range is specified in step S307 with the optic disc and the macular region as the center, but the processing flow is not limited to this. For example, after the comparison range is specified with the optic disc and the macular region as the center in the fundus front image, the layer thickness map may be created based on the boundary data included in the range.

[0049] <Step S308> In step S308, the two-dimensional data generating unit 335 generates two-dimensional data (also called an analysis map) of the optic disc periphery and the macular fovea periphery, which are compared with the normative data by the region adjustment comparing unit 334. In this embodiment, the analysis map generated by the two-dimensional data generating unit 335 is a significance map and a deviation map. The significance map is a map based on the normative data, in which a measurement value less than the 1st percentile of the confidence interval is abnormal, less than the 5th percentile is a borderline region, and the rest is within the normal range, and the display color is changed accordingly. The deviation map is a map that shows the difference from the average image of the layer thickness map calculated from the normative data by color. The layer thickness map has already been created in the analyzing unit 332 when the layer thickness was calculated.

[0050] The two-dimensional data generating unit 335 generates, for example, a significance map and a deviation map of the NFL around the optic disc. Also, around the fovea of ​​the macular region, a significance map and a deviation map of all layers including a layer combining the NFL, GCL, and IPL, and all layers from the NFL (or ILM) to the BM (or RPE) are generated. The significance map and the deviation map are examples of two-dimensional images (analysis maps) showing the analysis results.

[0051] Here, the analysis map for the optic disc may be a first comparison map showing a comparison result between the layer thickness information for the optic disc obtained by analyzing the three-dimensional data and statistical information (normative data) for normal eyes related to the optic disc. The analysis map for the macular region may be a second comparison map showing a comparison result between the layer thickness information for the macular region obtained by analyzing the three-dimensional data and statistical information (normative data) for normal eyes related to the macular region. In this case, the significance map and the deviation map are examples of the first comparison map and the second comparison map.

[0052] The two-dimensional data generating unit 335 is an example of an acquiring unit that acquires an analysis map of the optic disc and an analysis map of the macular region, which are analysis results of three-dimensional data acquired by photographing an area including the optic disc and the macular region of the fundus of the subject eye using optical coherence tomography. At this time, the acquiring unit can generate an analysis map of the optic disc and an analysis map of the macular region by analyzing the three-dimensional data. The acquiring unit may be configured to acquire the analysis map of the optic disc and the analysis map of the macular region as data from an external storage unit or an external system by wired or wireless communication.

[0053] <Step S309> In step S309, the captured image, layer thickness map, analysis map, and the like are displayed on the display unit 600. FIG. 5 shows an example of a screen displayed on the display unit 600. 505 indicates the entire screen, 501 indicates a patient tab, 502 indicates an imaging tab, 503 indicates a report tab, and 504 indicates a setting tab. The diagonal lines in the report tab 503 indicate the active state of the report screen. In this embodiment, an example of displaying a report screen will be described. FIGS. 5(a) and 5(b) show display examples of two types of report screens. In this embodiment, a wide range of scans are performed to capture images covering the optic disc and the macula. Therefore, for example, FIG. 5(a) shows a report example for diagnosing glaucoma with a layout centered on the optic disc. Also, FIG. 5(b) shows a report example for diagnosing macular diseases with a layout centered on the macula. These layouts are examples and are not limited to two types, but it is assumed that multiple layouts according to the purpose can be selected from one scan pattern.

[0054] In FIG. 5(a), 510 is a fundus photograph, and 511 is an image obtained by enlarging the optic disc area from the fundus photograph. If the fundus photograph 510 is not taken, other images such as an SLO image or an infrared image that captures the front of the fundus are displayed. 512 is a layer thickness map that represents the overall retinal thickness, and is an example of a two-dimensional image related to a region including the optic disc and the macula. 513 is a layer thickness map that represents the overall retinal thickness superimposed on the layer thickness map. The ETDRS grid 513 displays the average value of the overall retinal thickness in each sector. 514 is a layer thickness map that represents the thickness of the RNFL. Information 515 indicating the range (for example, a range of 10×10 mm) around the macula is an example of information indicating the position of the analysis map related to the macula in the two-dimensional image. The layer thickness map 514 may be either one of the layer thickness map showing the thickness of the RNFL layer and the layer thickness map showing the thickness of the NFL, GCL, and IPL combined (or the thickness of the GCL and IPL combined). These layer thickness maps may be configured to be switched from one to the other for display. This display switching may be configured to be executable using, for example, a SplitButton. It is not necessary to use a SplitButton, but may be a RadioButton, and it is sufficient if one can be selected from a plurality of options. Information 516 indicating the range around the optic disc (for example, a range of 6×6 mm) is an example of information indicating the position of the analysis map related to the optic disc in the above two-dimensional image. 517 indicates a circle around the optic disc, and is, for example, a circle with a diameter of 3.45 mm. 518 and 519 indicate a significance map and a deviation map generated based on normative data for the RNFL thickness in the range of 6×6 mm shown in 516. 520 and 521 show a significance map and a deviation map generated based on normative data for the thickness of the layer in which the NFL, GCL, and IPL are combined, for the area of ​​10×10 mm shown in 515.The layer thickness map, significance map, and deviation map are displayed by superimposing the results in color on an image of the front of the fundus generated from the tomographic image, and are superimposed semi-transparently (with a specified transparency) by setting the transparency.

[0055] Also, 522 indicates a tomographic image of the circle indicated by 517, and 523 indicates a thickness profile of the RNFL in the tomographic image of the circle 517. In the thickness profile 523, the thick lines indicate the measured RNFL thickness values, and the thin lines indicate the normal, borderline, and abnormal ranges of the normative data. Although the thickness profile 523 is displayed in TSNIT (Temporal, Superior, Nasal, Inferior), it is also possible to display it in the order of NSTIN. For switching between the display of TSNIT and NSTIN, for example, the initial display of TSNIT or NSTIN can be set in a setting screen (not shown). Alternatively, a menu can be displayed by right-clicking on the screen, and the display can be switched there. 524 indicates measured values ​​related to the optic disc and RNFL thickness, and displays, for example, the Disc area, Rim area, C / D ratio, and the average value of the TSNIT thickness of the RNFL. 525 indicates a thickness grid of the RNFL in the optic disc, which is divided into 4 and 12 parts. The number of divisions is not limited to this and may be six. In the sectors of the grid, the average thickness value and whether the value is normal, borderline, or abnormal are indicated by color. 526 is a thickness grid of the combined layer of the NFL, GCL, and IPL in the macula, divided into nine parts vertically and horizontally, and into the whole and two parts vertically. Like grid 525, in the sectors of grid 526, the average thickness value and whether the value is normal, borderline, or abnormal are indicated by color.

[0056] In FIG. 5(a), 520, 521, and 526 display information on the layer formed by combining the NFL, GCL, and IPL, but information on the layer formed by combining the GCL and IPL may be displayed. Regarding the switching of the display, the initial display of the layer formed by combining the NFL, GCL, and IPL or the layer formed by combining the GCL and IPL can be set on a setting screen (not shown). Alternatively, a menu can be displayed by right-clicking on the screen, and the display can be switched there. In the case of switching by right-clicking (an example of an instruction from the examiner), when a selection process is performed on any of the display areas 520, 521, and 526, the display of these three types (520, 521, and 526) is switched in conjunction with each other. In this case, if the layer thickness map 514 displays the thickness of the layer formed by combining the NFL, GCL, and IPL, not the RNFL, the layer thickness map 514 may also be configured to switch in conjunction with the display of the layer thickness map representing the thickness of the layer formed by combining the GCL and IPL. That is, when either the layer corresponding to the layer thickness map 514 or the layer corresponding to the second comparison map (520 or 521) is switched to another layer (for example, one of two types of composite layers) in response to an instruction from the examiner, the other may be switched to another layer in conjunction. This allows the layer thickness map 514 and the second comparison map to be configured to be changed in display to analysis maps corresponding to different layers. At this time, the layer corresponding to the analysis map of the optic disc (for example, the RNFL) may be configured not to be changed even if the instruction is given. This allows the observation of analysis maps corresponding to different layers in the macular region and the optic disc, so that, for example, diagnosis of glaucoma can be performed efficiently. Note that the display in FIG. 5(b) described below is also changed in conjunction with the display switching here.

[0057] In FIG. 5(b), the difference from FIG. 5(a) will be described. 530 is a horizontal scan tomographic image. In this embodiment, 3D imaging and horizontal scanning are performed as shown in step S302. The horizontal scan 530 is performed by imaging one or more images (for example, about 10 images). When imaging multiple images, it is desirable to perform averaging processing after aligning the multiple tomographic images with each other by an alignment unit (not shown) to generate one tomographic image. Then, the tomographic image subjected to the averaging processing is displayed as the horizontal scan 530. Although not shown, instead of the horizontal scan 530, a vertical scan tomographic image obtained by 3D imaging may be displayed, or a vertical scan tomographic image may be displayed alongside the horizontal scan 530. 532 is a significance map generated based on normative data regarding the thickness of the entire retina, and 533 is a layer thickness map showing the thickness of a layer obtained by combining the NFL, GCL, and IPL. Note that 533 and 520 display information on a layer in which NFL, GCL, and IPL are combined, and this is assumed to be switched in conjunction with the types described in FIG. 5(a). That is, when the display is switched to the layer in which GCL and IPL are combined on the screen in FIG. 5(a), the display of the two types (533, 520) in FIG. 5(b) displays information on the layer in which GCL and IPL are combined. As described in FIG. 5(a), the display of the layer types can be switched in FIG. 5(b) as well. Note that in FIG. 5(b), 532, 520, and 518 display a significance map but do not display a deviation map, but these can be switched to display them. The display of the significance map and the deviation map can be switched by, for example, SplitButton. Also, by changing any one of the significance maps to a deviation map, the other significance maps may be switched in conjunction with it.

[0058] Fig. 5 shows a display example in which one eye is displayed. Fig. 6 shows an example in which both eyes are displayed simultaneously. Fig. 6 displays the image and analysis map described in Fig. 5 in a state in which they can be compared. Note that Fig. 6 shows an example in which both eyes are displayed simultaneously, but this is not limiting, and for example, it may be an example in which images and analysis maps of the same eye taken at different times are displayed in a state in which they can be compared.

[0059] It should be noted that, on the display screen, it is possible to output the image displayed on the report screen as a report in a printable format by an output means (not shown).

[0060] <Step S310> In step S310, an instruction acquisition unit (not shown) externally acquires an instruction as to whether or not to end the capturing of tomographic images by the image processing system 100. This instruction is input by the operator using the input unit 700. When an instruction to end the process is acquired, the image processing system 100 ends the process. On the other hand, when the process is not to be ended and the capturing is to be continued, the process returns to step S302 and the capturing is continued. In this manner, the processing of the image processing system 100 is performed.

[0061] According to the above-described configuration, it is possible to display a report for diagnosing glaucoma with a layout centered on the optic disc, and a report for diagnosing macular diseases with a layout centered on the macular region. Therefore, in a tomographic image acquired over a wide area, processing according to the part can be suitably performed, and the analysis results can be effectively presented.

[0062] Second Embodiment In the first embodiment, a display form that allows a wide range of imaging to be performed and allows the analysis results of the optic disc and the macula to be easily understood is shown. In this embodiment, an example of displaying time-series image data obtained by performing wide-range imaging will be described. Descriptions of functions similar to those of the first embodiment will be omitted here.

[0063] Fig. 7 is a diagram showing the configuration of an image processing system 1007 including an image processing device 3007 according to this embodiment. As shown in Fig. 7, the image processing device 3007 includes an image processing unit 703, and the image processing unit 703 includes a time series unit 731. The time series unit 731 selects data to be compared as follow-up observation.

[0064] In this embodiment, the imaging as described in the first embodiment is performed multiple times on different days, and multiple data have been acquired. In this regard, FIG. 8 shows an example of displaying multiple time series data side by side as follow-up observation. In FIG. 8, FIG. 8(a) is an example of arranging data in a time series in which the area adjustment comparison unit 334 extracts the areas of the optic disc and the macular region from data obtained by performing wide-area imaging. FIG. 8(b) is an example of arranging data obtained by performing wide-area imaging and data obtained by performing detailed imaging of the optic disc and the macular region in a time series. FIG. 9 is a flowchart showing the flow of data selection of time series data by the time series unit 731 in this embodiment. FIG. 10 is a diagram for explaining data selection from multiple selection data candidate dates.

[0065] Here, the selection and display of time series data will be described with reference to FIGS.

[0066] <Step S901> In steps S901 to S903, in addition to the reference data, conditions for the data to be selected are obtained. As the data conditions, for example, a flag of 0 or 1 is obtained. When there are multiple conditions, the flag does not have to be limited to 0 or 1.

[0067] In step S901, when multiple images are taken on the same day with the same scan pattern, whether data from the same day is selectable or not is acquired. If data from the same day cannot be selected, a flag of 1 is acquired, and if data from the same day can be selected, a flag of 0 is acquired. If data from the same day cannot be selected and multiple data exists on the same day, NG is not set as the image data, and the latest data from the same day is selected. Note that in this embodiment, a description will be given assuming that a flag of 1 is acquired.

[0068] <Step S902> In step S902, when not only wide-range photography but also optic disc and macular data are photographed, whether or not optic disc and macular data are included as comparison targets is acquired. If only wide-range photography is used as a comparison target, a flag of 1 is acquired, and if data photographed of the optic disc and macular is included, a flag of 0 is acquired. In this embodiment, a display example when a flag of 1 is acquired in step S902 is shown in FIG. 8(a), and a display example when a flag of 0 is acquired in step S902 is shown in FIG. 8(b). FIG. 8 will be described later.

[0069] <Step S903> In step S903, a flag is obtained for the data to be selected in addition to the reference data. For example, the selection method may be to select multiple pieces of data with the latest date, or to select data at equal intervals between the reference data and the latest date. In this embodiment, a flag of 1 is obtained, and an example of data selection will be described with reference to the next step S905 and FIG. 10.

[0070] <Step S904> In step S904, the time series unit 731 selects reference data. Reference data refers to data that is used as a reference when measuring changes in analytical values. The reference data is selected from the data of the subject's eye that is the first to be photographed over a wide range. Alternatively, the operator can select any reference data using a reference data setting means (not shown). Note that if a flag of 0 is acquired in step S902, the data that is first to be photographed over a wide range and data that photographs the optic disc and macula is selected as the reference data.

[0071] <Step S905> In step S905, data other than the reference data and the latest data is selected. This data selection will be explained using FIG. 10. Here, we will explain the case where there are 10 candidate dates for selected data, and the selected data is for 3 days. Note that for each date, data captured with the desired scan pattern exists. FIG. 10(a) shows an example of selecting multiple data for the latest date (flag is 1 in step S903). FIG. 10(b) shows an example of selecting data at equal intervals between the reference data and the data for the latest date (flag is 0 in step S903). The circle 1001 is the reference data, the circle 1002 is the latest data, and the rectangle 1003 is an example of selected comparison data.

[0072] In FIG. 10, data for a period of 10 days is shown as the candidate dates for selected data. In this embodiment, the list of dates displayed as the candidate dates for selected data displays the dates on which wide-area photographing data, optic disc photographing data, and macular area photographing data have been acquired. If the desired photographing data does not exist, it is not displayed as a candidate in the list of candidate dates for selected data. That is, in step S902, if the flag is 1, the dates on which wide-area photographing data has been acquired are displayed in the list of candidate dates for selected data. In step S902, if the flag is 0, the dates on which wide-area photographing data, or data on which the optic disc or macular area has been acquired are displayed in the list of candidate dates for selected data.

[0073] FIG. 10 shows an example in which there is more photographed data than selected data. If there is less photographed data than selected data, the same data will be selected regardless of the flag setting in step S903. For example, if the selection data candidate dates are four days, excluding the reference data and the latest data, the number of comparison data is two days. In that case, if the selection data is three days, two days of comparison data will be selected regardless of the condition.

[0074] <Step S906> In step S906, the time series data is displayed. Display of the time series data will be described with reference to FIG.

[0075] In FIG. 8, FIG. 8(a) is an example in which data in which the area adjustment comparison unit 334 extracts the areas of the optic disc and the macula from data obtained by performing wide-area photography is arranged in a chronological order. That is, this is a case in which flag 1 is obtained in step S901, flag 1 in step S902, and flag 1 in step S903. In FIG. 8(a), 810 indicates a button for the operator to select any data. The display of the time-series data automatically displays an image selected based on the flow in FIG. 9. However, the operator can select data different from the automatically selected data by pressing button 810. 811 indicates an image in which the optic disc area is enlarged from a fundus photograph. Note that if a fundus photograph is not taken, an image in which the optic disc area is enlarged from another image in which the front of the fundus is photographed, such as an SLO image or an infrared image, is displayed.

[0076] Also, 812 indicates a layer thickness map showing the thickness of the RNFL. Also, 813 indicates a layer thickness map showing the thickness of the layer formed by combining the NFL, the GCL, and the IPL. Here, a display in which a plurality of these layer thickness maps are arranged in time series is referred to as a display of time series data. At this time, the past layer thickness maps in the time series data of the layer thickness maps 812 and 813 are examples of a first analysis map related to the optic disc and a second analysis map related to the macular region, which are analysis results of three-dimensional data obtained by photographing an area including the optic disc and the macular region at a first time using optical coherence tomography. Also, the current (latest) layer thickness map in the time series data of the layer thickness maps 812 and 813 are examples of a third analysis map related to the optic disc and a fourth analysis map related to the macular region, which are analysis results of three-dimensional data obtained by photographing an area including the optic disc and the macular region at a second time after the first time using optical coherence tomography. At this time, the display control unit 305 can control the display unit 600 to display a plurality of analysis maps (the first analysis map and the third analysis map) related to the optic disc in a chronological order in a first display area (for example, an area where the time-series data of the layer thickness map 812 is displayed). The display control unit 305 can also control the display unit 600 to display a plurality of analysis maps (the second analysis map and the fourth analysis map) related to the macular region in a chronological order in a second display area different from the first display area (for example, an area where the time-series data of the layer thickness map 813 is displayed).

[0077] Also, 814 shows a thickness profile of the RNFL at the circle position around the optic disc, and shows multiple thickness profiles for time series data. 815 shows a graph of the change in TSNIT thickness over time. The vertical axis of the graph shows thickness, and the horizontal axis of the graph shows the imaging date. 817 and 818 show measurement values ​​related to the optic disc and RNFL thickness, and display, for example, the Disc area, Rim area, C / D ratio, and the average value of the TSNIT thickness of the RNFL in time series. 816 shows a graph of the change in thickness of the layer formed by combining the NFL, GCL, and IPL over time. 819 displays the average value of the thickness of the layer formed by combining the NFL, GCL, and IPL in time series.

[0078] Here, 812 and 813 show examples of layer thickness maps showing thickness in chronological order. However, this is not limiting, and a significance map or deviation map may be displayed. Furthermore, a difference map showing the difference from a base map may be displayed. When changing one of these map images, the remaining maps are switched in conjunction.

[0079] FIG. 8(b) is an example in which data obtained by performing wide-area imaging and data obtained by performing detailed imaging of the optic disc and the macular region are arranged in chronological order. That is, this is a case where flag 1 is obtained in step S901, flag 0 in step S902, and flag 1 in step S903 are obtained. In FIG. 8(b), 822 and 823 show examples of data obtained by performing detailed imaging of the optic disc and the macular region, and the remaining data are examples of data obtained by performing wide-area imaging (an example of an analysis result of three-dimensional data obtained by imaging in an imaging mode that images an area including the optic disc and the macular region). 822 displays an icon indicating that the data is obtained by imaging the optic disc (an example of an analysis result of three-dimensional data obtained by imaging in an imaging mode that images the optic disc separately). Also, 823 displays an icon indicating that the data is obtained by imaging the macular region (an example of an analysis result of three-dimensional data obtained by imaging in an imaging mode that images the macular region separately). It is not necessary to use an icon, but rather, it is sufficient to display an indicator that lets the operator know that the image is different from wide-angle photography, such as by displaying a color around the image or changing the color of the date displayed at the top of the image. That is, the display control unit 305 may display information on the display unit 600 indicating that the first analysis map, the second analysis map, the third analysis map, and the fourth analysis map described below are identifiable as analysis results of three-dimensional data obtained by photographing in different photographing modes. In addition, when data is selected that captures the optic disc or macular region in detail, the graphs and measurement values ​​displayed in 814 to 819 are also displayed as values ​​calculated from the data.

[0080] That is, the data on detailed photographing of the optic disc among the time series data is, for example, a first analysis map which is an analysis result of three-dimensional data on the optic disc obtained by photographing an area including the optic disc at a first time using optical coherence tomography. The data on detailed photographing of the macular portion among the time series data is, for example, a second analysis map which is an analysis result of three-dimensional data on the macular portion obtained by photographing an area including the macular portion at a first time using optical coherence tomography. Note that the first time of photographing the area including the optic disc and the first time of photographing the area including the macular portion do not have to be exactly the same time, and may be, for example, the same date. The data on wide-area photographing among the time series data is, for example, a third analysis map on the optic disc and a fourth analysis map on the macular portion which are analysis results of three-dimensional data obtained by photographing an area including the optic disc and the macular portion at a second time after the first time using optical coherence tomography. At this time, the display control unit 305 can control the display unit 600 to display the first analysis map and the third analysis map in a chronological order in a first display area (for example, an area where the time-series data of the layer thickness map 812 is displayed). The display control unit 305 can also control the display unit 600 to display the second analysis map and the fourth analysis map in a chronological order in a second display area different from the first display area (for example, an area where the time-series data of the layer thickness map 813 is displayed).

[0081] FIG. 8(a) shows an example of arranging data in which the optic disc and macular regions are extracted from data obtained by wide-area photography. The example of arranging and displaying data obtained by wide-area photography in a time series is not limited to this, and may be displayed as shown in FIG. 11, for example. FIG. 11 shows an example of displaying the entire wide-area image without extracting and displaying the optic disc and macular regions from data obtained by wide-area photography. 1111 shows a fundus photograph, and 1113 shows a map showing the layer thickness of the entire retina. 1116 shows a graph showing the change over time in the layer thickness of the entire retina, and 1119 shows the average layer thickness of the entire retina in time series. The figure shows a layer thickness map showing the layer thickness of the entire retina, but a layer thickness map showing the thickness of the RNFL, a layer thickness map showing the thickness of the combined layer of the NFL, GCL, and IPL, a significance map, and a deviation map may be displayed.

[0082] According to the above-mentioned configuration, it is possible to display the analysis results of the optic disc and the macula from the data obtained by wide-area imaging as time-series image data. Furthermore, it is possible to display the data obtained by wide-area imaging and the data obtained by detailed imaging as time-series image data.

[0083] Third embodiment In the first and second embodiments, a display form that allows a wide range of OCT images to be captured and allows the analysis results of the optic disc and macula to be easily understood is shown. In this embodiment, an example of capturing a wide range of OCT images and generating and displaying a wide range of OCT angiography (OCTA) is shown with reference to Figs. 12 and 13. Note that in this embodiment, the configuration of an image processing system equipped with an image processing device such as that shown in Figs. 1 and 7 is not shown, but it is assumed that a motion contrast data generating unit is included.

[0084] Fig. 12 is a flow chart showing the operation process of the entire system in this embodiment, and Fig. 13 is a flow chart showing the process in this embodiment. Descriptions of parts having the same functions as those in the first and second embodiments will be omitted here.

[0085] <Step S1202> In step S1202, the subject's eye is scanned and photographed. When the operator selects to start scanning (not shown), the tomographic imaging apparatus 200 controls the drive control unit 202 to operate the galvanometer mirror 201 and perform scanning of a tomographic image.

[0086] In this embodiment, the scan pattern is a three-dimensional volume by raster scanning, and for example, a range of 13×10 mm is photographed at intervals of 500×250 (main scanning×sub-scanning). In the three-dimensional volume, the same line location is photographed repeatedly M times (M is 2 or more) to calculate the motion contrast. That is, when M is 2, 500×500 data is actually photographed, and 500×250 three-dimensional motion contrast data is generated from the data. In this embodiment, three-dimensional data and horizontal scan photography are performed as described in the first embodiment. One or multiple images (for example, about 10 images) are photographed for the horizontal scan. In this embodiment, the horizontal scan is described as taking multiple images.

[0087] In order to capture the same location for averaging, the tomographic imaging apparatus 200 can also scan the subject's eye while reducing the influence of fixational eye movement by tracking the subject's eye. Furthermore, if a movement that becomes an artifact when generating an image, such as blinking, is detected, a rescan is automatically performed at the location where the artifact occurred.

[0088] <Step S1204> In step S1204, the motion contrast data generating unit generates motion contrast data. This data generation will be described with reference to FIG. 13. MC indicates three-dimensional motion contrast data, and LMC indicates two-dimensional motion contrast data constituting the three-dimensional motion contrast data. Furthermore, LMC' indicates a horizontal scan that captures the center of a three-dimensional region. In this embodiment, the generation of the horizontal scan LMC' will be described as an example.

[0089] The motion contrast data generating unit first corrects the positional deviation between a plurality of tomographic images captured in the same area of ​​the subject's eye. Any method may be used to correct the positional deviation. For example, the motion contrast generating unit captures the same area M' times, and aligns the tomographic image data corresponding to the same location using features such as the fundus shape. Note that, in this embodiment, "identical" does not only refer to a location that is completely identical, but also refers to a location that includes a deviation of about several μm to several tens of μm that occurs even if the positional deviation of the eye due to tracking is corrected.

[0090] Specifically, one of the M' pieces of tomographic image data is selected as a template, and the similarity between the template and the other tomographic image data is calculated while changing the position and angle of the template, and the amount of positional deviation from the template is calculated.The motion contrast generating unit then corrects each piece of tomographic image data based on the calculated amount of positional deviation.

[0091] Next, the motion contrast generating unit obtains a decorrelation value M(x,z) between two pieces of tomographic image data whose imaging times are consecutive to each other, by using Equation 1.

[0092]

number

[0093] Here, A(x,z) indicates the brightness at the position (x,z) of the tomographic image data A, and B(x,z) indicates the brightness at the same position (x,z) of the tomographic image data B.

[0094] The decorrelation value M(x,z) is a value between 0 and 1, and the greater the difference between the two luminances, the greater the value of M(x,z). When M', which is repeatedly acquired at the same position, is 3 or more, the motion contrast generation unit can obtain multiple decorrelation values ​​M(x,z) at the same position (x,z). The motion contrast generation unit can generate final motion contrast data by performing statistical processing such as maximum value calculation or average calculation of the obtained multiple decorrelation values ​​M(x,z). Note that when the number of repetitions M is 2, statistical processing such as maximum value calculation or average calculation is not performed, and the decorrelation value M(x,z) of two adjacent tomographic images A and B becomes the value of the motion contrast at the position (x,z).

[0095] The motion contrast calculation formula shown in Equation 1 tends to be easily affected by noise. For example, if there is noise in the non-signal parts of multiple tomographic image data and the values ​​are different from each other, the decorrelation value becomes high and noise is superimposed on the motion contrast image. To avoid this, the motion contrast generation unit can, as a preprocessing step, consider tomographic data below a predetermined threshold to be noise and replace it with zero. This allows the image generation unit 332 to generate a motion contrast image with reduced effects of noise based on the generated motion contrast data.

[0096] Furthermore, the motion contrast generating unit generates a two-dimensional motion contrast front image (also called an OCTA image) from the three-dimensional motion contrast data. The motion contrast generating unit projects the motion contrast data corresponding to the range between the upper end and the lower end of the generation range specified for the three-dimensional motion contrast data onto a two-dimensional plane to generate an OCTA image. Specifically, the motion contrast generating unit performs processing such as average value projection (AIP) or maximum value projection (MIP) on the motion contrast data within the range based on the motion contrast data corresponding to the range between the upper end and the lower end of the generation range among the entire motion contrast data, thereby generating an OCTA image, which is a front image of the motion contrast image. The generation method of the OCTA image is not limited to the average value or maximum value. It may be generated using values ​​such as the minimum value, median value, variance, standard deviation, and sum.

[0097] For example, to generate an OCTA image of the superficial retina, the upper end of the generation range is the ILM / NFL boundary line, and the lower end of the generation range is the boundary line 50 μm below the GCL / IPL in the depth direction.Then, the OCTA image is generated using maximum intensity projection.

[0098] In addition, when generating an OCTA image, several images with different depth ranges may be generated in advance, or an image corresponding to the upper and lower ends of the generation range specified in the user interface may be generated at the timing of displaying on the display unit.

[0099] Steps S1205 to S1208 are equivalent to steps S304 to S307 described in the first embodiment, and therefore a description thereof will be omitted.

[0100] <Step S1209> In step S1209, in addition to generating the analysis map described in step S308, the two-dimensional data generating unit 335 generates an analysis map from the OCTA image. The analysis map is subjected to blood vessel enhancement processing based on a Hessian filter and edge selection sharpening, binarization processing using two types of blood vessel enhancement images, and shaping processing to identify a blood vessel region. Although an example of detection using image processing has been described regarding blood vessel detection, this is not limiting. Blood vessels may be detected from an input image by learning images and correct labels using a machine learning method such as Deep Learning.

[0101] When analyzing OCTA images, vessel area density (VAD) maps based on the area of ​​the vessel region and vessel length density (VLD) maps based on the vessel length are generated. VAD is vessel density (unit: %) defined as the proportion of the vessel region included in the measurement target. VLD is vessel density defined as the sum of the lengths of the vessels included per unit area (unit: mm-1).

[0102] Vascular density is an index for quantifying the extent of vascular blockage and the degree of sparseness of the vascular network, and VAD is the most commonly used index. However, since the contribution of the large blood vessel area to the measured value is large with VAD, VLD is used (as an index more sensitive to capillary blockage) when focusing on the pathology of capillaries such as diabetic retinopathy.

[0103] Without being limited to this, for example, Fractal Dimension, which quantifies the complexity of the vascular structure, or Vessel Diameter Index, which indicates the distribution of vascular diameters (distribution of vascular aneurysms and stenosis), may be measured.

[0104] <Step S1210> In step S1210, the captured image, the layer thickness map, the analysis map, and the like are displayed on the display unit 600. An example of a screen displayed on the display unit 600 is shown in FIG.

[0105] FIG. 14 shows an example of displaying a motion contrast tomographic image and a motion contrast front image (OCTA image) based on the display example shown in FIG. 5(b).

[0106] Reference numeral 1401 denotes an OCTA image of the superficial retina, 1402 denotes an OCTA image of the deep retina, and 1403 denotes an OCTA image of the choriocapillaris. Although the figure shows the display of an OCTA image, it is also possible to display not only the OCTA image but also a map showing vascular density (VAD and VLD) in pseudocolor by setting the transparency on the OCTA image and superimposing it on the OCTA image using a user interface (e.g., a button, a right-click menu, tab switching, a check box, a long tap, a multi-tap, etc.) (not shown).

[0107] Reference numeral 1430 denotes a horizontal scan tomographic image, and region 1431 denotes an example in which data equal to or greater than a threshold value in motion contrast data generated from a plurality of horizontal scan tomographic images is superimposed on the tomographic image 1430. The superimposed display of the motion contrast data can be switched between display and non-display. Furthermore, execution of a suppression process for projection artifacts can also be specified. Although not shown, a vertical scan tomographic image captured in 3D may be displayed instead of the horizontal scan 1430, or a vertical scan tomographic image may be displayed alongside the horizontal scan 1430. In this case, it is desirable to superimpose and display the motion contrast data on the vertical scan as well.

[0108] 5(b), the thickness of the layer formed by combining the NFL, GCL, and IPL, the significance map, and the deviation map are not displayed, but it is possible to switch and display them instead of the map 512 representing the overall retinal thickness and the significance map for the overall retinal thickness. When switching between the overall retinal thickness and the thickness of the layer formed by combining the NFL, GCL, and IPL, the grid displaying the measurement values ​​is also switched from 513 to 526 accordingly.

[0109] Although not shown, OCTA images and blood vessel density map images can be displayed in time series in addition to the layer thickness map, significance map, and deviation map as exemplified in Figures 8 and 11. For example, in explanation using Figure 11, instead of the layer thickness map 1113 showing the thickness of the layers of the entire retina, an OCTA image 1401 or a blood vessel density map superimposed on the OCTA image may be displayed.

[0110] According to the configuration described above, it is possible to capture OCT images over a wide range, generate and display OCT angiograms over a wide range, perform processing appropriate to the area, and effectively present the analysis results.

[0111] (Fourth embodiment) In the first to third embodiments, a display form has been shown that allows the user to easily grasp the analysis results of the optic disc and macular region and the analysis results of OCTA by taking a wide range of OCT images. In this embodiment, an example will be described in which image quality improvement means is used to perform image quality improvement processing on at least one of a wide range of OCT images, OCTA images, and fundus images. Note that image quality improvement processing refers to noise removal, super-resolution, and contrast adjustment processing.

[0112] Fig. 15 is a diagram showing the configuration of an image processing system 1015 including an image processing device 3015 according to this embodiment. As shown in Fig. 15, the image processing device 3015 includes an image processing unit 1503, and the image processing unit 1503 includes an image quality improvement unit 1531. The image quality improvement unit 1531 improves the image quality by using a machine learning model.

[0113] In this embodiment, the improvement of image quality by noise removal will be described later. FIG. 16 shows an example of the configuration of a machine learning model in this embodiment. The configuration shown in FIG. 16 is composed of a plurality of layers that process and output a group of input values. As shown in FIG. 16, the types of layers included in the above configuration include a convolution layer, a downsampling layer, an upsampling layer, and a merging layer. The convolution layer is a layer that performs convolution processing on a group of input values ​​according to parameters such as the kernel size of the set filter, the number of filters, the stride value, and the dilation value. The number of dimensions of the kernel size of the filter may also be changed according to the number of dimensions of the input image. The downsampling layer is a process that reduces the number of output value groups to be less than the number of input value groups by thinning out or merging the input value groups. Specifically, there is, for example, a Max Pooling process. The upsampling layer is a process that increases the number of output value groups to be more than the number of input value groups by duplicating the input value groups or adding values ​​interpolated from the input value groups. Specifically, for example, there is a linear interpolation process. The synthesis layer is a layer that inputs a group of values, such as a group of output values ​​of a certain layer or a group of pixel values ​​constituting an image, from a plurality of sources and performs a process of synthesizing them by connecting or adding them. In such a configuration, the group of values ​​output from the group of pixel values ​​constituting the input image Im1610 through the convolution processing block and the group of pixel values ​​constituting the input image Im1610 are synthesized in the synthesis layer. After that, the synthesized group of pixel values ​​is formed into a high-quality image Im1620 in the last convolution layer. Although Im1610 and Im1620 in FIG. 16 show tomographic images, a motion contrast tomographic image, a motion contrast front image (OCTA image), an Enface image, an SLO image, and a front image of the retina such as a fundus photograph are also possible. The shooting range is not limited to the posterior part of the retina, and may be the anterior part.

[0114] Although not shown in the figure, as an example of a modification of the CNN configuration, for example, a batch normalization layer or an activation layer using a rectifier linear unit may be incorporated after the convolution layer.

[0115] The image quality improvement engine of this embodiment inputs a low quality image obtained by adding a first noise component to an image obtained from the image capture device 200 or another device, and performs training using an image obtained by adding a second noise component to an image obtained from the image capture device 200 or another device as output data as a high quality image. That is, the teacher image used during learning in this embodiment is a common image for the low quality image and the high quality image, and the noise components in each image are different. Since the same image is used, alignment is not required when pairing images.

[0116] Here, the learning unit (not shown) may include an error detection unit and an update unit. The error detection unit obtains an error between the output data output from the output layer of the neural network according to the input data input to the input layer and the teacher data. The error detection unit may use a loss function to calculate the error between the output data from the neural network and the teacher data. Furthermore, the update unit updates the connection weighting coefficients between the nodes of the neural network, etc., based on the error obtained by the error detection unit, so as to reduce the error. This update unit updates the connection weighting coefficients, etc., using, for example, an error backpropagation method. The error backpropagation method is a method of adjusting the connection weighting coefficients between the nodes of each neural network so as to reduce the above-mentioned error.

[0117] At this time, since the GPU can perform efficient calculations by processing more data in parallel, it is effective to use the GPU for processing when performing learning multiple times using a learning model such as deep learning. Therefore, in this embodiment, the GPU is used in addition to the CPU for processing by the learning unit. Specifically, when executing a learning program including a learning model, the CPU and GPU work together to perform calculations to perform learning. Note that the processing of the learning unit may be performed by only the CPU or the GPU. Also, the image quality improvement unit 1531 may use a GPU like the learning unit.

[0118] In addition, noise components such as Gaussian noise and modeled noise specific to the target image are added as noise. However, the first and second noises are different from each other. Different noises mean that the spatial locations (pixel positions) to which the noise is added are different, or the noise values ​​are different. For example, in the case of OCT, noise specific to the target image can be estimated based on data captured without placing a model eye or a test eye, and used as a noise model. In the case of OCTA, noise appearing in the avascular zone (FAZ) or noise appearing in an image captured of a model eye that reproduces blood flow in a schematic manner can be used as a noise model.

[0119] Here, in the case of Gaussian noise, the standard deviation or variance value is defined as the noise magnitude, and noise is randomly added to the image based on these values. The overall average value may be left unchanged as a result of adding random noise. In other words, the average value of the noise added to each pixel of one image is set to 0. Here, it is not necessary to make the average value 0, as long as noise of different patterns can be added to the input data and the output data.

[0120] In this embodiment, a high-quality image is generated using an image obtained by adding a first noise component and a second noise component different from the first noise component to a low-quality image obtained from the photographing device 200 or another device, but the configuration for performing these processes is not limited to this. For example, the image to which noise is added may be a high-quality image that has been subjected to an averaging process (overlay) after positioning, and the first and second noise components may be added to the image to be added. That is, a configuration may be used in which an image obtained by adding a first noise component to an image subjected to overlay processing is learned as a low-quality image, and an image obtained by adding a second noise component to an image subjected to overlay processing is learned as a high-quality image. Furthermore, in this embodiment, an example of learning using the first and second noise components has been described, but this is not limited to this. For example, a configuration may be used in which the first noise component is added only to the image to be used as a low-quality image, and learning is performed without adding a noise component to the image to be used as a high-quality image.

[0121] Regarding the high image quality processing, in the case of super-resolution processing, a low-resolution image and a high-resolution image may be learned as a pair of images. In the case of contrast adjustment processing, a low-contrast image and a high-contrast image are learned as a pair of images. The high-contrast image to be the correct answer does not need to be adjusted uniformly over the entire screen, but can be learned as an image with appropriate contrast adjustment locally and for each region. These high image quality processing processes can be executed not only once on one screen. By having multiple models that have been trained separately, it is possible to execute any high image quality processing, and one or multiple types of high image quality processing can be executed. The high image quality processing may be executed automatically when the screen transitions to the report screen, or the operator may be able to select the high image quality processing to be executed. In addition, when multiple high image quality processing processes are executed, it is desirable to display to the operator which high image quality processing is being executed (by changing the color of a button, displaying a message, displaying an icon, displaying a check box, etc.).

[0122] In the image display on the report screen, the operator can select whether or not to perform the image quality improvement process shown in this embodiment by using a user interface (e.g., a button, a right-click menu, a tab switching, a check box, a long tap, a multi-tap, etc.) not shown. For example, the case of performing image quality improvement process on a tomographic image, a motion contrast tomographic image, and a motion contrast front image (OCTA) will be described with reference to FIG. 14. In FIG. 14, by pressing an image quality improvement button not shown, image quality improvement process is performed on the tomographic image 1430, the motion contrast tomographic image 1431, and the OCTA images 1401 to 1403, and the high-image quality images are displayed. It is not necessary for the operator to select the image quality improvement process on the OCTA blood vessel density map for the high-image quality image, and the image quality improvement process may be automatically performed when the operator selects the examination data and transitions to the report screen.

[0123] Although not shown in the figures, image quality improvement processing can also be performed on the time-series screen as described in the first to third embodiments. For example, by performing image quality improvement processing on any of the time-series fundus photographs, OCT images, or OCTA images as described in the third embodiment, the time-series images can all be made into high-image quality images.

[0124] According to the configuration described above, it is possible to capture a wide range of OCT images and generate and display a wide range of OCT angiograms, making it possible to effectively present high-quality images and analysis results.

[0125] Fifth embodiment In the first to fourth embodiments, a display form that enables easy comprehension of the analysis results of the optic disc and the macular region and the analysis results of the OCTA by capturing a wide range of OCT images has been shown. In this embodiment, an example is shown in which data captured in detail of the optic disc and the macular region separately is displayed on one screen.

[0126] Here, the description will be given with reference to FIG. 5(a). In this embodiment, data obtained by separately photographing the optic disc and the macular region in detail is laid out on one screen as shown in FIG. 5(a). For example, in FIG. 5(a), the images and analysis data related to the optic disc of 511, 518, 519, 522, 523, 524, and 525 are obtained by acquiring information from data obtained by photographing the optic disc, and the images and analysis data related to the macular region other than the optic disc are obtained by acquiring information from data obtained by photographing the macular region. When separately photographed data is displayed on one screen, the operator may select and display the optic disc photographed data and the macular region photographed data by pressing a data selection button (not shown). Alternatively, if there is data obtained by photographing the optic disc and the macular region of the same eye on the same day selected by the operator, the data selection unit (not shown) may automatically select and display the data.

[0127] According to the configuration described above, detailed data of the optic disc and macular region can be viewed separately on a single screen, so that the analysis results can be presented effectively without having to switch data display.

[0128] (Variation 1) In this embodiment, a series of steps from shooting to display is shown, but this is not limiting. For example, analysis may be performed using data that has already been shot. In this case, processing related to shooting is skipped, and images that have already been shot are acquired instead. This makes it possible to display a report when necessary, without performing analysis processing during shooting. Therefore, when shooting, it is possible to concentrate only on shooting.

[0129] (Variation 2) In the present embodiment, the data around the optic disc and the data around the macula are stored separately as normative data, but this is not limiting. For example, if data of a wide range scan including the optic disc and the macula is stored as normative data, a comparison with that data may be performed.

[0130] (Other embodiments) The present invention can also be realized by executing the following process. That is, the present invention can also be realized by supplying software (programs) that realize one or more functions of the various embodiments and modifications described above to a system or device via a network or a storage medium, and having a computer (or a CPU, MPU, or the like) of the system or device read and execute the programs. The computer has one or more processors or circuits, and may include multiple separate computers or a network of multiple separate processors or circuits to read and execute computer-executable instructions.

[0131] In this case, the processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field programmable gateway (FPGA). The processor or circuitry may also include a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

Claims

1. an acquisition unit that acquires a first analysis map that is an analysis result of first three-dimensional data related to the optic disc obtained by photographing an area including the optic disc of the fundus of the test eye at a first time using optical coherence tomography, a second analysis map that is an analysis result of second three-dimensional data related to the macular region obtained by photographing an area including the macular region of the fundus of the test eye at the first time using optical coherence tomography, and a third analysis map related to the optic disc and a fourth analysis map related to the macular region that are analysis results of third three-dimensional data obtained by photographing an area including the optic disc and the macular region at a second time after the first time using optical coherence tomography; a display control unit that controls a display unit to display the first analysis map and the third analysis map in a chronological order in a first display area, and controls the display unit to display the second analysis map and the fourth analysis map in a chronological order in a second display area different from the first display area; an adjustment unit that adjusts a resolution of at least one of the first three-dimensional data, the second three-dimensional data, the three-dimensional data of the optic disc in the third three-dimensional data, and the three-dimensional data of the macular region in the third three-dimensional data based on a resolution of statistical information of a normal eye; An image processing device comprising:

2. The image processing device of claim 1, wherein the display control unit causes a display unit to display a two-dimensional image of an area including the optic disc and the macular region, the first analysis map and the third analysis map of the optic disc, and the second analysis map and the fourth analysis map of the macular region side by side.

3. the first analysis map regarding the optic disc is a first comparison map showing a comparison result between layer thickness information regarding the optic disc obtained by analyzing the first three-dimensional data and statistical information regarding the optic disc of a normal eye, The image processing device described in claim 2, wherein the second analysis map regarding the macular region is a second comparison map showing a comparison result between layer thickness information regarding the macular region obtained by analyzing the second three-dimensional data and statistical information of normal eyes regarding the macular region.

4. 4. The image processing device according to claim 3, wherein the two-dimensional image is a front image obtained using at least a portion of the third three-dimensional data in the depth direction of the test eye, or a layer thickness map obtained by superimposing layer thickness information obtained by analyzing at least a portion of the third three-dimensional data in the depth direction of the test eye as color information on the front image.

5. When either the layer corresponding to the layer thickness map or the layer corresponding to the second comparison map is switched to another layer in response to an instruction from an examiner, the other is switched to the other layer in conjunction with the other, whereby the display of the layer thickness map and the second comparison map is changed to an analysis map corresponding to the other layer, The image processing device according to claim 4 , configured so that the layer corresponding to the first analysis map relating to the optic disc is not changed even when the instruction is received.

6. The image processing device according to any one of claims 2 to 5, wherein the display control unit causes information indicating the position of the first analysis map relative to the optic disc and information indicating the position of the second analysis map relative to the macular region to be superimposed on the two-dimensional image and displayed on the display unit.

7. 7. The image processing device according to claim 1, wherein the adjustment unit adjusts the number of A scans and B scans in the third three-dimensional data by adjusting the position and size of a portion of the third three-dimensional data for comparison with statistical information of normal eyes, based on the position of the optic disc or the macular region in the third three-dimensional data.

8. The image processing device according to claim 1 , wherein the third three-dimensional data is a single piece of three-dimensional data obtained by photographing a region including the optic disc and the macular region.

9. 9. The image processing device according to claim 1, wherein the display control unit causes the display unit to display information indicating that the first analysis map, the second analysis map, the third analysis map, and the fourth analysis map are identifiable as analysis results of three-dimensional data obtained by photographing in different shooting modes.

10. an ophthalmologic imaging apparatus including a detection unit that detects interference light between a return light from the fundus irradiated with a measurement light and a reference light, and the optical coherence tomography; An image processing device according to any one of claims 1 to 9; A system comprising:

11. acquiring a first analysis map which is an analysis result of first three-dimensional data relating to the optic disc obtained by photographing an area including the optic disc of the fundus of the test eye at a first time using optical coherence tomography, a second analysis map which is an analysis result of second three-dimensional data relating to the macular region obtained by photographing an area including the macular region of the fundus of the test eye at the first time using optical coherence tomography, and a third analysis map relating to the optic disc and a fourth analysis map relating to the macular region which are analysis results of third three-dimensional data obtained by photographing an area including the optic disc and the macular region at a second time after the first time using optical coherence tomography; controlling a display unit to display the first analysis map and the third analysis map in a chronological order in a first display area, and controlling the display unit to display the second analysis map and the fourth analysis map in a chronological order in a second display area different from the first display area; adjusting a resolution of at least one of the first three-dimensional data, the second three-dimensional data, the three-dimensional data of the optic disc in the third three-dimensional data, and the three-dimensional data of the macular region in the third three-dimensional data based on a resolution of statistical information of a normal eye; An image processing method comprising:

12. A program for causing a computer to execute each step of the image processing method according to claim 11.

13. an acquisition unit that acquires a first analysis map for the optic disc and a second analysis map for the macular region, which are analysis results of first three-dimensional data obtained by imaging an area including the optic disc and the macular region using optical coherence tomography at a first time, and a third analysis map for the optic disc and a fourth analysis map for the macular region, which are analysis results of second three-dimensional data obtained by imaging an area including the optic disc and the macular region using optical coherence tomography at a second time after the first time; a display control unit that controls a display unit to display the first analysis map and the third analysis map in a chronological order in a first display area, and controls the display unit to display the second analysis map and the fourth analysis map in a chronological order in a second display area different from the first display area; an adjustment unit that adjusts a resolution of at least one of the three-dimensional data of the optic disc in the first three-dimensional data, the three-dimensional data of the macular region in the first three-dimensional data, the three-dimensional data of the optic disc in the second three-dimensional data, and the three-dimensional data of the macular region in the second three-dimensional data, based on a resolution of statistical information of a normal eye; An image processing device comprising:

Images