Ophthalmologic apparatus, method for processing ophthalmologic image, and method for controlling ophthalmologic apparatus

Abstract

To solve a problem caused by an image filter used for blood vessel enhancement.SOLUTION: An ophthalmologic apparatus according to some aspects includes an image acquisition unit, an image projection processing unit, a blood vessel enhancement processing unit, a noise removal processing unit, and an image synthesis processing unit. The image acquisition unit acquires an optical coherence tomography angiographic image of a fundus of a subject's eye. The image projection processing unit applies projection processing to the optical coherence tomography angiographic image to generate a projection image. The blood vessel enhancement processing unit generates a blood vessel enhanced image by applying a blood vessel enhancement filter for enhancing a blood vessel image to the projection image. The noise removal processing unit generates a noise-removed image by applying noise removal processing to the projection image. The image composition processing unit generates a composite image by applying image composition processing to the projection image, the blood vessel enhanced image, and the noise removed image.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to an ophthalmic device, a method for processing ophthalmic images, and a method for controlling an ophthalmic device. [Background technology]

[0002] Optical coherence tomography (OCT) is one of the imaging modalities used in ophthalmology. OCT can be used for both structural and functional imaging, and is one of the modalities that has attracted the most attention in recent years.

[0003] OCT angiography (OCTA) is one of the functional imaging techniques using OCT. OCT angiography is a functional imaging modality for depicting blood flow and is typically used to obtain fundus vascular images (retinal vascular images, choroidal vascular images, etc.) (see, for example, Patent Documents 1 and 2). OCT angiography is an imaging technique that focuses on the fact that signals from fundus tissues (structures) do not change over time, while signals from blood flow inside blood vessels change over time. A vascular image is constructed by emphasizing areas where such temporal changes exist (blood flow signals). OCT angiography is also called OCT motion contrast imaging. Images constructed by OCT angiography are called (OCT) angiography images, (OCT) angiograms, motion contrast images, etc.

[0004] The techniques described in Patent Documents 1 and 2 use a multiscale Frangi filter to enhance blood vessels. Details of this filter are described in, for example, Non-Patent Document 1. Briefly, the Frangi filter (also called the Frangi filter) is a filter for extracting and enhancing linear and tubular structures using two eigenvalues ​​of a Hessian matrix whose elements are second-order derivatives with respect to each direction of the image, and evaluates the likelihood of each pixel being a blood vessel (i.e., the proportion of that pixel that is a blood vessel). The multiscale Frangi filter is a method that uses multiple different scales to extract structures of various dimensions. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] US Patent Application Publication No. 2022 / 0151568 [Patent Document 2] U.S. Patent No. 10,136,812 [Non-patent literature]

[0006] [Non-Patent Document 1] Alejandro F. Frangi, Wiro J. Niessen, Koen L. Vincken & Max A. Viergever. Multiscale vessel enhancement filtering. In International Conference on Medical Image Computing and Computer-Assisted Intervention - MICCAI'98: First International Conference, Cambridge, MA, USA, October 11-13, 1998, Proceedings (Lecture Notes in Computer Science, 1496), pp. 130-137, Springer Berlin Heidelberg. Summary of the Invention [Problem to be solved by the invention]

[0007] Multi-scale Frangi filters are used in a variety of fields, and are widely used in the medical field to enhance vascular and neural images. However, the inventors of the present disclosure have discovered that there are cases in which some problems arise in images obtained by applying a multi-scale Frangi filter.

[0008] For example, when a multiscale frangi filter is applied to angiographic images generated by OCT angiography, brightness variations may occur in images of relatively large blood vessels, images of relatively small blood vessels may be missing, and blood vessel-like noise may appear in images of the foveal avascular zone (FAZ).

[0009] The problem of uneven brightness in images of large blood vessels is that the brightness in the vicinity of the center line (axis) of the large blood vessel decreases, causing the blood vessel to appear dark.

[0010] The lack of images of thin blood vessels is a problem in that the visibility of thin blood vessels is relatively reduced due to the emphasis on thick blood vessels.

[0011] The problem with noise in the foveal avascular zone is that not only the image of blood vessels but also the noise is emphasized, resulting in noise that appears to be blood vessels in the foveal avascular zone, where there should be no signal (image) corresponding to blood vessels.

[0012] It has also been confirmed that similar problems can occur when using filters other than the multiscale Frange filter, such as the Gabor filter, non-local means filter, and wavelet filter.

[0013] One objective of the present disclosure is to solve the problems caused by image filters used for vessel enhancement.

[0014] An objective of some aspects of the present disclosure is to solve at least one of the following problems: uneven brightness in images of large blood vessels; missing images of small blood vessels; and noise in the foveal avascular zone. [Means for solving the problem]

[0015] An ophthalmologic apparatus according to some exemplary aspects of the present disclosure includes an image acquisition unit that acquires an optical coherence tomography angiography image of the fundus of a subject's eye, an image projection processing unit that applies a projection process to the optical coherence tomography angiography image to generate a projection image, a vessel enhancement processing unit that applies a vessel enhancement filter for enhancing a blood vessel image to the projection image to generate a vessel enhancement image, a noise reduction processing unit that applies a noise reduction process to the projection image to generate a noise-removed image, and an image synthesis processing unit that applies an image synthesis process to the projection image, the vessel enhancement image, and the noise-removed image to generate a synthesis image.

[0016] A method according to some exemplary aspects of the present disclosure is a method for processing an optical coherence tomography angiography image of the fundus of a test eye by a computer including a processor, a storage device, and a data input interface, the method including: an input processing step of inputting the optical coherence tomography angiography image to the computer via the data input interface; a storage processing step of storing the input optical coherence tomography angiography image by the storage device; an image projection processing step of applying a projection processing to the optical coherence tomography angiography image stored in the storage device to generate a projection image by the processor; a vessel enhancement processing step of applying a vessel enhancement filter for enhancing a blood vessel image to the projection image to generate a vessel enhancement image by the processor; a noise reduction processing step of applying a noise reduction processing to the projection image to generate a noise-removed image by the processor; and an image synthesis processing step of applying an image synthesis processing to the projection image, the vessel enhancement image, and the noise-removed image to generate a synthesis image by the processor.

[0017] A method according to some exemplary aspects of the present disclosure is a method for controlling an ophthalmic apparatus including a processor, a storage device, and an image acquisition device, the method including: an image acquisition control step of causing the image acquisition device to acquire the optical coherence tomography angiography image; a storage control step of causing the storage device to store the acquired optical coherence tomography angiography image; an image projection control step of causing the processor to apply a projection process to the optical coherence tomography angiography image stored in the storage device to generate a projection image; a vessel enhancement control step of causing the processor to apply a vessel enhancement filter for enhancing a blood vessel image to the projection image to generate a vessel enhancement image; a noise reduction control step of causing the processor to apply a noise reduction process to the projection image to generate a noise-removed image; and an image synthesis control step of causing the processor to apply an image synthesis process to the projection image, the vessel enhancement image, and the noise-removed image to generate a composite image.

[0018] A program according to some exemplary aspects of the present disclosure is a program for causing a computer to execute each step of a method according to any exemplary aspect of the present disclosure.

[0019] A recording medium according to some exemplary aspects of the present disclosure is a computer-readable non-transitory recording medium on which a program according to any exemplary aspect of the present disclosure is recorded. [Effects of the Invention]

[0020] According to some exemplary aspects of the present disclosure, it is possible to solve the problems caused by image filters used for vessel enhancement. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a schematic diagram of a configuration of an ophthalmic apparatus according to a non-limiting embodiment. [Figure 2] 1 is a schematic diagram of a configuration of an ophthalmic apparatus according to a non-limiting embodiment. [Figure 3] 1 is a schematic diagram of a configuration of an ophthalmic apparatus according to a non-limiting embodiment. [Figure 4] 1 is a schematic diagram of a configuration of an ophthalmic apparatus according to a non-limiting embodiment. [Figure 5A] 10 is a flowchart of the operation of an ophthalmologic apparatus according to a non-limiting embodiment. [Figure 5B] 1 is a schematic diagram of the operation of an ophthalmic device according to a non-limiting embodiment. [Figure 6] 10 is a comparative example with the operation of an ophthalmic apparatus according to a non-limiting embodiment. [Figure 7] 10 is a flowchart of the operation of an ophthalmologic apparatus according to a non-limiting embodiment. [Figure 8A] 10 is a flowchart of the operation of an ophthalmologic apparatus according to a non-limiting embodiment. [Figure 8B] 1 is a schematic diagram of the operation of an ophthalmic device according to a non-limiting embodiment. [Figure 9] 10 is a flowchart of the operation of an ophthalmologic apparatus according to a non-limiting embodiment. [Figure 10A] 10 is a flowchart of the operation of an ophthalmologic apparatus according to a non-limiting embodiment. [Figure 10B] 1 is a schematic diagram of the operation of an ophthalmic device according to a non-limiting embodiment. [Figure 11A] 10 is a flowchart of the operation of an ophthalmologic apparatus according to a non-limiting embodiment. [Figure 11B] 1 is a schematic diagram of the operation of an ophthalmic device according to a non-limiting embodiment. [Figure 12] 10 is a flowchart of the operation of an ophthalmologic apparatus according to a non-limiting embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0022] Several exemplary aspects of embodiments according to the present disclosure will be described. In the present disclosure, several exemplary aspects will be described for each of an embodiment of an ophthalmic device (e.g., an ophthalmic imaging device, an ophthalmic image processing device, etc.), an embodiment of a method for processing an ophthalmic image, an embodiment of a method for controlling an ophthalmic device, an embodiment of a program, and an embodiment of a recording medium. Each exemplary aspect provides a non-limiting embodiment.

[0023] Embodiments of the present disclosure can be employed to solve problems caused by image filters used for vascular enhancement. While various problems arise from image filters used for vascular enhancement, the present disclosure focuses in particular on the following three problems. It will be understood by those skilled in the art that the problems that can be solved by the technology of the present disclosure are not limited to these.

[0024] As a premise for understanding the problem focused on in this disclosure, we will explain the Frangi filter, which is a representative example of an image filter used for vessel enhancement. The Frangi filter in this disclosure is an image filter configured to detect vascular structures in an eye image using two eigenvalues ​​of a Hessian matrix. In a multi-scale Frangi filter, multiple different scales are used to extract blood vessels of various dimensions (thicknesses).

[0025] The inventors of the present disclosure have investigated various images obtained by applying a multiscale Frangi filter to a large number of OCT angiography images and have found that various problems arise. In particular, they have found that the following three problems are relatively prominent in terms of frequency and severity:

[0026] The first problem that may occur in an image obtained by applying a multiscale Frangi filter to an OCT angiography image (herein referred to as a vessel-enhanced image) is that brightness unevenness occurs in the image of relatively large blood vessels among the images of blood vessels of various diameters depicted in the vessel-enhanced image. More specifically, the first problem is that the brightness of the region near the center line in the image of a relatively large blood vessel is displayed as low, that is, the region near the center line is displayed as dark.

[0027] The second problem is that images of relatively thin blood vessels, which should be clearly depicted in the vessel-enhanced image, are missing. This second problem is thought to be caused by the fact that the multiscale Franzi filter enhances images of relatively thick blood vessels, which reduces the visibility of images of relatively thin blood vessels.

[0028] The third problem is that in a vessel-enhanced image of an area that includes avascular regions of the fundus (e.g., the foveal avascular region), noise that looks like blood vessels appears within the image of the avascular region. This third problem is thought to be caused by the fact that the multiscale Franzi filter enhances not only the images of blood vessels but also the noise, causing the noise in the avascular region to appear as blood vessels.

[0029] Each exemplary aspect of the present disclosure aims to solve at least one of these three problems. Some exemplary aspects may also solve other problems. Some non-limiting example aspects of embodiments according to the present disclosure are listed below.

[0030] A first embodiment is an ophthalmologic apparatus including an image acquisition unit, an image projection processing unit, a blood vessel enhancement processing unit, a noise reduction processing unit, and an image synthesis processing unit. The image acquisition unit is configured to acquire an OCT angiography image of the fundus of the subject's eye. The image projection processing unit is configured to apply a projection process to the OCT angiography image to generate a projection image. The blood vessel enhancement processing unit is configured to apply a blood vessel enhancement filter for enhancing the blood vessel image to the projection image to generate a blood vessel enhancement image. The noise reduction processing unit is configured to apply a noise reduction process (a process for removing or reducing noise) to the projection image to generate a noise-removed image. The image synthesis processing unit is configured to apply an image synthesis process to the projection image, the blood vessel enhancement image, and the noise-removed image to generate a synthesized image.

[0031] A second embodiment is the ophthalmic device of the first embodiment, wherein the vessel emphasis filter includes a multi-scale frangi filter.

[0032] The vessel enhancement filter is not limited to a multi-scale Frangi filter, but may be any type of image filter that can be used for processing focusing on blood vessel images (for example, blood vessel detection, enhancement, etc.). Furthermore, the vessel enhancement filter may be a single image filter or a combination of two or more image filters.

[0033] A third embodiment is an ophthalmologic apparatus according to the first or second embodiment, wherein the projection processing may include at least one of maximum intensity projection and average intensity projection. Maximum intensity projection is excellent for depicting blood vessels, and is basically used in the projection processing of OCT angiography. However, maximum intensity projection has the disadvantage of being prone to noise contamination. Average intensity projection is one of the projection methods that is less susceptible to noise contamination.

[0034] The projection process is not limited to maximum intensity projection and average intensity projection, and may be any type of projection process. Furthermore, the projection process may be a single projection process or a combination of two or more projection processes.

[0035] A fourth embodiment is an ophthalmologic device according to any one of the first to third embodiments, wherein the image projection processing unit is configured to generate a projection image by applying a projection process to an image area in the OCT angiography image that corresponds to a predetermined layer tissue of the fundus.

[0036] The layer tissue of the fundus to which the projection process is applied may be, for example, the retina, one or more sub-tissues of the retina, the choroid, one or more sub-tissues of the choroid, the sclera, etc. The layer tissue of the fundus to which the projection process is applied may be identified and extracted using, for example, any segmentation process. The segmentation process used may be determined, selected, or configured depending on, for example, the type of layer tissue to be identified, the type of projection process to be applied, the type and state of the OCT angiography image, etc.

[0037] A fifth embodiment is the ophthalmologic apparatus of any one of the first to fourth embodiments, wherein the noise removal processing unit is configured to be able to perform a plurality of processes different from each other in the noise removal process.

[0038] The noise removal processing unit may be configured to select and execute one or more processes from the plurality of processes, or may be configured to execute all of the plurality of processes.

[0039] A sixth embodiment is the ophthalmologic apparatus of the fifth embodiment, wherein the noise reduction processing unit is configured to select at least one process from a plurality of processes based on the angle of view of the OCT angiography image. Furthermore, the noise reduction processing unit is configured to generate a noise-removed image by applying the selected at least one process to the projection image. Additionally, the image synthesis processing unit is configured to generate a synthesized image by applying an image synthesis process to the noise-removed image generated by applying the selected at least one process to the projection image, the projection image, and the blood vessel-enhanced image.

[0040] A seventh embodiment is the ophthalmologic apparatus of the fifth embodiment, wherein the noise reduction processing unit is configured to select at least one process from a plurality of processes based on at least one of a fixation position and a scan area for generating an OCT angiography image. The noise reduction processing unit is further configured to generate a noise-removed image by applying the selected at least one process to a projection image. Additionally, the image synthesis processing unit is configured to generate a composite image by applying an image synthesis process to the noise-removed image generated by applying the selected at least one process to the projection image, the projection image, and the vascular enhancement image.

[0041] An eighth embodiment is the ophthalmologic apparatus of any of the first to seventh embodiments, wherein the image synthesis process includes alpha blending, which is a processing technique for synthesizing two or more images at a specific ratio.

[0042] The image synthesis process is not limited to alpha blending, and may be any type of image synthesis process. Furthermore, the image synthesis process may be a single image synthesis process or a combination of two or more image synthesis processes.

[0043] A ninth embodiment is the ophthalmologic apparatus of any one of the first to eighth embodiments, wherein the image acquisition unit includes a scanning unit and an image construction unit. The scanning unit is configured to collect data by applying an OCT scan to the fundus. The image construction unit is configured to construct an OCT angiography image based on the data collected by the scanning unit.

[0044] A tenth aspect is the ophthalmologic apparatus of any one of the first to ninth aspects, wherein the image acquisition unit includes an image reception unit that receives an OCT angiography image from outside.

[0045] The OCT angiography image received by the image receiving unit may be an image generated by the ophthalmic apparatus of this embodiment, or may be an image generated by another ophthalmic apparatus.

[0046] An eleventh embodiment is the ophthalmologic apparatus of any one of the first to tenth embodiments, wherein the image acquisition unit includes a data acceptance unit and an image construction unit. The data acceptance unit is configured to accept data collected by applying an OCT scan to the fundus. The image construction unit is configured to construct an OCT angiography image based on the data accepted by the data acceptance unit.

[0047] The data received by the data receiving unit may be data generated by the ophthalmologic apparatus of this aspect, or may be data generated by another ophthalmologic apparatus.

[0048] The first to eleventh example embodiments can be used to address various problems caused by image filters used for vascular enhancement, including the first problem (uneven brightness in images of large blood vessels), the second problem (missing images of small blood vessels), and the third problem (noise in images of avascular regions) that are the focus of the present disclosure.

[0049] A twelfth aspect of the present invention is the ophthalmologic apparatus of any one of the first to eleventh aspects, wherein the noise removal processor is configured to apply a blood vessel image extraction process to the projection image to extract blood vessel images having diameters within a first range. The noise removal processor is further configured to apply an erosion process to the image generated by the blood vessel image extraction process to generate an eroded image in which a reduced blood vessel image is depicted, the diameters of which are reduced from those of the blood vessel images extracted by the blood vessel image extraction process. The blood vessel enhancement processor is further configured to apply a multi-scale Frangi filter to the projection image to generate a first blood vessel enhancement image. The blood vessel enhancement processor is further configured to apply a Frangi filter having a scale corresponding to a second range less than the first range to the projection image, and to apply gamma correction to the image generated by the Frangi filter to increase the brightness of the blood vessel image, thereby generating a second blood vessel enhancement image. The noise removal processor is further configured to generate a noise-removed image based on the eroded image, the first blood vessel enhancement image, and the second blood vessel enhancement image. In addition, the image synthesis processing unit is configured to generate a synthesized image by applying image synthesis processing to a noise-removed image generated from the eroded image, the first vascular enhancement image, and the second vascular enhancement image, the projection image, and the first vascular enhancement image.

[0050] A thirteenth embodiment is the ophthalmologic apparatus of the twelfth embodiment, wherein the noise removal processor is configured to perform the following series of processes. First, the noise removal processor identifies a first partial image of the first vessel-enhanced image corresponding to a reduced blood vessel image of the erode image. Second, the noise removal processor identifies a second partial image of the second vessel-enhanced image corresponding to a reduced blood vessel image of the erode image. Third, the noise removal processor generates a noise-removed image by selecting the larger luminance value of each pixel of the first partial image and the luminance value of a corresponding pixel of the second partial image. Furthermore, the image synthesis processor is configured to generate a synthesized image by applying image synthesis processing to the noise-removed image generated from the first and second partial images, the projection image, and the first vessel-enhanced image.

[0051] A fourteenth embodiment is the ophthalmologic apparatus of the thirteenth embodiment, wherein the noise removal processor is configured to determine a first partial image by applying mask processing based on a reduced blood vessel image in the eroded image to the first blood vessel-enhanced image, and further configured to determine a second partial image by applying the mask processing to the second blood vessel-enhanced image.

[0052] A fifteenth aspect is the ophthalmologic apparatus according to any one of the twelfth to fourteenth aspects, wherein the projection processing for generating the projection image to which the blood vessel image extraction processing is applied includes maximum intensity projection.

[0053] A sixteenth aspect is the ophthalmologic apparatus according to any one of the twelfth to fifteenth aspects, wherein the blood vessel image extraction process includes Otsu's binarization.

[0054] A seventeenth aspect of the present invention is the ophthalmologic apparatus of any one of the first to sixteenth aspects, wherein the noise removal processing unit is configured to perform the following series of processes. First, the noise removal processing unit applies a blood vessel image extraction process to the projection image to extract blood vessel images having diameters within a first range. Second, the noise removal processing unit analyzes the blood vessel images extracted by the blood vessel image extraction process to determine a center line of the blood vessel images. Third, the noise removal processing unit calculates a luminance distribution with respect to the distance from the center line. Fourth, the noise removal processing unit processes the image generated by the blood vessel image extraction process based on the luminance distribution to generate a processed image in which a reduced blood vessel image is depicted by reducing the diameter of the blood vessel image extracted by the blood vessel image extraction process. Furthermore, the blood vessel enhancement processing unit is configured to apply a multi-scale Frangi filter to the projection image to generate a first blood vessel enhancement image. Furthermore, the blood vessel enhancement processing unit is configured to apply a Frangi filter having a scale corresponding to a second range smaller than the first range to the projection image, and to generate a second blood vessel enhancement image by applying gamma correction to the image generated by the Frangi filter to enhance the luminance of the blood vessel images. The noise removal processing unit is configured to generate a noise-removed image based on the processed image, the first vascular enhancement image, and the second vascular enhancement image. Additionally, the image synthesis processing unit is configured to generate a synthesized image by applying image synthesis processing to the noise-removed image generated from the processed image, the first vascular enhancement image, and the second vascular enhancement image, the projection image, and the first vascular enhancement image.

[0055] An eighteenth embodiment is the ophthalmologic apparatus of the seventeenth embodiment, wherein the noise removal processing unit is configured to perform the following series of processes. First, the noise removal processing unit identifies a first partial image of the first vessel-enhanced image corresponding to the reduced vessel image of the processed image. Second, the noise removal processing unit identifies a second partial image of the second vessel-enhanced image corresponding to the reduced vessel image of the processed image. Third, the noise removal processing unit generates a noise-removed image by selecting the larger luminance value of each pixel of the first partial image and the luminance value of the corresponding pixel of the second partial image. Furthermore, the image synthesis processing unit is configured to generate a synthesized image by applying image synthesis processing to the noise-removed image generated from the first and second partial images, the projection image, and the first vessel-enhanced image.

[0056] A 19th example embodiment is the ophthalmologic apparatus of the 18th example embodiment, wherein the noise removal processing unit is configured to determine a first partial image by applying mask processing based on a reduced blood vessel image in the processed image to the first blood vessel emphasis image, and further configured to determine a second partial image by applying the mask processing to the second blood vessel emphasis image.

[0057] A twentieth embodiment is the ophthalmologic apparatus of any one of the seventeenth to nineteenth embodiments, wherein the projection processing for generating the projection image to which the blood vessel image extraction processing is applied includes maximum intensity projection.

[0058] A twenty-first embodiment is the ophthalmologic apparatus according to any one of the seventeenth to twentieth embodiments, wherein the blood vessel image extraction process includes Otsu's binarization.

[0059] The 12th to 21st embodiments can be used primarily to address the first problem (uneven brightness in images of large blood vessels), but can also be used to address other problems caused by image filters used to enhance blood vessels.

[0060] A 22nd example embodiment is the ophthalmologic apparatus of any one of the first to 21st examples, wherein the image projection processing unit is configured to apply a first projection processing to the OCT angiography image to generate a first projection image. The image projection processing unit is further configured to apply a second projection processing different from the first projection processing to the OCT angiography image to generate a second projection image. The vascular enhancement processing unit is further configured to apply a multiscale Frangi filter to the first projection image to generate a first vascular enhancement image. The vascular enhancement processing unit is further configured to apply a Frangi filter with a scale corresponding to the range of diameter dimensions of capillaries to the second projection image, and to apply gamma correction to the image generated by the Frangi filter to increase the brightness of the blood vessel image, thereby generating a second vascular enhancement image as a noise-removed image. Additionally, the image synthesis processing unit is configured to apply image synthesis processing to the first projection image, the first vascular enhancement image, and the second vascular enhancement image to generate a synthesized image.

[0061] A 23rd embodiment is the ophthalmic apparatus of the 22nd embodiment, wherein the first projection processing is maximum value projection, and the second projection processing is average value projection.

[0062] A 24th embodiment is the ophthalmic device of the 22nd or 23rd embodiment, wherein the OCT angiography image is an image depicting a region of the fundus including radial peripapillary capillaries (RPC).

[0063] The 22nd to 24th embodiments can be used primarily to address the second problem (missing images of thin blood vessels), but can also be used to address other problems caused by image filters used for vascular enhancement.

[0064] A 25th embodiment is the ophthalmologic apparatus of any one of the first to 24th embodiments, wherein the vascular enhancement processing unit is configured to apply a multiscale Franzi filter to the projection image to generate a vascular enhancement image. The noise removal processing unit is configured to apply avascular region identification processing to the vascular enhancement image to identify an avascular region image corresponding to an avascular region of the fundus. The noise removal processing unit is configured to generate a noise-removed image by applying mask processing based on the avascular region image identified by the avascular region identification processing to the vascular enhancement image. The image synthesis processing unit is configured to generate a composite image by applying image synthesis processing to the projection image and the noise-removed image generated by applying mask processing based on the avascular region image to the vascular enhancement image.

[0065] The image synthesis of the first embodiment (and embodiments citing it) is a process of synthesizing three images: a projection image, a vascular enhancement image, and a noise-removed image. On the other hand, the image synthesis of the 25th embodiment is a process of synthesizing two images: a projection image and a noise-removed image obtained from the vascular enhancement image. Here, the noise-removed image of the 25th embodiment can be considered as an image obtained by integrating the vascular enhancement image and the noise-removed image, so the image synthesis of the 25th embodiment corresponds to a specific embodiment of the image synthesis of the first embodiment.

[0066] A 26th embodiment is an ophthalmologic device according to the 25th embodiment, wherein the noise removal processing unit is configured to perform a first filter process in which a dispersion filter is applied to the vascular enhancement image during the avascular region identification process.

[0067] A 27th example embodiment is an ophthalmic device according to the 26th example embodiment, wherein the noise removal processing unit is configured to perform, in the avascular region identification processing, a first brightness threshold determination processing that determines a first brightness threshold based on a dispersion filter image generated by a first filter processing, and a first threshold processing that applies threshold processing using the first brightness threshold to the dispersion filter image to generate a first mask image.

[0068] A 28th embodiment is an ophthalmic device according to the 26th embodiment, wherein the noise removal processing unit is configured to perform a second filter process in the avascular region identification process, which applies an average filter to the projection image.

[0069] A 29th embodiment is an ophthalmic device according to the 27th embodiment, wherein the noise removal processing unit is configured to perform a second filter process in the avascular region identification process, which applies an average filter to the projection image.

[0070] A 30th example embodiment is an ophthalmic device according to the 29th example embodiment, wherein the noise removal processing unit is configured to perform, in the avascular region identification processing, a second brightness threshold determination processing that determines a second brightness threshold based on an average filter image generated by the second filter processing, and a second threshold processing that applies threshold processing using the second brightness threshold to the average filter image to generate a second mask image.

[0071] A 31st embodiment is an ophthalmologic device according to the 30th embodiment, wherein the noise removal processing unit is configured to perform, in the avascular region identification process, a process of combining the first mask image and the second mask image to generate a composite mask image, and a process of generating an avascular region image based on the composite mask image.

[0072] A 32nd embodiment is an ophthalmic device according to the 31st embodiment, wherein the noise removal processing unit is configured to generate a sum image of the first mask image and the second mask image as a composite mask image in the avascular region identification process.

[0073] A 33rd embodiment is an ophthalmologic device according to any one of the 25th to 32nd embodiments, wherein the noise removal processing unit is configured to generate an avascular region image using a Gaussian filter in the avascular region identification process.

[0074] A 34th embodiment is an ophthalmic device according to the 31st embodiment, wherein the noise removal processing unit is configured to generate an avascular region image by applying a Gaussian filter to the composite mask image in the avascular region identification process.

[0075] A 35th embodiment is an ophthalmic device according to the 32nd embodiment, wherein the noise removal processing unit is configured to generate an avascular region image by applying a Gaussian filter to the sum image in the avascular region identification process.

[0076] A 36th example embodiment is the ophthalmologic apparatus of any one of the first to 35th examples, wherein the vascular enhancement processing unit is configured to apply a multiscale Franzi filter to the projection image to generate a vascular enhancement image. The noise removal processing unit is configured to apply a high-density vascular region identification process to the projection image or the vascular enhancement image to identify a high-density vascular region image corresponding to a high-density vascular region of the fundus. The noise removal processing unit is further configured to generate a noise-removed image by applying a mask process based on the high-density vascular region image identified by the high-density vascular region identification process to the vascular enhancement image. Additionally, the image synthesis processing unit is configured to generate a composite image by applying an image synthesis process to the projection image and the noise-removed image generated by applying the mask process based on the high-density vascular region image to the vascular enhancement image.

[0077] A 37th example embodiment is the ophthalmologic apparatus of the 36th example embodiment, wherein the noise removal processing unit is configured to generate a mask image by expanding the high-density blood vessel region image identified by the high-density blood vessel region identification processing, and further configured to generate the noise-removed image by applying mask processing using the mask image to the blood vessel emphasis image.

[0078] The 25th to 37th embodiments can be used primarily to address the third problem (noise in images of avascular regions), but can also be used to address other problems caused by image filters used for vascular enhancement.

[0079] A thirty-eighth example embodiment is a method for processing ophthalmologic images, more specifically, a method for processing OCT angiography images of the fundus of a test eye by a computer. The computer includes a processor, a storage device, and a data input interface. The method of this example embodiment includes an input processing step, a storage processing step, an image projection processing step, a vascular enhancement processing step, a noise reduction processing step, and an image synthesis processing step. The input processing step inputs the OCT angiography image to the computer via the data input interface. The storage processing step stores the input OCT angiography image in the storage device. The image projection processing step involves the processor applying a projection processing to the OCT angiography image stored in the storage device to generate a projection image. The vascular enhancement processing step involves the processor applying a vascular enhancement filter for enhancing the vascular image to the projection image to generate a vascular enhancement image. The noise reduction processing step involves the processor applying a noise reduction processing to the projection image to generate a noise-removed image. The image synthesis processing step involves the processor applying an image synthesis processing to the projection image, the vascular enhancement image, and the noise-removed image to generate a synthesized image.

[0080] It is possible to at least partially combine steps corresponding to features (configuration, operation, processing, etc.) according to any of the second to thirty-sixth embodiments with the method of the thirty-eighth embodiment.

[0081] A thirty-ninth example embodiment is a method for controlling an ophthalmic device. The ophthalmic device includes a processor, a storage device, and an image acquisition device. The method of this example embodiment includes an image acquisition control step, a storage control step, an image projection control step, a vascular enhancement control step, a noise reduction control step, and an image synthesis control step. The image acquisition control step causes the image acquisition device to acquire an OCT angiography image. The storage control step causes the acquired OCT angiography image to be stored in the storage device. The image projection control step causes the processor to apply a projection process to the OCT angiography image stored in the storage device to generate a projection image. The vascular enhancement control step causes the processor to apply a vascular enhancement filter for enhancing a vascular image to the projection image to generate a vascular enhancement image. The noise reduction control step causes the processor to apply a noise reduction process to the projection image to generate a noise-removed image. The image synthesis control step causes the processor to apply an image synthesis process to the projection image, the vascular enhancement image, and the noise-removed image to generate a composite image.

[0082] It is possible to at least partially combine steps corresponding to features (configuration, operation, processing, etc.) according to any of the second to thirty-sixth embodiments with the method of the thirty-ninth embodiment.

[0083] A fortieth example embodiment is a program for causing a computer to execute each step of the method of the thirty-eighth or thirty-ninth example embodiment.

[0084] It is possible to combine the program of the 40th example embodiment with program elements that cause a computer to at least partially execute steps corresponding to the features (configuration, operation, processing, etc.) of any of the 2nd to 36th example embodiments.

[0085] A forty-first embodiment is a computer-readable non-transitory recording medium on which the program of the fortieth embodiment is recorded.

[0086] The recording medium of the 41st embodiment may have recorded thereon a program that combines program elements for causing a computer to at least partially execute steps corresponding to the features (configuration, operation, processing, etc.) of any of the 2nd to 36th embodiments.

[0087] The 38th to 41st embodiments can be used to address various problems caused by image filters used for vessel enhancement, including the first problem (uneven brightness in images of large blood vessels), the second problem (missing images of small blood vessels), and the third problem (noise in images of avascular regions). Furthermore, it is possible to select elements to be combined with each embodiment depending on the problem of particular interest.

[0088] The present disclosure describes various non-limiting aspects, including the first to forty-first aspects described above. The present disclosure mainly describes non-limiting aspects of ophthalmic devices (e.g., ophthalmic imaging devices, ophthalmic image processing devices), non-limiting aspects of methods for processing ophthalmic images, non-limiting aspects of methods for controlling ophthalmic devices, non-limiting aspects of programs, and non-limiting aspects of recording media. However, the categories of aspects of the embodiments are not limited to these non-limiting categories of aspects. For example, it would be understood by a person skilled in the relevant technical field that the embodiments of the present disclosure can provide various aspects in categories such as devices, systems, methods, programs, and recording media related to medical fields other than ophthalmology, as well as various aspects in categories such as devices, systems, methods, programs, and recording media related to technical fields other than the medical field.

[0089] <Embodiment of Ophthalmic Device> Some non-limiting aspects of the ophthalmic apparatus according to the embodiment will be described below. The ophthalmic apparatus according to the embodiment has a function of generating an OCT angiography image or a function of externally acquiring the OCT angiography image, and a function of processing the OCT angiography image.

[0090] The ophthalmic device of the embodiment mainly described in the present disclosure functions as an OCT device capable of performing OCT angiography (OCT scanning and image construction processing). The ophthalmic device of another embodiment may not be capable of performing at least one of OCT scanning and image construction processing.

[0091] The OCT method may be any method, such as spectral-domain OCT or swept-source OCT. Spectral-domain OCT splits light from a low-coherence light source into measurement light and reference light, superimposes return light from the test object on the reference light to generate interference light, detects the spectral distribution of this interference light using a spectrometer, and performs processing such as Fourier transform on the detected spectral distribution to construct an image. Swept-source OCT splits light from a tunable light source into measurement light and reference light, superimposes return light from the test object on the reference light to generate interference light, detects this interference light using a photodetector (such as a balanced photodiode), and performs processing such as Fourier transform on the detection data collected in response to wavelength sweeps and measurement light scans to construct an image. That is, spectral-domain OCT is an OCT method that acquires spectral distributions using spatial division, while swept-source OCT is an OCT method that acquires spectral distributions using time division. Note that other OCT methods, such as time-domain OCT, may also be used.

[0092] The ophthalmic device of the aspect mainly described in this disclosure has the function of a fundus camera capable of photographing the fundus, but may also have the function of any ophthalmic imaging modality, such as a scanning laser ophthalmoscope (SLO), a slit lamp microscope, or a surgical microscope.

[0093] In this disclosure, unless otherwise specified, no distinction is made between "image data" and "images," which are visual information based on the image data. Furthermore, unless otherwise specified, no distinction is made between a site or tissue of the subject's eye and its image (image data).

[0094] <Configuration of ophthalmic equipment> The configuration of an ophthalmologic apparatus according to a non-limiting embodiment is shown in Figs. 1 to 3. The ophthalmologic apparatus 1 includes a fundus camera unit 2, an OCT unit 100, and an arithmetic and control unit 200. The fundus camera unit 2 includes elements of a fundus camera capable of photographing the fundus and the anterior segment, and elements of an OCT scanner. The OCT unit 100 includes elements of an OCT scanner. The arithmetic and control unit 200 includes one or more processors configured to perform various processes (such as calculation, analysis, and control).

[0095] At least a portion of the functionality of the elements of the embodiments of the present disclosure is implemented using circuitry or processing circuitry. The circuitry or processing circuitry may be a general-purpose processor, a special-purpose processor, an integrated circuit, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), a field programmable gate array (FPGA)), or a combination of these devices configured and / or programmed to perform at least a portion of the disclosed functionality. The term "circuitry," "unit," "means," or the like refers to hardware that performs at least a portion of the disclosed functions or that is programmed to perform at least a portion of the disclosed functions. The hardware may be the hardware disclosed herein or may be known hardware that is programmed and / or configured to perform at least a portion of the described functions. In the case of a processor, where the hardware can be considered a type of circuitry, the term "circuitry," "unit," "means," or the like refers to a combination of hardware and software, where the software is used to configure the hardware and / or the processor.

[0096] <Fundus camera unit 2> The fundus camera unit 2 is provided with an optical system for photographing the fundus Ef (and the anterior segment) of the subject's eye E. The digital image acquired by the fundus camera unit 2 is typically a front image. The fundus camera unit 2 can acquire an observed image by video shooting using near-infrared fixed light as illumination light, and can acquire a photographed image by shooting using visible flash light as illumination light, for example.

[0097] The fundus camera unit 2 includes an illumination optical system 10 and an imaging optical system 30. The illumination optical system 10 irradiates illumination light onto the subject's eye E. The imaging optical system 30 detects return light of the illumination light irradiated onto the subject's eye E. Measurement light provided from the OCT unit 100 is guided to the subject's eye E through an optical path within the fundus camera unit 2. Return light of the measurement light projected onto the subject's eye E is guided to the OCT unit 100 through an optical path within the fundus camera unit 2.

[0098] The observation illumination light output from the observation light source 11 of the illumination optical system 10 is reflected by the concave mirror 12, passes through the condenser lens 13, and passes through the visible cut filter 14 to become near-infrared light. It is then focused near the imaging light source 15, reflected by the mirror 16, and passed through the relay lens system 17, the relay lens 18, the aperture 19, and the relay lens system 20 to be guided to the aperture mirror 21. It is reflected by the mirror portion around the central hole of the aperture mirror 21, passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the subject's eye E (fundus Ef). The return light of the observation illumination light projected onto the subject's eye E is refracted by the objective lens 22, transmitted through the dichroic mirror 46, passes through the central hole of the aperture mirror 21, transmitted through the dichroic mirror 55, passes through the photographing focusing lens 31, is reflected by the mirror 32, transmitted through the half mirror 33A, reflected by the dichroic mirror 33, and is imaged on the light-receiving surface of the image sensor 35 by the imaging lens 34. The image sensor 35 detects the return light at regular time intervals. The focus of the photographing optical system 30 is adjusted according to the photographing region.

[0099] The imaging illumination light output from the imaging light source 15 travels along the same path as the observation illumination light and is projected onto the fundus oculi Ef. The return light of the imaging illumination light from the subject's eye E travels along the same path as the return light of the observation illumination light and is guided to the dichroic mirror 33, passes through the dichroic mirror 33, is reflected by a mirror 36, and is imaged by an imaging lens 37 on the light-receiving surface of an image sensor 38.

[0100] A liquid crystal display (LCD) 39 displays a fixation target (fixation target image) for guiding and fixing the gaze. The light beam output from the LCD display 39 is reflected by the half mirror 33A, reflected by the mirror 32, passes through the photographing focusing lens 31 and the dichroic mirror 55, passes through the central hole of the aperture mirror 21, transmits through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef. This allows the subject to visually recognize the fixation target.

[0101] The alignment optical system 50 generates an alignment index for aligning the ophthalmic apparatus 1 with respect to the subject's eye E. Alignment light output from a light-emitting diode (LED) 51 passes through an aperture 52, an aperture 53, and a relay lens 54, is reflected by a dichroic mirror 55, passes through the central hole of the aperture mirror 21, transmits through the dichroic mirror 46, and is projected onto the subject's eye E via the objective lens 22. The return light of the alignment light from the subject's eye E is guided to the image sensor 35 via the same path as the return light of the observation illumination light. Manual alignment or automatic alignment can be performed by referring to the received light image (alignment index image).

[0102] The focusing optical system 60 generates a split index used for focus adjustment of the subject's eye E. The focusing optical system 60 moves along the optical path (illumination optical path) of the illumination optical system 10 in conjunction with the movement of the photographing focusing lens 31 along the optical path (photography optical path) of the photographing optical system 30. The reflecting rod 67 is inserted into and removed from the illumination optical path. When performing focus adjustment, the reflective surface of the reflecting rod 67 is tilted relative to the illumination optical path. The focusing light output from the LED 61 passes through the relay lens 62, is split into two beams by the split index plate 63, passes through the two-hole diaphragm 64, is reflected by the mirror 65, is focused by the condenser lens 66 on the reflective surface of the reflecting rod 67, is reflected, passes through the relay lens 20, is reflected by the aperture mirror 21, passes through the dichroic mirror 46, and is projected onto the subject's eye E via the objective lens 22. The returning light of the focusing light from the subject's eye E is guided to the image sensor 35 along the same path as the returning light of the alignment light. By referring to the received light image (split target image), manual focusing and autofocusing can be performed.

[0103] A diopter correction lens 70 (plus lens) for correcting severe myopia and a diopter correction lens 71 (minus lens) for correcting severe myopia are selectively inserted into the photographing optical path between the aperture mirror 21 and the dichroic mirror 55.

[0104] The dichroic mirror 46 combines the optical path for fundus imaging and the optical path for OCT (measurement arm). The dichroic mirror 46 reflects light in the wavelength band used for OCT and transmits light for fundus imaging. The measurement arm is provided with, in order from the OCT unit 100 side, a collimator lens unit 40, a retroreflector 41, a dispersion compensation member 42, an OCT focusing lens 43, an optical scanner 44, and a relay lens 45. The retroreflector 41 is movable along the optical path of the measurement light LS incident thereon and is used to correct the optical path length according to the axial length and adjust the interference state. The dispersion compensation member 42 is used for dispersion compensation between the measurement arm and the reference arm. The OCT focusing lens 43 is movable along the measurement arm and is used to adjust the focus of the measurement arm. Focus adjustment of the ophthalmologic apparatus 1 is performed by coordinating the movement of the imaging focusing lens 31, the movement of the focusing optical system 60, and the movement of the OCT focusing lens 43. The optical scanner 44 is aligned to be disposed at a position substantially conjugate with the pupil of the subject's eye E, and changes the traveling direction of the measurement light LS. The optical scanner 44 is, for example, a galvano scanner capable of two-dimensional scanning.

[0105] <OCTユニット100> The exemplary OCT unit 100 shown in FIG. 2 is provided with a spectral domain OCT optical system. This OCT optical system includes an interference optical system. This interference optical system splits light from a low-coherence light source (broadband light source) into measurement light LS and reference light LR, and generates interference light LC by superimposing the return light of the measurement light LS projected onto the subject's eye E and the reference light LR that has passed through the reference optical path. The generated interference light LC is detected by a spectroscope 130. This allows a signal indicating the spectral distribution of the interference light LC to be obtained. This detection signal is sent to the arithmetic and control unit 200.

[0106] The light source unit 101 outputs broadband low-coherence light L0. The light source unit 101 includes an optical output device such as a superluminescent diode (SLD), an LED, or a semiconductor optical amplifier (SOA).

[0107] Low-coherence light L0 output from light source unit 101 is guided by optical fiber 102 to polarization controller 103, where its polarization state is adjusted, and then guided by optical fiber 104 to fiber coupler 105, where it is split into measurement light LS and reference light LR. The measurement light LS is guided by a sample arm, and the reference light LR is guided by a reference arm.

[0108] The reference light LR is guided by optical fiber 110 to collimator 111 and converted into a parallel beam, passes through optical path length compensation element 112 for compensating for the optical distance between the measurement arm and the reference arm, passes through dispersion compensation element 113 for dispersion compensation between the measurement arm and the reference arm, and is guided to retroreflector 114. Retroreflector 114 is movable along the optical path of the reference light LR incident thereon and is used to correct the optical path length according to the axial length and adjust the interference state. After passing through retroreflector 114, the reference light LR passes through dispersion compensation element 113 and optical path length compensation element 112 and is converted from a parallel beam into a focused beam by collimator 116, is guided through optical fiber 117 to polarization controller 118 for adjusting its polarization state, is guided through optical fiber 119 to attenuator 120 for adjusting its light intensity, and reaches fiber coupler 122 via optical fiber 121.

[0109] On the other hand, the measurement light LS generated by the fiber coupler 105 is guided through the optical fiber 127 to the collimator lens unit 40, where it is converted into a parallel beam, passes through the retroreflector 41, the dispersion compensation member 42, the OCT focusing lens 43, the optical scanner 44, and the relay lens 45, is reflected by the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the subject's eye E. The measurement light LS is scattered and reflected at various depth positions in the subject's eye E. Return light of the measurement light LS from the subject's eye E travels in the opposite direction through the measurement arm, is guided to the fiber coupler 105, and reaches the fiber coupler 122 via the optical fiber 128.

[0110] The fiber coupler 122 generates interference light LC by superimposing the measurement light LS incident via the optical fiber 128 and the reference light LR incident via the optical fiber 121. The interference light LC generated by the fiber coupler 122 is guided to the spectrometer 130 via the optical fiber 129. The spectrometer 130 converts the incident interference light LC into a parallel beam using a collimator lens, resolves the parallel beam of interference light LC into multiple spectral components using a diffraction grating, and projects the multiple spectral components generated by the diffraction grating onto an image sensor via a lens 114. This image sensor is, for example, a line sensor, and detects the multiple spectral components of the interference light LC to generate an electrical signal (detection signal). The generated detection signal contains information about the spectral distribution of the interference light LC and is sent to the arithmetic and control unit 200.

[0111] When swept-source OCT is used instead of spectral-domain OCT, the light source unit 101 includes, for example, a tunable light source (e.g., a near-infrared tunable laser) that rapidly changes the wavelength of emitted light. In the swept-source OCT optical system, interference light LC, generated by superimposing measurement light LS and reference light LS, is split at a predetermined splitting ratio (e.g., 1:1) to generate a pair of interference light beams, which are then detected by a photodetector. The photodetector includes a balanced photodiode. The balanced photodiode includes a pair of photodetectors that respectively detect the pair of interference light beams and outputs the difference between the pair of detection signals obtained by these. The photodetector sends this difference signal to a data acquisition system (DAQ). A clock is supplied to the data acquisition system from the light source unit 101. This clock is generated in the light source unit 101 in synchronization with the output timing of each wavelength swept within a predetermined wavelength range by the tunable light source. The light source unit 101, for example, splits light of each output wavelength to generate two split lights, optically delays one of the split lights, and then combines the two split lights. The resulting combined light is detected, and a clock is generated based on the detection signal. The data collection system samples the detection signal (differential signal) input from the photodetector based on this clock. The data obtained by this sampling is used for processing such as image construction.

[0112] In the examples shown in FIGS. 1 and 2, both the measurement arm and the reference arm are provided with optical path length changing elements (retroreflectors 41 and 114), but only one of them may be provided. Furthermore, the optical path length changing elements are not limited to retroreflectors. For example, the optical path length changing element of the reference arm may be a movable reflective member (reference mirror). More generally, some exemplary embodiments include an element configured to change the relative lengths of the measurement arm and the reference arm (i.e., change the optical path length difference between the measurement arm and the reference arm), thereby allowing the coherence gate position to be moved.

[0113] <Arithmetic and control unit 200> The arithmetic and control unit 200 controls each part of the ophthalmologic apparatus 1 and performs various calculations and analyses. For example, the arithmetic and control unit 200 performs signal processing such as Fourier transform on the spectral distribution acquired by the spectroscope 130 to calculate a reflection intensity profile of a line (A-line) extending in the depth direction (z-direction) at each projection position of the signal light LS. Furthermore, the arithmetic and control unit 200 generates image data by imaging the reflection intensity profile of each A-line. The arithmetic and control unit 200 may perform the same calculations as those used to construct images in conventional spectral domain OCT. The arithmetic and control unit 200 includes, for example, a processor, RAM, ROM, a hard disk drive, a communication interface, etc. Various computer programs are stored in the storage device such as the hard disk drive. The arithmetic and control unit 200 may also include an operation device, an input device, a display device, etc.

[0114] As shown in FIG. 3, the user interface 240 includes a display unit 241 and an operation unit 242. The display unit 241 includes, for example, the display device 3 of FIG. 1. The operation unit 242 includes various operation devices and input devices. The user interface 240 may include a touch panel. In some exemplary embodiments, at least a part of the user interface is provided as a peripheral device connected to the ophthalmic apparatus 1. Furthermore, the movement mechanism 150 is configured to move the optical system of the ophthalmic apparatus 1, for example, to move at least the fundus camera unit 2 three-dimensionally.

[0115] <Processing system> The ophthalmologic apparatus 1 includes a control unit 210, an image constructing unit 220, and a data processing unit 230. These are provided in an arithmetic and control unit 200.

[0116] <Control unit 210> The control unit 210 includes a processor and controls each unit of the ophthalmologic apparatus 1. The control unit 210 includes a main control unit 211 and a storage unit 212.

[0117] <Main control unit 211> The main controller 211 includes a processor and is configured to control each element (including the elements shown in FIGS. 1 to 3) of the ophthalmic apparatus 1. The main controller 211 may also be configured to be able to control an apparatus, device, or system connected to the ophthalmic apparatus 1. The functions of the main controller 211 are realized, for example, by cooperation between hardware including circuits and control software.

[0118] <Storage section 212> The storage unit 212 stores various types of data and includes a storage device such as a hard disk drive or a solid state drive.

[0119] <Image construction unit 220> The image construction unit 220 processes data collected by applying an OCT scan to the fundus Ef of the subject's eye E to generate OCT image data. The image construction unit 220 includes a processor. The functions of the image construction unit 220 are realized, for example, by cooperation between hardware including circuits and image construction software.

[0120] The image construction unit 220 can construct cross-sectional image data based on the data acquired by the spectrometer 130. This image construction process includes signal processing such as sampling (A / D conversion), noise removal (noise reduction), filtering, and fast Fourier transform (FFT), similar to conventional spectral domain OCT.

[0121] The OCT image data constructed by the image construction unit 220 is a data set including a group of image data (a group of A-scan image data) that visualizes the reflection intensity profiles at multiple A-lines arranged in the area where the OCT scan was applied.

[0122] The OCT image data may be, for example, stack data constructed by embedding multiple B-scan image data in a single three-dimensional coordinate system. The image constructor 220 can construct volume data (voxel data) by applying voxelization processing to the stack data. Stack data and volume data are typical examples of three-dimensional image data expressed in a three-dimensional coordinate system.

[0123] The image constructing unit 220 can process the three-dimensional image data. For example, the image constructing unit 220 can construct new image data by applying rendering to the three-dimensional image data. Rendering techniques include volume rendering, surface rendering, multi-planar reconstruction (MPR), maximum intensity projection (MIP), minimum intensity projection (MIP), and average intensity projection (AIP). The image constructing unit 220 can construct projection data by integrating the three-dimensional image data in the z direction. The image constructing unit 220 can construct a shadowgram by integrating a portion of the three-dimensional image data (three-dimensional partial image data) in the z direction. The three-dimensional partial image data is extracted from the three-dimensional image data using any segmentation method.

[0124] The ophthalmic apparatus 1 may be capable of performing OCT angiography. In OCT angiography, the ophthalmic apparatus 1 repeatedly performs a predetermined number of scans targeting the same region of the fundus Ef. The image construction unit 220 constructs a motion contrast image based on difference information in the data sets collected by these repeated scans. This motion contrast image is an image obtained by emphasizing interference signals that change over time due to blood flowing in the fundus blood vessels, and is an angiography image that represents the distribution of blood vessels in the fundus Ef (actually, the distribution of blood flow). Typically, the ophthalmic apparatus 1 applies OCT angiography to a three-dimensional region of the fundus Ef to obtain multiple three-dimensional data (data sets), and creates three-dimensional angiography image data that represents the three-dimensional distribution of the fundus blood vessels based on these data sets.

[0125] The image constructing unit 220 can construct any 2D angiographic image data and / or any pseudo-3D angiographic image data from this 3D angiographic image data. For example, the image constructing unit 220 can construct 2D angiographic image data representing any cross section of the fundus oculi Ef by applying multiplanar reconstruction to the 3D angiographic image data. The image constructing unit 200 can also construct en face image data from an image region (slab) corresponding to a specific tissue identified by applying segmentation to the 3D angiographic image data. This en face image data is an example of a shadowgram. Typically, en face image data is constructed for various depth areas, such as the superficial retina, deep retina, and choroid. OCT angiography is described, for example, in Japanese Patent Application Laid-Open No. 2019-42264 filed by the present applicant. The ophthalmic apparatus 1 may be configured to have the data processing unit 230 perform at least a portion of the processing related to OCT angiography.

[0126] <Data processing unit 230> The data processing unit 230 performs various types of data processing. For example, the data processing unit 230 can apply various types of processing to images (fundus images, anterior segment images, etc.) acquired by the fundus camera unit 2 and images acquired using OCT scanning (OCT images). The data processing unit 230 includes a processor. The data processing unit 230 is realized, for example, by cooperation between hardware including circuits and data processing software.

[0127] <Functional configuration of ophthalmic equipment> The following describes the functional configuration of the ophthalmic apparatus 1 realized by the elements (hardware elements, software elements) shown in Figures 1 to 3. An example of the functional configuration of the ophthalmic apparatus 1 is shown in Figure 4.

[0128] The ophthalmologic apparatus 1 in FIG. 4 includes an image acquisition unit 1000, an image projection processing unit 1100, a blood vessel enhancement processing unit 1200, a noise removal processing unit 1300, and an image synthesis processing unit 1400.

[0129] <Image Acquisition Unit 1000> The image acquisition unit 1000 is configured to acquire an OCT angiography image of the fundus Ef of the subject's eye E. Some non-limiting examples of the image acquisition unit 1000 are described below.

[0130] The OCT angiography image acquired by the image acquisition unit 1000 may be in any form. For example, the OCT angiography image may be a three-dimensional OCT angiography image representing a three-dimensional region of the fundus oculi Ef. This three-dimensional OCT angiography image may be, for example, any of a data set consisting of a plurality of A-scan angiography images, stack data based on a plurality of A-scan angiography images, a data set consisting of a plurality of B-scan angiography images, stack data based on a plurality of B-scan angiography images, and volume data constructed from a plurality of A-scan angiography images or stack data based on a plurality of B-scan angiography images, or may be image data in a form other than these.

[0131] 1 to 3, the image acquisition unit 1000 can be realized by combining the fundus camera unit 2 (a group of elements forming the measurement arm), the OCT unit 100, the image construction unit 220, and the main controller 211 that controls them. In this case, the image acquisition unit 1000 includes a scan unit (fundus camera unit 2, OCT unit 100, main controller 211) that applies OCT scanning to the fundus Ef to collect data, and an image construction unit 220 that constructs an OCT angiography image based on the data collected by the scan unit.

[0132] In another example, the image acquisition unit 1000 may be realized by a communication interface (described above) included in the arithmetic and control unit 200. In the image acquisition unit 1000 of this example, the communication interface functions as an image receiving unit that receives OCT angiography images from an external device. The image acquisition unit 1000 of this example acquires OCT angiography images stored in an external device (e.g., an image archiving system, an ophthalmic imaging device, a storage device, etc.) via a communication line. In a similar example, the image acquisition unit 1000 includes a drive device that reads out OCT angiography images recorded on a recording medium. In the example of FIG. 1, the drive device may be provided in the arithmetic and control unit 200.

[0133] In yet another example, the image acquisition unit 1000 includes a data receiving unit (e.g., the communication interface or drive device described above) that receives data collected by applying an OCT scan to the fundus Ef, and an image construction unit 220 that constructs an OCT angiography image based on the data received by the data receiving unit.

[0134] <Image projection processing unit 1100> The image projection processing unit 1100 is configured to apply projection processing to the OCT angiography image acquired by the image acquisition unit 1000 to generate a projection image.

[0135] For example, the image projection processing unit 1100 may be configured to generate a two-dimensional OCT angiography image defined in a plane perpendicular to the projection direction (e.g., the xy plane) by projecting a three-dimensional OCT angiography image representing a three-dimensional region of the fundus Ef in a particular direction (e.g., the z direction).

[0136] The projection direction may be determined in advance, or may be determined based on various conditions such as the type, features, and characteristics of the 3D OCT angiography image to which the projection process is applied, the part of the fundus Ef represented by the 3D OCT angiography image, the type and parameters of the projection process, and the type, features, and characteristics of the resulting image (2D OCT angiography image) of the projection process.

[0137] In this disclosure, a detailed description will be given of cases where maximum intensity projection (MIP) and average intensity projection (AIP) are used as projection processes, but the types of projection processes that can be employed in the embodiments are not limited to these.

[0138] The projection process may be applied to the entire OCT angiography image acquired by the image acquisition unit 1000, or to only a portion of the image. In the latter case, for example, the image projection processing unit 1100 applies segmentation to the OCT angiography image acquired by the image acquisition unit 1000 to extract an image region corresponding to a specific layer of tissue in the fundus Ef, and then applies projection processing to this image region to generate a projection image. This layer of tissue is typically at least a portion of the retina, but may also be at least a portion of the choroid, at least a portion of the sclera, at least a portion of the vitreous body, the space between the retina and the vitreous body, or a combination of at least two of these.

[0139] According to the configuration shown in FIGS. 1 to 3, the image construction unit 220 and / or the data processing unit 230 can realize the image projection processing unit 1100.

[0140] <Vessel enhancement processing unit 1200> The vessel enhancement processing unit 1200 is configured to generate a vessel enhancement image by applying a vessel enhancement filter for enhancing a vessel image to the projection image generated by the image projection processing unit 1100. The vessel enhancement image is an image in which the vessel image in the projection image is emphasized.

[0141] This disclosure provides a detailed description of the case where a multiscale Frangi filter is used as a vessel enhancement filter, but those skilled in the art will understand that the techniques disclosed herein are also useful when other image filters (e.g., Gabor filters, non-local means filters, wavelet filters, etc.) are used.

[0142] Non-limiting examples of image processing methods that can be used for vascular enhancement include: a method using vector concentration using density gradients; a method using black top-hat transforms; a method using double ring filters; a method combining black top-hat transforms and double ring filters; a method for detecting gray-level ridges from the green component image of a color image; a method using Gabor wavelets; a method using matched filters; a method using the Hough transform; a method using the Contourlet transform; a method using the Curvelet transform; a method based on ensemble learning; a method using a morphological filter bank; and a method using machine learning.

[0143] <Noise removal processing unit 1300> The noise removal processing unit 1300 is configured to apply noise removal processing to the projection image generated by the image projection processing unit 1100 to generate a noise-removed image.

[0144] The noise removal process executed by the noise removal processor 1300 means not only a process for removing noise but also a process for reducing noise.

[0145] The noise removal processing unit 1300 may be configured to be able to perform a plurality of different processes in the noise removal process. The plurality of processes in the noise removal process may include processes according to three embodiments (first to third embodiments that focus on solving the first to third problems described above) that will be described later.

[0146] When multiple processes are executable, the noise removal processing unit 1300 may be configured to select at least one process from the multiple processes based on the angle of view of the OCT angiography image acquired by the image acquisition unit 1000. In other words, the noise removal processing unit 1300 may be configured to select at least one process from the multiple processes based on the width of the range (scan area) of the OCT scan applied to the fundus Ef to generate the OCT angiography image. The noise removal processing unit 1300 can generate a noise-removed image by applying each process selected from the multiple processes to the projection image generated by the image projection processing unit 1100.

[0147] For example, when processing a wide-angle OCT angiography image that includes both the optic disc and the macula, the OCT angiography image depicts thick blood vessels around the optic disc, thin blood vessels (capillaries) in various locations, and the foveal avascular region. In this case, the noise removal processor 1300 can execute processing according to a first embodiment that addresses the problem of uneven brightness in the image of thick blood vessels (first problem), processing according to a second embodiment that addresses the problem of missing thin blood vessels (second problem), and processing according to a third embodiment that addresses the problem of noise in the avascular region (third problem).

[0148] As another example, when processing an OCT angiography image with a narrow field of view that includes only the optic disc and its surroundings, the thick blood vessels around the optic disc are depicted in this OCT angiography image. In this case, the noise removal processor 1300 can execute the processing according to the first embodiment, which addresses the problem of uneven brightness in the image of the thick blood vessels (the first problem). In addition, it may execute the processing according to the second embodiment, which addresses the problem of missing thin blood vessels (the second problem).

[0149] As another example, when processing an OCT angiography image with a narrow field of view that includes only the macula and its surroundings, the foveal avascular region is depicted in the OCT angiography image. In this case, the noise removal processor 1300 can perform processing according to the third embodiment, which addresses the problem of noise in the avascular region (the third problem). In addition, the noise removal processor 1300 may perform processing according to the second embodiment, which addresses the problem of missing thin blood vessels (the second problem).

[0150] When multiple processes are executable, the noise removal processing unit 1300 may be configured to select at least one process from the multiple processes based on at least one of the fixation position and the scan area applied to generate the OCT angiography image. In other words, the noise removal processing unit 1300 may be configured to select at least one process from the multiple processes based on the region of the fundus Ef depicted in the OCT angiography image. The noise removal processing unit 1300 can generate a noise-removed image by applying each process selected from the multiple processes to the projection image generated by the image projection processing unit 1100.

[0151] For example, when an OCT angiography image is generated by OCT angiography with the fixation position or scan area set to the optic disc, the noise removal processor 1300 can execute the processing according to the first embodiment, which addresses the problem of uneven brightness in the image of large blood vessels (the first problem). Alternatively, when an OCT angiography image is generated by OCT angiography with the fixation position or scan area set to the macula, the noise removal processor 1300 can execute the processing according to the third embodiment, which addresses the problem of noise in avascular regions (the third problem). In these cases, in addition to the processing according to the first or third embodiment, the noise removal processor 1300 may execute the processing according to the second embodiment, which addresses the problem of missing small blood vessels (the second problem).

[0152] As another example, when an OCT angiography image is generated by OCT angiography with the fixation position or scan area set at the center of the fundus (the position between the optic disc and the macula), the noise removal processor 1300 can execute the processing according to the first embodiment that addresses the problem of uneven brightness in the image of thick blood vessels (the first problem) and the processing according to the third embodiment that addresses the problem of noise in the avascular region (the third problem).In addition, the noise removal processor 1300 may execute the processing according to the second embodiment that addresses the problem of missing thin blood vessels (the second problem).

[0153] When multiple processes can be performed, the noise removal processing unit 1300 may be configured to select at least one process from the multiple processes based on the angle of view of the OCT angiography image acquired by the image acquisition unit 1000 and at least one of the fixation position and the scan area applied to generate the OCT angiography image. The noise removal processing unit 1300 can generate a noise-removed image by applying each process selected from the multiple processes to the projection image generated by the image projection processing unit 1100.

[0154] For example, when an OCT angiography image is generated by OCT angiography with a wide field of view in which the fixation position or scan area is set at the center of the fundus, the noise removal processing unit 1300 can execute processing according to the first embodiment that addresses the problem of uneven brightness in the image of thick blood vessels (first problem), processing according to the second embodiment that addresses the problem of missing thin blood vessels (second problem), and processing according to the third embodiment that addresses the problem of noise in avascular regions (third problem).

[0155] <Image synthesis processing unit 1400> The image synthesis processing unit 1400 is configured to apply image synthesis processing to the projection image generated by the image projection processing unit 1100, the blood vessel enhancement image generated by the blood vessel enhancement processing unit 1200, and the noise-removed image generated by the noise removal processing unit 1300. The image generated by this image synthesis processing is called a synthesized image.

[0156] The type of image synthesis processing executed by the image synthesis processing unit 1400 may be any type, for example, alpha blending.

[0157] <Operation of ophthalmic device> The operation of the ophthalmologic apparatus 1 will be described with reference to Fig. 5A and Fig. 5B. Also, differences between the operation example shown in Fig. 5A and Fig. 5B and the comparative example shown in Fig. 6 will be described. Note that the comparative example in Fig. 6 is not presented as a conventional technique, but is presented as an aid to the explanation of the operation example in Fig. 5A and Fig. 5B.

[0158] 5A and 5B, first, the ophthalmologic apparatus 1 acquires an OCT angiography image 2000 of the fundus Ef of the subject's eye E by the image acquisition unit 1000 (S1). The acquired OCT angiography image 2000 is stored in the storage unit 212 by the main control unit 211.

[0159] Next, the ophthalmologic apparatus 1 generates a projection image 2020 by applying a projection process 2010 to the OCT angiography image 2000 using the image projection processing unit 1100 (S2). The generated projection image 2020 is stored in the storage unit 212 by the main control unit 211. The projection method used in the projection process 2010 may be, for example, maximum intensity projection (MIP) or average intensity projection (AIP).

[0160] Next, the ophthalmologic apparatus 1 applies a vessel enhancement filter 2030 for enhancing the blood vessel image to the projection image 2020 by the vessel enhancement processing unit 1200 to generate a vessel enhancement image 2040 (S3). The generated vessel enhancement image 2040 is stored in the storage unit 212 by the main control unit 211. The vessel enhancement filter 2030 may be, for example, a multiscale frangi filter. The vessel enhancement filter 2030 may cause problems such as uneven brightness in the image of thick blood vessels, missing thin blood vessels, and noise in avascular regions.

[0161] Furthermore, the ophthalmologic apparatus 1 causes the noise removal processing unit 1300 to apply noise removal processing 2050 to the projection image 2020 to generate a noise-removed image 2060 (S4). The generated noise-removed image 2060 is stored in the storage unit 212 by the main control unit 211.

[0162] In this operation example, generation of a denoised image (S4) is performed after generation of a vascular emphasis image (S3), but in another operation example, generation of a vascular emphasis image may be performed after generation of a denoised image. In yet another operation example, generation of a vascular emphasis image and generation of a denoised image may be performed at least partially in parallel (in other words, they may be performed at least partially simultaneously).

[0163] Next, the ophthalmologic apparatus 1 generates a composite image 2080 by applying an image synthesis process 2070 to the projection image 2020, the blood vessel enhancement image 2040, and the noise-removed image 2060 using the image synthesis processing unit 1400 (S5). The generated composite image 2080 is stored in the storage unit 212 by the main control unit 211. The image synthesis process 2070 may be, for example, alpha blending.

[0164] Conventionally, a vascular enhancement image generated by applying a vascular enhancement filter to an OCT angiography image is typically displayed. In the image synthesis process 2070 of this operational example, a vascular enhancement image 2040 is synthesized with a projection image 2020 and a noise-removed image 2060. One reason for synthesizing the noise-removed image 2060 is to eliminate problems that occur in the vascular enhancement image 2040 (such as uneven brightness in the image of large blood vessels, missing thin blood vessels, and noise in avascular regions). Another reason for synthesizing the projection image 2020 is to prevent the texture of the synthesized image 2080 from deviating from the texture of the original images (OCT angiography image 2000, projection image 2020) to which the vascular enhancement filter 2030 and noise removal process 2050 have not been applied. This effect of the image synthesis process 2070 is particularly notable and unique to this embodiment. This concludes this operational example (END).

[0165] Some non-limiting examples of processes that can be performed based on the images (and various information associated therewith) handled in this operation example will be described below.

[0166] The ophthalmic apparatus 1 can display a composite image 2080 using the main control unit 211 and the display unit 241. The ophthalmic apparatus 1 can also display one or more of an OCT angiography image 2000, a projection image 2020, a blood vessel enhancement image 2040, and a noise-removed image 2060. The ophthalmic apparatus 1 can also display a fundus image and an anterior eye image acquired by the fundus camera unit 2. A user can perform image interpretation and report creation by referring to the displayed images.

[0167] The ophthalmologic apparatus 1 can apply processing to one or more of the OCT angiography image 2000, the projection image 2020, the vessel-enhanced image 2040, the noise-removed image 2060, the composite image 2080, the fundus image, and the anterior segment image by using either or both of the image constructing unit 220 and the data processing unit 230. This processing may be image processing or signal processing, non-limiting examples of which include image analysis, image evaluation, image quality improvement, segmentation, rendering, etc.

[0168] The processing performed by either or both of the image constructing unit 220 and the data processing unit 230 may include processing using a model constructed by machine learning. As a non-limiting example, the machine learning model may be configured to be capable of performing any of the following processes: interpretation and report generation of any one or more images (e.g., the composite image 2080) handled by the ophthalmic apparatus 1; comparison of any two or more images (e.g., the OCT angiography image 2020 and the composite image 2080); or generation of training data for further machine learning (e.g., updating the training dataset).

[0169] The ophthalmologic device 1 can transmit one or more of the OCT angiography image 2000, the projection image 2020, the vascular enhancement image 2040, the noise-removed image 2060, the composite image 2080, the fundus image, and the anterior segment image to an external device via a communication interface included in the arithmetic and control unit 200.

[0170] The ophthalmologic device 1 can record one or more of the OCT angiography image 2000, the projection image 2020, the vascular enhancement image 2040, the noise-removed image 2060, the composite image 2080, the fundus image, and the anterior segment image on a recording medium using a drive device included in the arithmetic control unit 200.

[0171] Next, differences between the operational example of FIGS. 5A and 5B and the comparative example of FIG. 6 will be described, along with some non-limiting features of this operational example.

[0172] In the comparative example of Figure 6, an OCT angiography image 2005 is acquired, a projection process 2015 is applied to the OCT angiography image 2005 to generate a projection image 2025, a vascular enhancement filter 2035 is applied to the projection image 2025 to generate a vascular enhancement image 2045, and an image synthesis process 2075 is applied to the projection image 2025 and the vascular enhancement image 2045 to generate a synthesized image 2085.

[0173] Comparing this operation example in Figures 5A and 5B with the comparative example in Figure 6, this operation example differs from the comparative example in that it executes a process (noise removal process 2050) to generate a noise-removed image 2060 from a projection image 2020, and in that it synthesizes not only the projection image 2020 and the vascular enhancement image 2040 but also the noise-removed image 2060 in the image synthesis process 2070.

[0174] In the comparative example, the projection image and the vascular enhancement image are synthesized without performing processing equivalent to the noise removal processing 2050 of this operational example. Therefore, the resulting synthesized image still has defects caused by the vascular enhancement filter 2035 (e.g., uneven brightness in the image of thick blood vessels, missing images of thin blood vessels, noise in the avascular area of ​​the fovea, etc.), which degrade the quality of the representation of the fundus blood vessels. In contrast, this operational example can eliminate or reduce these defects, thereby providing a high-quality fundus blood vessel image (OCT angiography image with enhanced blood vessels). In addition, this operational example can provide a fundus blood vessel image with a texture similar to that of the OCT angiography image 2000 and the projection image 2020.

[0175] Fig. 7 shows another example of operation of the ophthalmic apparatus 1. In this example of operation, similar to steps S1 to S3 in Fig. 5A, the ophthalmic apparatus 1 acquires an OCT angiography image of the fundus Ef of the subject's eye E (S11), applies projection processing to this OCT angiography image to generate a projection image (S12), and applies a vessel enhancement filter for enhancing the blood vessel image to this projection image to generate a vessel enhancement image (S13).

[0176] Furthermore, the ophthalmologic apparatus 1, for example, by the main controller 211 or the noise removal processor 1300, selects one or more processes from a plurality of processes prepared for noise removal based on the conditions of OCT angiography applied to the fundus Ef to generate the OCT angiography image acquired in step S11 (S14). The selected type of process is stored in the storage unit 212 by the main controller 211.

[0177] The multiple processes prepared for noise removal may include, for example, processes according to three embodiments described below. That is, the multiple processes in this operation example may include one or more processes according to a first embodiment that addresses the problem of uneven brightness in images of thick blood vessels, one or more processes according to a second embodiment that addresses the problem of missing thin blood vessels, and one or more processes according to a third embodiment that addresses the problem of noise in avascular regions.

[0178] The OCT angiography conditions may be, for example, one or more of the field of view, fixation position, and scan area, and are acquired together with the OCT angiography image in step S11. The information indicating the OCT angiography conditions may take any form. As a non-limiting example, the OCT angiography condition information may be information recorded as supplementary information of the OCT angiography image (e.g., DICOM tags, etc.), information recorded in information about the subject (e.g., electronic medical records, radiology reports, etc.), or information obtained by applying analysis processing to the OCT angiography image or an image generated based thereon (e.g., a projection image, a vascular-enhanced image, etc.).

[0179] The ophthalmologic apparatus 1 generates a noise-removed image by applying the noise removal processing including the processing selected in step S14 to the projection image generated in step S12 using the noise removal processing unit 1300 (S15). In this operation example, the order of generating the vessel enhancement image and the noise-removed image may also be arbitrary.

[0180] Next, the ophthalmologic apparatus 1 generates a composite image by applying image synthesis processing to the projection image generated in step S12, the blood vessel enhancement image generated in step S13, and the noise-removed image generated in step S15 using the image synthesis processing unit 1400 (S16). This completes this operation example (END).

[0181] 7 can achieve the same effects as those of the operation examples of FIGS. 5A and 5B, and can also perform noise removal processing using processing selected according to the conditions of OCT angiography. The latter effect, which is unique to this operation example, allows for the selection of appropriate processing according to the conditions of OCT angiography, and for the execution of noise removal processing while excluding unnecessary or less necessary processing. This makes it possible to perform noise removal processing effectively and efficiently.

[0182] <First Example> In the first embodiment, several aspects of processing for solving the first problem (problem of uneven brightness in images of large blood vessels) caused by the use of a vessel emphasis filter will be described.

[0183] A vessel enhancement filter configured to extract blood vessels of various thicknesses, such as a multi-scale Frangi filter, detects blood vessels of various thicknesses by changing the value of a predetermined scale parameter. Although the problem of uneven brightness can be (almost) resolved by using a scale parameter corresponding to thick blood vessels, other problems arise, such as blurred blood vessel images and the appearance of two different blood vessels connected together. The first embodiment addresses these secondary issues while resolving the problem of uneven brightness in images of thick blood vessels.

[0184] In the first embodiment, the noise removal processing unit 1300 is configured to apply a blood vessel image extraction process to the projection image generated from the OCT angiography image by the image projection processing unit 1100, in order to extract blood vessel images having diameter dimensions that belong to a first range.

[0185] The first range for the diameter of the blood vessel to be extracted may be determined in advance, or may be determined based on an OCT angiography image or an image (e.g., a projection image) generated based on the OCT angiography image. In the former case, the first range may be determined based on a standard value of the blood vessel diameter. In some embodiments, multiple standard values ​​are prepared according to multiple conditions based on patient attributes such as age, sex, and race, and imaging conditions such as the location on the fundus, and the standard value according to the image to which the blood vessel image extraction process is applied is selectively used as the first range. In the latter case, the ophthalmologic apparatus 1 (e.g., the noise removal processing unit 1300) may be configured to analyze the OCT angiography image or the projection image to obtain a distribution or statistics of the blood vessel diameter, and to determine the first range based on the distribution or statistics.

[0186] Furthermore, the noise removal processing unit 1300 is configured to apply erosion processing to the image generated by this blood vessel image extraction processing, thereby generating an eroded image in which a reduced blood vessel image is depicted, with the diameter dimensions of the blood vessel image extracted by this blood vessel image extraction processing reduced.

[0187] Generally, erosion processing is a type of morphological processing that shrinks a region of interest by replacing the pixel values ​​of the region of interest with the values ​​of neighboring pixels. In the first embodiment, it acts to reduce the dimensions of the blood vessel image (region of interest) depicted in the image generated by the blood vessel image extraction processing. This generates an erosion image in which blood vessel images (reduced blood vessel images) with smaller diameters than the blood vessel images extracted by the blood vessel image extraction processing are depicted.

[0188] The vascular enhancement processing unit 1200 is configured to generate a first vascular enhancement image by applying a multiscale Frangi filter to the projection image generated from the OCT angiography image by the image projection processing unit 1100.

[0189] Furthermore, the vascular enhancement processing unit 1200 is configured to apply a Frangi filter having a scale corresponding to a second range that is smaller than the first range used in the vascular image extraction processing to the projection image generated from the OCT angiography image by the image projection processing unit 1100.

[0190] In addition, the vascular enhancement processing unit 1200 is configured to generate a second vascular enhancement image by applying gamma correction to an image generated by applying a Frangi filter of a scale corresponding to the second range to the projection image, in order to increase the brightness of the vascular image.

[0191] The noise removal processing unit 1300 is configured to generate a noise-removed image based on an eroded image depicting a reduced blood vessel image in which the diameter dimensions of the blood vessel image extracted in the blood vessel image extraction process are reduced, a first blood vessel enhancement image generated by applying a multi-scale Frange filter to the projection image, and a second blood vessel enhancement image generated by applying a Frange filter of a scale corresponding to a second range smaller than the first range of the blood vessel image extraction process and gamma correction to the projection image.

[0192] The image synthesis processor 1400 is configured to generate a synthesized image by applying an image synthesis process to the denoised image generated from the eroded image, the first vascular image, and the second vascular image, the projection image, and the first vascular image. The image synthesis process may be performed using alpha blending or another image synthesis method.

[0193] In the first embodiment, by executing a series of processes that can be realized with this configuration, it is possible to generate a vessel-enhanced OCT angiography image in which there is no brightness unevenness in the image of large blood vessels. The mechanism by which this effect is achieved will be explained after explaining some specific examples.

[0194] First, a specific example (first specific example) of the noise-removed image generation process in the first embodiment will be described.

[0195] In a first specific example of the first embodiment, the noise removal processor 1300 is configured to analyze the first vessel-enhanced image to identify a first partial image in the first vessel-enhanced image, which corresponds to a reduced vessel image in the eroded image. For this processing, for example, a threshold process on brightness values, a segmentation method, a segmentation method using a machine learning model, or the like is used.

[0196] Similarly, the noise removal processing unit 1300 is configured to analyze the second vessel enhancement image to identify a second partial image in the second vessel enhancement image, which corresponds to a reduced vessel image of the eroded image.

[0197] Furthermore, the noise removal processing unit 1300 is configured to generate a noise-removed image by selecting the larger luminance value of each pixel of the first partial image in the first blood vessel enhancement image and the luminance value of the corresponding pixel of the second partial image in the second blood vessel enhancement image.

[0198] Because the first and second vascular image are generated from the same OCT angiography image, a natural correspondence relationship is defined between the pixels constituting the first and second vascular image. By using this correspondence relationship, it is possible to identify the pixels (corresponding pixels) in the second partial image that correspond to each pixel in the first partial image.

[0199] The noise removal processing unit 1300 compares the luminance values ​​of each pair of pixels of the first partial image and the second partial image that have been associated in this way, and identifies the pixel with the larger luminance value. A noise-removed image is generated by using the pixel group identified by performing this process for each pixel pair.

[0200] The image synthesis processing unit 1400 is configured to generate a synthetic image by applying image synthesis processing to the noise-removed image generated from the first partial image and the second partial image, the projection image, and the first vascular enhancement image in this manner.

[0201] Next, a description will be given of a further specific example (second specific example) of the first example of the first embodiment. The second specific example of the first embodiment provides a further specific example of the noise removal process in the first specific example.

[0202] In a second specific example of the first embodiment, the noise removal processing unit 1300 is configured to determine a first partial image in the first vessel-enhanced image by applying mask processing based on the reduced vessel image in the eroded image to the first vessel-enhanced image.

[0203] Furthermore, the noise removal processing unit 1300 is configured to determine a second partial image in the second vessel enhancement image by applying mask processing based on the reduced vessel image in the eroded image to the second vessel enhancement image.

[0204] The masking process applied to the first and second vascular enhancement images may be the same. For example, the noise removal processor 1300 applies a mask to an image region in the first vascular enhancement image that corresponds to a reduced vascular image in the eroded image in the masking process for the first vascular enhancement image, and applies a mask to an image region in the second vascular enhancement image that corresponds to a reduced vascular image in the eroded image in the masking process for the second vascular enhancement image. This allows a first partial image to be identified from the first vascular enhancement image, and a second partial image to be identified from the second vascular enhancement image.

[0205] Next, a description will be given of a further specific example (third specific example) of the first or second specific example of the first embodiment. In the third specific example of the first embodiment, the image projection processing unit 1100 is configured to generate a projection image (maximum intensity projection image) by executing maximum intensity projection (MIP) as a projection process on the OCT angiography image. Maximum intensity projection is an image projection method that selects and projects the maximum value from the luminance values ​​of a group of pixels arranged in the projection direction.

[0206] Next, a description will be given of a further specific example (fourth specific example) of any of the first to third specific examples of the first embodiment. In the fourth specific example of the first embodiment, the noise removal processing unit 1300 is configured to perform Otsu's binarization as a blood vessel image extraction process applied to the projection image in order to extract blood vessel images with diameter dimensions belonging to a first range.

[0207] In general, Otsu's thresholding is an algorithm for binarizing luminance images, and is particularly effective for binarizing images whose luminance histograms have two peaks (images with bimodal histograms). Otsu's thresholding algorithm is configured to search for a threshold that minimizes the intra-class variance, which is defined as the weighted sum of the variances of two classes. In other words, this algorithm is configured to find a threshold that lies between the two peaks of the bimodal histogram, that is, to find a threshold that minimizes the intra-class variance of the two classes classified by the thresholding.

[0208] In the fourth specific example of the first embodiment, by using Otsu's binarization, it is possible to selectively extract images of relatively thick blood vessels (blood vessel images with diameters falling within a first range) from images of blood vessels of various diameters depicted in a projection image. The noise removal processing unit 1300 applies erosion processing to the images of thick blood vessels extracted using Otsu's binarization, thereby generating an eroded image in which reduced blood vessel images of the thick blood vessels are depicted.

[0209] Next, the operation of the ophthalmologic apparatus 1 in the first embodiment will be described with reference to Figures 8A and 8B. The operation examples in Figures 8A and 8B include the processes according to the first to fourth specific examples described above.

[0210] 8A and 8B, first, the ophthalmologic apparatus 1 acquires an OCT angiography image 3000 of the fundus Ef of the subject's eye E by the image acquisition unit 1000 (S21). The acquired OCT angiography image 3000 is stored in the storage unit 212 by the main control unit 211.

[0211] Next, the ophthalmologic apparatus 1 generates a maximum intensity projection image 3020 by applying a maximum intensity projection 3010 to the OCT angiography image 3000 using the image projection processing unit 1100 (S22). The generated maximum intensity projection image 3020 is stored in the storage unit 212 by the main control unit 211.

[0212] After step S22, three processes are executed: steps S23 and S24 (reference numerals 3030 to 3060 in FIG. 8B), step S25 (reference numerals 3070 and 3080 in FIG. 8B), and steps S26 and S27 (reference numerals 3090 to 3120 in FIG. 8B). The order in which these three processes are executed may be arbitrary. Furthermore, at least two of the three processes may be executed at least partially in parallel.

[0213] The ophthalmologic apparatus 1 applies Otsu's binarization 3030 as a blood vessel image extraction process for extracting blood vessel images having diameters within a first range to the maximum intensity projection image 3020 by the noise removal processing unit 1300, thereby generating a binary image 3040 in which images of thick blood vessels are selectively depicted (S23). The generated binary image 3040 is stored in the storage unit 212 by the main control unit 211.

[0214] Furthermore, the ophthalmologic apparatus 1 applies erosion processing 3050 to the binary image 3040 by the noise removal processing unit 1300 to reduce the diameter of the thick blood vessel images depicted in the binary image 3040 (S24). An eroded image 3060 generated from the binary image 3040 by the erosion processing 3050 depicts reduced blood vessel images, which are blood vessel images obtained by reducing the diameter of the thick blood vessel images. The generated eroded image 3060 is stored in the storage unit 212 by the main control unit 211.

[0215] Furthermore, the ophthalmologic apparatus 1 generates a first vessel enhancement image 3080 by applying a multiscale Franzi filter 3070 to the maximum intensity projection image 3020 using the vessel enhancement processing unit 1200 (S25). The generated first vessel enhancement image 3080 is stored in the storage unit 212 by the main control unit 211.

[0216] Furthermore, the ophthalmologic apparatus 1 generates a Frangi filter image 3100 by applying a Frangi filter 3090 having a scale corresponding to a second range smaller than the first range in the blood vessel image extraction process (Otsu's binarization 3030) in step S23 to the maximum intensity projection image 3020 using the blood vessel emphasis processing unit 1200 (S26). The generated Frangi filter image 3100 is stored in the storage unit 212 by the main control unit 211.

[0217] Furthermore, the ophthalmologic apparatus 1 generates a second vessel enhancement image 3120 by applying gamma correction 3110 to the Frangi-filter image 3100 using the vessel enhancement processing unit 1200 to increase the brightness of the vessel image (S27). The generated second vessel enhancement image 3120 is stored in the storage unit 212 by the main control unit 211.

[0218] Next, the ophthalmologic apparatus 1 applies mask processing 3130 based on the reduced blood vessel image depicted in the eroded image 3060 to the first blood vessel emphasized image 3080 and the second blood vessel emphasized image 3120 by the noise removal processing unit 1300 (S28 and S29).

[0219] Specifically, the ophthalmologic apparatus 1 determines a first partial image in the first vessel enhancement image 3080 by applying mask processing 3130 to the first vessel enhancement image 3080 using the noise removal processing unit 1300 (S28). The main control unit 211 stores the first partial image or coordinate information in the first vessel enhancement image 3080 corresponding to the first partial image in the storage unit 212.

[0220] Furthermore, the ophthalmologic apparatus 1 determines a second partial image in the second blood vessel emphasis image 3120 by applying mask processing 3130 to the second blood vessel emphasis image 3120 using the noise removal processing unit 1300 (S29). The second partial image or coordinate information in the second blood vessel emphasis image 3120 corresponding to the second partial image is stored in the storage unit 212 by the main control unit 211.

[0221] Next, the ophthalmologic apparatus 1 generates a noise-removed image 3140 by causing the noise removal processing unit 1300 to compare the luminance values ​​of the pixel group constituting the first partial image identified from the first blood vessel enhancement image 3080 in step S28 with the luminance values ​​of the pixel group constituting the second partial image identified from the second blood vessel enhancement image 3120 in step S29 (S30). The generated noise-removed image 3140 is stored in the storage unit 212 by the main control unit 211.

[0222] More specifically, the ophthalmologic apparatus 1 generates the noise-removed image 3140 by using the noise removal processing unit 1300 to select the larger luminance value between the luminance value of each pixel of the first partial image in the first blood vessel enhancement image 3080 and the luminance value of the corresponding pixel of the second partial image in the second blood vessel enhancement image 3120. In this manner, in this operation example, the luminance of the region near the center line of the image of a thick blood vessel is increased, thereby addressing the problem of that region being represented with low luminance.

[0223] Next, the ophthalmologic apparatus 1 generates a composite image 3160 by applying image synthesis processing 3150 to the noise-removed image 3140, the maximum intensity projection image 3020, and the first blood vessel-enhanced image 3080 using the image synthesis processing unit 1400 (S31). The generated composite image 3160 is stored in the storage unit 212 by the main control unit 211. This completes this operation example (END).

[0224] The ophthalmic apparatus 1 can display one or more of an OCT angiography image 3000, a maximum intensity projection image 3020, a binary image 3040, an eroded image 3060, a first blood vessel-enhanced image 3080, a second blood vessel-enhanced image 3120, a noise-removed image 3140, and a composite image 3160. The ophthalmic apparatus 1 can also apply processing to one or more of these images. The ophthalmic apparatus 1 can also transmit one or more of these images to an external device and / or record them on a recording medium.

[0225] The ophthalmic device 1 of the first embodiment generates a first vascular enhancement image depicting images of blood vessels of various diameters by applying a multi-scale Frangi filter to a projection image of an OCT angiography image, and also generates a Frangi filter image by extracting blood vessel images from the same projection image using a Frangi filter of an appropriate scale for extracting images of blood vessels with diameter dimensions that belong to a second range smaller than the first range corresponding to the diameter dimensions of thick blood vessels.

[0226] Since this Frangi filter image is generated using a Frangi filter with a scale corresponding to the diameter dimension of the thick blood vessel to be extracted, the density of the blood vessel image depicted in this Frangi filter image is lower than the density of the image of the thick blood vessel to be extracted. In the first embodiment, gamma correction is applied to the Frangi filter image to increase the density of the blood vessel image depicted in the Frangi filter image.

[0227] Furthermore, in the first embodiment, a denoised image based on a Frangi-filtered image (second vessel-enhanced image) in which the density (brightness, contrast) of the vessel image has been increased by gamma correction is synthesized with a first vessel-enhanced image generated by applying a multiscale Frangi-filter to an OCT angiography image. The denoised image is generated using a masking process. This masking process allows subsequent processing to be applied to the appropriate vessel region.

[0228] According to the first embodiment, it is possible to eliminate or reduce the problem of uneven brightness occurring in images of large blood vessels, and also to avoid the problems that occur when using a Frangi filter with a scale corresponding to a larger blood vessel diameter (such as a blurred blood vessel image or two different blood vessels being depicted as being connected).

[0229] Furthermore, according to the first embodiment, by synthesizing not only the first vessel-enhanced image and the noise-removed image but also the projected image, it is possible to generate a synthesized image having a texture close to that of the original image.

[0230] 9 shows another example of operation of the ophthalmologic apparatus 1. In this example of operation, instead of the above-described erosion processing, a brightness distribution (brightness map) relating to the distance from the center line of the blood vessel image is used to increase the brightness of the region near the center line of the blood vessel image, thereby dealing with brightness unevenness in the image of thick blood vessels.

[0231] 9, similar to steps S21 to S23 in FIG. 8A, the ophthalmologic apparatus 1 acquires an OCT angiography image of the fundus Ef of the subject's eye E (S41), applies maximum intensity projection to this OCT angiography image to generate a maximum intensity projection image (S42), and applies Otsu's binarization to this maximum intensity projection image to generate a binary image in which images of thick blood vessels are depicted (S43). The generated binary image is stored in the storage unit 212 by the main control unit 211.

[0232] Next, the ophthalmologic apparatus 1 uses the noise removal processing unit 1300 to analyze the thick blood vessel image extracted in the binary image generated in step S43, determine the center line of the thick blood vessel image, and calculate the luminance distribution relative to the center line (S44). The process of determining the center line of the thick blood vessel image may include, for example, thinning processing or skeletonization processing. The luminance distribution relative to the center line of the thick blood vessel image is information that associates the distance from the center line with the luminance value, and is a map that represents the distribution of luminance values ​​in the thick blood vessel image with the position of the center line as a reference. Note that the method of representing the luminance distribution in the thick blood vessel image is not limited to this example and may be selected or defined arbitrarily.

[0233] Next, the ophthalmologic apparatus 1 applies processing based on the luminance distribution generated in step S44 to the binary image generated in step S43 using the noise removal processing unit 1300 (S45). The image generated in this way is called a processed image. The generated processed image is stored in the storage unit 212 by the main control unit 211.

[0234] Similar to the erode image described above, the processed image depicts reduced blood vessel images in which the diameters of the large blood vessels are reduced. That is, the processing based on the brightness distribution applied to the binary image in step S45 is processing for reducing the diameters of the large blood vessels. This processing is, for example, processing for identifying areas with relatively low brightness near the center lines of the large blood vessels based on the brightness distribution.

[0235] Furthermore, the ophthalmologic apparatus 1 generates a first vessel enhancement image by applying a multiscale Franzi filter to the maximum intensity projection image generated in step S42 using the vessel enhancement processing unit 1200 (S46). The generated vessel enhancement image is stored in the storage unit 212 by the main control unit 211.

[0236] Furthermore, the ophthalmologic apparatus 1 generates a Frangi filter image by applying a Frangi filter having a scale corresponding to a second range smaller than the first range in the blood vessel image extraction process (Otsu's binarization) in step S23 to the maximum intensity projection image generated in step S42 using the blood vessel emphasis processing unit 1200 (S47). The generated Frangi filter image is stored in the storage unit 212 by the main control unit 211.

[0237] Furthermore, the ophthalmologic apparatus 1 generates a second vessel enhancement image by applying gamma correction to the Frangi-filter image generated in step S47 using the vessel enhancement processing unit 1200 to increase the brightness of the vessel image (S48). The generated second vessel enhancement image is stored in the storage unit 212 by the main control unit 211.

[0238] Next, the ophthalmologic apparatus 1 determines a first partial image in the first vessel-enhanced image and a second partial image in the second vessel-enhanced image by applying mask processing based on the reduced blood vessel image depicted in the processed image generated in step S45 to the first vessel-enhanced image generated in step S46 and the second vessel-enhanced image generated in step S48 using the noise removal processing unit 1300 (S49 and S50). The main control unit 211 stores in the storage unit 212 coordinate information in the first partial image or the first vessel-enhanced image corresponding thereto, and coordinate information in the second partial image or the second vessel-enhanced image corresponding thereto.

[0239] Next, the ophthalmologic apparatus 1 generates a noise-removed image by causing the noise removal processing unit 1300 to compare the luminance values ​​of the pixel group constituting the first partial image identified from the first vessel-enhanced image in step S49 with the luminance values ​​of the pixel group constituting the second partial image identified from the second vessel-enhanced image in step S50 (S51). The generated noise-removed image is stored in the storage unit 212 by the main control unit 211.

[0240] More specifically, the ophthalmologic apparatus 1 generates a noise-removed image by selecting the larger luminance value of each pixel of the first partial image in the first vessel-enhanced image or the luminance value of the corresponding pixel of the second partial image in the second vessel-enhanced image using the noise removal processing unit 1300. In this manner, in this operation example, the luminance of the region near the center line of the image of a thick blood vessel is increased, thereby addressing the problem of the region being represented with low luminance.

[0241] Next, the ophthalmologic apparatus 1 generates a composite image by applying image synthesis processing to the noise-removed image generated in step S51, the maximum intensity projection image generated in step S42, and the first vessel-enhanced image generated in step S46 using the image synthesis processing unit 1400 (S52). The generated composite image is stored in the storage unit 212 by the main control unit 211. This completes this operation example (END).

[0242] The ophthalmologic apparatus 1 can display the image acquired or generated in the operation example of FIG. 9, apply processing to the image, transmit the image to an external device, and record the image on a recording medium.

[0243] 8A and 8B, the operation example of FIG. 9 can also eliminate or reduce the problem of uneven brightness occurring in images of large blood vessels. Furthermore, problems that arise when using a Frangi filter with a scale corresponding to a larger blood vessel diameter (such as a blurred blood vessel image or two different blood vessels being depicted as being connected together) do not occur. Furthermore, it is possible to generate a composite image with a texture similar to that of the original image.

[0244] <Second Example> In the second embodiment, several aspects of processing for solving the second problem (missing images of thin blood vessels) caused by the use of a vessel enhancement filter will be described.

[0245] The second problem addressed by the second embodiment is the phenomenon in which images of relatively thin blood vessels that should be clearly depicted in a blood vessel-enhanced image are missing. For example, if images of thin blood vessels that actually exist in the arcade region of the fundus are not depicted, disease may be suspected during image interpretation or a false positive may be determined during analysis. The second embodiment can avoid such situations.

[0246] In the second embodiment, the image projection processing unit 1100 is configured to apply two different types of projection processing to the OCT angiography image acquired by the image acquisition unit 1000. That is, the image projection processing unit 1100 is configured to apply a first projection processing to the OCT angiography image to generate a first projection image, and to apply a second projection processing, different from the first projection processing, to the OCT angiography image to generate a second projection image.

[0247] In some embodiments, the first projection process is maximum intensity projection, and the second projection process is average intensity projection. Maximum intensity projection has excellent vascular visualization capabilities and is widely used in various angiography methods, including OCT angiography. However, maximum intensity projection has the disadvantage of being prone to noise contamination. Average intensity projection has the characteristic of being less prone to noise contamination and has a noise reduction effect. That is, in the second embodiment, average intensity projection functions as both an image projection process and a noise removal process.

[0248] In the second embodiment, the vascular enhancement processing unit 1200 is configured to generate a first vascular enhancement image by applying a multiscale Franzi filter to a first projection image generated from an OCT angiography image by a first projection process.

[0249] The vessel enhancement processor 1200 is also configured to apply a Frangi filter with a scale corresponding to the range of diameters of capillaries to the second projection image generated from the OCT angiography image by the second projection processing. The image generated from the second projection image by this Frangi filter is called a Frangi filter image.

[0250] Furthermore, the vessel enhancement processing unit 1200 is configured to generate a second vessel enhancement image by applying gamma correction to the Frangi-filtered image to increase the brightness of the vessel image. In an embodiment in which a projection method having a noise reduction effect, such as average projection, is used in the second projection processing, the second vessel enhancement image generated by the second projection processing and the Frangi-filter may be treated as a noise-removed image generated by the noise removal processing unit 1300.

[0251] In a second embodiment, the image synthesis processor 1400 is configured to generate a synthesized image by applying an image synthesis process to a first projection image generated from the OCT angiography image by a first projection process, a first vascular enhancement image generated from the first projection image by a multiscale Frangi filter, and a second vascular enhancement image generated by applying a Frangi filter and gamma correction to a second projection image generated from the OCT angiography image by a second projection process. The image synthesis process may be performed using alpha blending or another image synthesis method.

[0252] In the second embodiment, by executing a series of processes that can be realized with such a configuration, it is possible to generate a vessel-enhanced OCT angiography image in which images of thin blood vessels (such as capillaries) are not lost.

[0253] Next, the operation of the ophthalmologic apparatus 1 in the second embodiment will be described with reference to FIGS. 10A and 10B.

[0254] 10A and 10B, first, the ophthalmologic apparatus 1 acquires an OCT angiography image 4000 of the fundus Ef of the subject's eye E by the image acquisition unit 1000 (S61). The acquired OCT angiography image 4000 is stored in the storage unit 212 by the main control unit 211.

[0255] After step S61, two series of processes are executed: steps S62 and S63 (reference numerals 4010 to 4040 in FIG. 10B) and steps S64 to S66 (reference numerals 4050 to 4100 in FIG. 10B). The order in which these two series of processes are executed may be arbitrary. Furthermore, the two series of processes may be executed at least partially in parallel.

[0256] The ophthalmologic apparatus 1 generates a maximum intensity projection image 4020 by applying a maximum intensity projection 4010 to the OCT angiography image 4000 using the image projection processing unit 1100 (S62). The generated maximum intensity projection image 4020 is stored in the storage unit 212 by the main control unit 211.

[0257] Next, the ophthalmologic apparatus 1 generates a first vessel-enhanced image 4040 by applying a multiscale Franzi filter 4030 to the maximum intensity projection image 4020 using the vessel-enhancement processing unit 1200 (S63). The generated first vessel-enhanced image 4040 is stored in the storage unit 212 by the main control unit 211.

[0258] The ophthalmologic apparatus 1 generates an average value projection image 4060 by applying the average value projection 4050 to the OCT angiography image 4000 using the image projection processing unit 1100 (S64). The generated average value projection image 4060 is stored in the storage unit 212 by the main control unit 211.

[0259] Next, the ophthalmologic apparatus 1 generates a Frangi filter image 4080 in which the capillaries are depicted by applying a Frangi filter 4070 having a scale corresponding to the range of diameter dimensions of the capillaries to the average value projection image 4060 by the blood vessel enhancement processing unit 1200 (S65). The generated Frangi filter image 4080 is stored in the storage unit 212 by the main control unit 211.

[0260] Furthermore, the ophthalmologic apparatus 1 generates a second vessel enhancement image 4100 by applying gamma correction 4090 to the Frangi filter image 4080 using the vessel enhancement processing unit 1200 to increase the brightness of the vessel image (S66). The generated second vessel enhancement image 4100 is stored in the storage unit 212 by the main control unit 211.

[0261] Next, the ophthalmologic apparatus 1 generates a composite image 4120 by applying image synthesis processing 4110 to the first blood vessel emphasis image 4040, the second blood vessel emphasis image 4100, and the maximum intensity projection image 4020 using the image synthesis processing unit 1400 (S67). The generated composite image 4120 is stored in the storage unit 212 by the main control unit 211. This completes this operation example (END).

[0262] The ophthalmologic device 1 can display the images acquired or generated in the operational examples of Figures 10A and 10B, apply processing to the images, transmit the images to an external device, and record the images on a recording medium.

[0263] The ophthalmologic apparatus 1 according to the second embodiment generates first and second projection images from an OCT angiography image using two different image projection methods. For example, a maximum intensity projection image (first projection image) with excellent blood vessel visualization capability and a low-noise average value projection image (second projection image) are generated.

[0264] The ophthalmic device 1 generates a first vessel-enhanced image depicting images of blood vessels of various thicknesses by applying a multi-scale Frangi filter to the first projection image, and generates a Frangi filter image in which images of thin blood vessels are emphasized by applying a Frangi filter of a specific scale to the second projection image.

[0265] Thus, although the Frangi filter image emphasizes thin blood vessels, the signal intensity representing the thin blood vessels is low. Therefore, maximum intensity projection may select noise with a higher signal intensity than the thin blood vessels. On the other hand, average projection averages signal intensity in the depth direction, so this problem is less likely to occur. Therefore, the second projection image generated from the OCT angiography image using average projection depicts the thin blood vessels in the OCT angiography image with high fidelity. Furthermore, the Frangi filter image generated from the second projection image using a Frangi filter of a specific scale also depicts the thin blood vessels in the OCT angiography image with high fidelity.

[0266] In the second embodiment, gamma correction is applied to the Frangi-filter image to increase the brightness (contrast) of the image of thin blood vessels depicted in the Frangi-filter image and make them clearer.

[0267] Furthermore, in the second embodiment, a Frangi-filtered image (a second vessel-enhanced image used as a noise-removed image) in which thin blood vessels have been made clear by gamma correction is synthesized with a first vessel-enhanced image generated by applying a multiscale Frangi-filter to an OCT angiography image. This allows missing or unclear thin blood vessels in the first vessel-enhanced image generated using maximum intensity projection to be compensated for by the clearly depicted thin blood vessels in the second vessel-enhanced image.

[0268] In addition, in the second embodiment, by combining not only the second vascular enhancement image as a noise-removed image but also the first projection image (e.g., a maximum intensity projection image with high vascular visualization capabilities) with the first vascular enhancement image generated using the multi-scale Franzi filter, it is possible to generate a composite image having a texture close to that of the original image.

[0269] As described above, in some embodiments, maximum intensity projection is used as the first projection process and average intensity projection is used as the second projection process, but the two different projection processes are not limited to this combination. For example, the first projection process may be performed using any type of image projection method that has excellent blood vessel visualization capabilities, and / or the second projection process may be performed using any type of image projection method that has excellent noise reduction capabilities.

[0270] In some aspects, the processing according to the second embodiment may be applied to a specific region of the fundus as a target. Since the second embodiment aims to eliminate the lack of images of thin blood vessels, the target may be a region where capillaries exist, and in particular, a region where many capillaries exist.

[0271] A non-limiting example of a target blood vessel is the radial peripapillary capillary (RPC). The radial peripapillary capillary is a retinal blood vessel originating from the central retinal artery and located in the outermost layer of the retina. In this example, the OCT angiography image acquired by the image acquisition unit 1000 is an image depicting a region of the fundus Ef including the radial peripapillary capillary. Such an image can be acquired by applying an OCT scan to a region of the fundus Ef including the radial peripapillary capillary.

[0272] Even when blood vessels other than the radial peripapillary capillaries are targeted, the processing according to the second embodiment and the processing for acquiring OCT angiography images can be performed in the same manner as described above.

[0273] <Third Example> In the third embodiment, several aspects of processing for eliminating the third problem (noise in images of avascular regions) caused by the use of a vessel enhancement filter will be described.

[0274] The third problem addressed by the third embodiment is the phenomenon in which noise generated in areas of the fundus where blood vessels are not supposed to exist is emphasized by a blood vessel emphasis filter, resulting in an image that looks like a blood vessel. This phenomenon has been confirmed to be particularly noticeable in the foveal avascular zone (FAZ).

[0275] In the third embodiment, the blood vessel enhancement processing unit 1200 is configured to generate a blood vessel enhancement image by applying a multiscale Frangi filter to a projection image generated from an OCT angiography image by the image projection processing unit 1100.

[0276] In some embodiments, the projection image is an image generated using a projection method that has excellent vascular visualization capabilities, and may be, for example, a maximum intensity projection image generated using maximum intensity projection.

[0277] In the third embodiment, the noise removal processing unit 1300 is configured to apply an avascular region identification process to the vascular enhancement image generated by the vascular enhancement processing unit 1200 in order to identify an avascular region image corresponding to the avascular region of the fundus Ef of the subject's eye E.

[0278] The avascular region to be identified may be a predetermined area (e.g., the foveal avascular region) or an area set based on a vascular enhancement image (e.g., an area corresponding to an image area where the density of blood vessel images is below a predetermined threshold).

[0279] The avascular region identification process may include any process. In some embodiments, the process described in International Publication No. 2019 / 203056 may be used. In some embodiments, the process described in the following exemplary embodiments and operational examples may be used.

[0280] In the third embodiment, the noise removal processing unit 1300 is further configured to generate a noise-removed image by applying mask processing based on the avascular region image identified by the avascular region identification processing to the vascular enhancement image generated by the vascular enhancement processing unit 1200. Non-limiting examples of the mask processing will be described later.

[0281] In the third embodiment, the image synthesis processor 1400 is configured to generate a synthesized image by applying image synthesis processing to a noise-removed image generated by the noise removal processor 1300 by applying mask processing based on the avascular region image to the vascular enhancement image, and a projection image generated from the OCT angiography image by the image projection processor 1100. The image synthesis processing may be performed using alpha blending or another image synthesis method.

[0282] In some aspects, the noise removal processing unit 1300 is configured to perform a first filtering process in the avascular region identification process, which applies a dispersion filter to the vascular enhancement image generated by the vascular enhancement processing unit 1200.

[0283] Additionally, in some embodiments, the noise removal processor 1300 is configured to determine a first brightness threshold based on a dispersion-filtered image generated from the vascular enhancement image by a first filtering process using a dispersion filter in the avascular region identification process. This process is referred to as a first brightness threshold determination process. Furthermore, the noise removal processor 1300 is configured to generate a first mask image by applying threshold processing using the first brightness threshold determined in the first brightness threshold determination process to a dispersion-filtered image generated from the vascular enhancement image by a first filtering process using a dispersion filter in the avascular region identification process. This process is referred to as a first threshold processing.

[0284] In some embodiments, the noise removal processor 1300 is configured to perform a second filtering process in the avascular region identification process, which applies a mean filter to the projection image generated from the OCT angiography image by the image projection processor 1100.

[0285] In some embodiments, the noise removal processing unit 1300 is configured to perform two filter processes in the avascular region identification process: a first filter process that applies a variance filter to the vascular enhancement image generated by the vascular enhancement processing unit 1200, and a second filter process that applies an average filter to the projection image generated by the image projection processing unit 1100.

[0286] In some embodiments, the noise removal processing unit 1300 is configured to perform, in the avascular region identification processing, a first brightness threshold determination processing that determines a first brightness threshold based on a variance filter image generated by the first filter processing, a first threshold processing that applies threshold processing using the first brightness threshold to the variance filter image to generate a first mask image, and a second filter processing that applies an average filter to the projection image generated by the image projection processing unit 1100.

[0287] In addition, in some embodiments, the noise removal processing unit 1300 is configured to perform, in the avascular region identification process, a second brightness threshold determination process that determines a second brightness threshold based on the average filter image generated by the second filter process, and a second threshold process that applies threshold process using the second brightness threshold to the average filter image to generate a second mask image.

[0288] Furthermore, in some aspects, the noise removal processing unit 1300 is configured to generate a composite mask image by combining the first mask image and the second mask image in the avascular region identification process, and generate an avascular region image based on the composite mask image.

[0289] Furthermore, in some aspects, the noise removal processor 1300 is configured to generate a sum image of the first mask image and the second mask image as a composite mask image in the avascular region identification process.

[0290] Additionally, in some embodiments, the noise removal processing unit 1300 is configured to generate an avascular region by applying a Gaussian filter to a sum image of the first mask image and the second mask image in the avascular region identification process.

[0291] More generally, in some embodiments, the noise removal processing unit 1300 may be configured to generate an avascular region in the avascular region identification process by applying a Gaussian filter to a composite mask image generated by combining the first mask image and the second mask image.

[0292] More generally, in some aspects, the noise removal processor 1300 may be configured to generate an avascular region image using a Gaussian filter in the avascular region identification process.

[0293] In the third embodiment, by executing a series of processes that can be realized with such a configuration, it is possible to generate a vascular-enhanced OCT angiography image that does not have the problem of noise in the image of the avascular region.

[0294] Next, the operation of the ophthalmologic apparatus 1 in the third embodiment will be described with reference to FIGS. 11A and 11B.

[0295] 11A and 11B, first, the ophthalmologic apparatus 1 acquires an OCT angiography image 5000 of the fundus Ef of the subject's eye E by the image acquisition unit 1000 (S71). The acquired OCT angiography image 5000 is stored in the storage unit 212 by the main control unit 211.

[0296] Next, the ophthalmologic apparatus 1 generates a maximum intensity projection image 5020 by applying a maximum intensity projection 5010 to the OCT angiography image 5000 using the image projection processing unit 1100 (S72). The generated maximum intensity projection image 5020 is stored in the storage unit 212 by the main control unit 211. In this operation example, maximum intensity projection, which has excellent blood vessel visualization capabilities, is used, but another image projection method may also be used.

[0297] After step S72, two series of processes are executed: steps S73 to S76 (reference numerals 5030 to 5100 in FIG. 11B) and steps S77 to S79 ​​(reference numerals 5110 to 5160 in FIG. 10B). The order in which these two series of processes are executed may be arbitrary. Furthermore, the two series of processes may be executed at least partially in parallel.

[0298] The ophthalmologic apparatus 1 generates a blood vessel enhancement image 5040 by applying a multiscale Franzi filter 5030 to the maximum intensity projection image 5020 using the blood vessel enhancement processing unit 1200 (S73). The generated blood vessel enhancement image 5040 is stored in the storage unit 212 by the main control unit 211.

[0299] The vessel-enhanced image 5040 generated using the multiscale Frangi filter 5030 depicts images of blood vessels of various sizes, as well as an image of the foveal avascular zone (FAZ). The image region in the vessel-enhanced image 5040 corresponding to the foveal avascular zone (FAZ) is called the avascular zone image. The avascular zone image of the vessel-enhanced image 5040 contains noise due to the enhancement of noise signals. If a noise signal with gradient information similar to blood vessels is included in the original image (OCT angiography image 5000, maximum intensity projection image 5020), this noise signal is enhanced by the multiscale Frangi filter 5030 and appears as blood vessel-like noise within the avascular zone image of the vessel-enhanced image 5040. This noise remains enhanced even when the Frangi filter scale is adjusted. In this example, the following series of processes are performed to address this noise.

[0300] The ophthalmologic apparatus 1 generates a dispersion filtered image 5060 by applying a dispersion filter 5050 to the blood vessel emphasis image 5040 using the noise removal processing unit 1300 (S74). The generated dispersion filtered image 5060 is stored in the storage unit 212 by the main control unit 211.

[0301] The identification of an avascular region image by the dispersion filter 5050 is a filter process that utilizes the characteristic that the magnitude of the dispersion value of an area where a vascular signal is present is different from the magnitude of the dispersion value of an area where a vascular signal is not present, and is performed to prevent vascular enhancement from being reflected in the foveal avascular zone (FAZ). The dispersion filter 5050 acts to identify image areas where there are no vascular signals in the vascular enhancement image 5040. In this operation example, an avascular region image is detected in the vascular enhancement image 5040. The dispersion filter image 5060 generated by applying the dispersion filter 5050 to the vascular enhancement image 5040 includes an avascular region image.

[0302] If the size of the dispersion filter 5050 is too small, not only the avascular region image but also small regions will be detected and masked in subsequent processing. In order to selectively detect the avascular region image, a dispersion filter 5050 of an appropriate size is prepared. Alternatively, the size of the dispersion filter 5050 may be determined according to the size of the image (OCT angiography image 5000, maximum intensity projection image 5020, vascular enhancement image 5040, etc.), or the size of the dispersion filter 5050 may be determined according to the object depicted in the image.

[0303] Next, the ophthalmologic apparatus 1 causes the noise removal processing unit 1300 to execute a first brightness threshold determination process 5070 for determining a first brightness threshold 5080 based on the dispersion filtered image 5060 (S75). The determined first brightness threshold 5080 is stored in the storage unit 212 by the main control unit 211.

[0304] In the dispersion filter image 5060, a signal (vascular signal) of a certain intensity is present in an area where blood vessels exist (vascular area), and no vascular signal is present in an area where blood vessels do not exist (background area). In the first brightness threshold determination process 5070, for example, a histogram of the brightness values ​​of the dispersion filter image 5060 is created, and a threshold for distinguishing between signals corresponding to blood vessels (areas corresponding to blood vessels) and signals not corresponding to blood vessels (areas not corresponding to blood vessels) is determined based on this histogram. This threshold is used as the first brightness threshold 5080.

[0305] Next, the ophthalmologic apparatus 1 generates a first mask image 5100 by applying a first threshold process 5090 using a first brightness threshold 5080 to the dispersion filter image 5060 by the noise removal processing unit 1300 (S76). The generated first mask image 5100 is stored in the storage unit 212 by the main control unit 211.

[0306] The first mask image 5100 provides a mask that shields the range of the avascular region image determined by the processing of steps S74 to S76 (reference numerals 5050 to 5090 in FIG. 11B).

[0307] In some aspects, the processes of steps S81 to S83 may be performed using the first mask image 5100 generated in steps S74 to S76, without performing the processes of steps S77 to S80. Advantages of this method include simplification and reduction in processing time. On the other hand, a disadvantage is that the robustness of the process for identifying an avascular region image may be reduced. In this operation example, the robustness of the process for identifying an avascular region image is improved by performing the processes of steps S77 to S80.

[0308] The ophthalmologic apparatus 1 generates an average filtered image 5120 by applying the average filter 5110 to the maximum intensity projection image 5020 generated in step S72 (S77). The generated average filtered image 5120 is stored in the storage unit 212 by the main control unit 211.

[0309] Identification of an avascular region image by the averaging filter 5110 is a filtering process that utilizes the characteristic that the signal intensity of a vascular region differs from that of a background region, and is performed to prevent vascular enhancement from being reflected in the foveal avascular zone (FAZ), similar to identification of an avascular region image by the variance filter 5050. The averaging filter 5110 acts to identify image regions in the vascular enhancement image 5040 that are free of vascular signals. In this operation example, an avascular region image is detected in the vascular enhancement image 5040. The averaging filter image 5120 generated by applying the averaging filter 5110 to the vascular enhancement image 5040 includes an avascular region image.

[0310] In subsequent processing, a lower limit value for brightness is set to mask low-brightness areas. The lower limit value for brightness may be set in advance, or may be set according to the size of the image (OCT angiography image 5000, maximum intensity projection image 5020, etc.), or may be set according to the objects depicted in the image. In a typical embodiment, the size of the average filter 5110 is equal to the size of the variance filter 5050. However, the sizes of the variance filter 5050 and the average filter 5110 may be different from each other.

[0311] Next, the ophthalmologic apparatus 1 causes the noise removal processing unit 1300 to execute a second brightness threshold determination process 5130 for determining a second brightness threshold 5140 based on the average filtered image 5120 (S78). The determined second brightness threshold 5140 is stored in the storage unit 212 by the main control unit 211.

[0312] In the average filtered image 5120, a blood vessel signal of a certain intensity exists in the blood vessel region, and no blood vessel signal exists in the background region. In the second brightness threshold determination process 5130, for example, a histogram of the brightness values ​​of the average filtered image 5120 is created, and a threshold for distinguishing between the blood vessel signal (blood vessel region) and the background signal (background region) is determined based on this histogram. This threshold is used as the second brightness threshold 5140.

[0313] Next, the ophthalmologic apparatus 1 generates a second mask image 5160 by applying a second threshold process 5150 using a second brightness threshold 5140 to the average filtered image 5120 by the noise removal processing unit 1300 (S79). The generated second mask image 5160 is stored in the storage unit 212 by the main control unit 211.

[0314] The second mask image 5160 provides a mask that shields the range of the avascular region image determined by the processing of steps S77 to S79 ​​(reference numerals 5110 to 5150 in FIG. 11B).

[0315] In this manner, in this operation example, two mask images are obtained: a first mask image 5100 representing the range of the avascular region image determined by the processing of steps S74 to S76 (reference numerals 5050 to 5090 in FIG. 11B ), and a second mask image 5160 representing the range of the avascular region image determined by the processing of steps S77 to S79 ​​(reference numerals 5110 to 5150 in FIG. 11B ). Because these two mask images are generated by different processes, the range of the avascular region image represented by the first mask image 5100 and the range of the avascular region image represented by the second mask image 5160 generally differ from each other. In this operation example, by using both of the two mask images 5100 and 5160, the robustness of the process of identifying the avascular region image is improved.

[0316] After completing the two series of processes, steps S73 to S76 (reference numerals 5030 to 5100 in FIG. 11B) and steps S77 to S79 ​​(reference numerals 5110 to 5160 in FIG. 10B), the ophthalmologic apparatus 1 generates a composite mask image 5180 by applying a synthesis process 5170 to the first mask image 5100 and the second mask image 5160 using the noise removal processing unit 1300 (S80). The generated composite mask image 5180 is stored in the storage unit 212 by the main control unit 211.

[0317] In this operation example, a logical OR operation is performed on the first mask image 5100 and the second mask image 5160 in the synthesis process 5170. That is, an image (sum image) as the union of the first mask image 5100 and the second mask image 5160 is generated as a synthesized mask image 5180. As a result, a synthesized mask image 5180 is obtained that shows the region determined to be an avascular region image by either or both of the variance filter 5050 and the mean filter 5110.

[0318] In some embodiments, another operation can be performed in the synthesis process 5170. For example, in the synthesis process 5170, a logical AND operation between the first mask image 5100 and the second mask image 5160 may be performed to generate a composite mask image 5180, which is an image (product image) that is the intersection of the first mask image 5100 and the second mask image 5160. In this case, the composite mask image 5180 is obtained, which shows the region determined to be an avascular region image by both the variance filter 5050 and the mean filter 5110. Note that the synthesis process 5170 may be configured to selectively perform multiple operations.

[0319] Next, the ophthalmologic apparatus 1 causes the noise removal processing unit 1300 to apply a Gaussian filter 5190 to the composite mask image 5180 (S81).

[0320] The contours (boundaries) of the composite mask image 5180 generated using threshold processing (5090 and 5150) are sharp, but the boundaries can be smoothed (blurred) by using a Gaussian filter 5190. The composite mask image 5180 to which the Gaussian filter 5190 has been applied is called a smoothed composite mask image 5200. The smoothed composite mask image 5200 is stored in the storage unit 212 by the main control unit 211. Note that application of the Gaussian filter 5190 may be optional.

[0321] Next, the ophthalmologic apparatus 1 generates a noise-removed image 5220 by executing mask processing 5210, which applies the smoothed composite mask image 5200 to the blood vessel enhancement image 5040, using the noise removal processing unit 1300 (S82). The generated noise-removed image 5220 is stored in the storage unit 212 by the main control unit 211. The noise-removed image 5220 is the blood vessel enhancement image 5040 in which a mask has been applied to an area corresponding to the smoothed composite mask image 5200.

[0322] Next, the ophthalmologic apparatus 1 generates a composite image 5240 by applying image synthesis processing 5230 to the noise-removed image 5220 and the maximum intensity projection image 5020 using the image synthesis processing unit 1400 (S83).

[0323] In the image synthesis process 5230, for a mask region based on the smoothed synthesis mask image 5200, a corresponding region in the maximum intensity projection image 5020 is used, and for a region other than the mask region, the blood vessel emphasis image 5040 and the maximum intensity projection image 5020 are synthesized. The generated synthesis image 5240 is stored in the storage unit 212 by the main control unit 211. This completes this operation example (END).

[0324] The ophthalmic device 1 can display the images acquired or generated in the operation examples of Figures 11A and 11B, apply processing to the images, transmit the images to an external device, and record the images on a recording medium.

[0325] The ophthalmologic apparatus 1 according to the third embodiment is configured to identify an avascular region image from an OCT image to be processed and not apply vascular enhancement by the multiscale Frangi filter to this avascular region image. In other words, the ophthalmologic apparatus 1 according to the third embodiment is configured to apply vascular enhancement by the multiscale Frangi filter only to regions excluding the avascular region image. This prevents noise signals present inside the avascular region image from being enhanced by the multiscale Frangi filter, thereby solving the third problem (noise in the image of the avascular region). On the other hand, for portions other than the avascular region image, blood vessels can be enhanced by the multiscale Frangi filter. Therefore, the vascular enhancement effect of the multiscale Frangi filter can be enjoyed in portions where blood vessels should be enhanced, while the vascular enhancement effect of the multiscale Frangi filter can be prevented from being applied to portions where blood vessels should not be enhanced.

[0326] Furthermore, in the third embodiment, as described above, various measures are taken to identify avascular region images. For example, one of the significant effects of the third embodiment is that robustness can be improved by using both a variance filter and an average filter. By using both a variance filter and an average filter, for example, regions that are actually blood vessels but are determined not to be blood vessels in processing using the variance filter can be excluded by using the average filter.

[0327] Furthermore, according to the third embodiment, by synthesizing a projection image with a noise-removed image generated from the first vessel-enhanced image, it is possible to generate a synthesized image having a texture similar to that of the original image.

[0328] 12 shows another example of the operation of the ophthalmologic apparatus 1. In this example of operation, instead of performing mask processing based on the avascular region image identified using the above-described filter processing (variance filter, average filter), the problem of noise in the avascular region image is addressed by performing mask processing based on a region with a relatively high density of blood vessels (referred to as a high-density vascular region).

[0329] 12, similar to steps S71 to S73 in Fig. 11A, the ophthalmologic apparatus 1 acquires an OCT angiography image of the fundus Ef of the subject's eye E (S91), applies maximum intensity projection to this OCT angiography image to generate a maximum intensity projection image (S92), and applies a multiscale Franzi filter to this maximum intensity projection image to generate a blood vessel enhancement image (S93). The generated blood vessel enhancement image is stored in the storage unit 212 by the main controller 211.

[0330] Next, the ophthalmologic apparatus 1 applies processing for identifying a high-density vascular region of the fundus oculi Ef (high-density vascular region identification processing) to the maximum intensity projection image generated in step S92 or the vessel-enhanced image generated in step S93 by the noise removal processing unit 1300 (S94). An image corresponding to the high-density vascular region of the fundus oculi Ef identified from the maximum intensity projection image or the vessel-enhanced image, or position information of the image of the identified high-density vascular region in the maximum intensity projection image or the vessel-enhanced image, is stored in the storage unit 212 by the main control unit 211.

[0331] The process for identifying a high-density blood vessel region may be any process, for example, in some embodiments, the process for identifying a high-density blood vessel region may include binarization, segmentation, a variance filter, and the like.

[0332] Next, the ophthalmologic apparatus 1 generates a mask image based on the image of the high-density blood vessel region identified in step S94 using the noise removal processing unit 1300 (S95). The generated mask image is stored in the storage unit 212 by the main control unit 211.

[0333] The process of generating a mask image from an image of a high-density vascular region may be any process. For example, in consideration of the fact that signals present in the vicinity (surrounding) of (reliable) signals in a high-density vascular region can also be estimated to be vascular signals corresponding to blood vessels, the noise removal processing unit 1300 may be configured to generate a mask image by expanding the image of the high-density vascular region. The area outside the expanded area of ​​the image of the high-density vascular region can be considered as the avascular region described above.

[0334] Next, the ophthalmologic apparatus 1 generates a noise-removed image by applying mask processing using the mask image generated in step S95 to the blood vessel enhancement image generated in step S93 using the noise removal processing unit 1300 (S96). The generated noise-removed image is stored in the storage unit 212 by the main control unit 211.

[0335] The masking process in step S96 applies the vascular enhancement effect of the multiscale Frangi filter to the dilated region of the image of the high-density vascular region in the vascular enhancement image, while preventing the vascular enhancement effect of the multiscale Frangi filter from being applied to the region outside the dilated region (avascular region). Therefore, noise inside the avascular region is not enhanced. The noise-removed image generated in step S96 is an image in which only the blood vessels in the dilated region of the high-density vascular region are enhanced, and the avascular region is masked.

[0336] Next, the ophthalmologic apparatus 1 generates a composite image by applying image synthesis processing to the noise-removed image generated in step S96 and the maximum intensity projection image generated in step S92 using the image synthesis processing unit 1400 (S97). The generated composite image is stored in the storage unit 212 by the main control unit 211. This completes this operation example (END).

[0337] The ophthalmologic apparatus 1 can display the image acquired or generated in the operation example of FIG. 12, apply processing to the image, transmit the image to an external device, and record the image on a recording medium.

[0338] 11A and 11B, the example of operation in Fig. 12 can also eliminate or reduce the problem of noise in images of avascular regions, and can also generate a synthetic image with a texture that is close to that of the original image.

[0339] In the example of operation shown in FIG. 12, a smoothing process may be performed to smooth (blur) the contours (boundaries) of the mask image using a Gaussian filter.

[0340] The robustness of the processing according to the third embodiment can be improved by combining one or both of the two types of processing (processing using a variance filter and processing using an average filter) in the operational examples of Figures 11A and 11B with the processing using the high-density blood vessel region in the operational example of Figure 12. The processing that can be adopted for this purpose is not limited to the three types of processing given in the operational examples of Figures 11A and 11B and Figure 12, and other processing may also be used.

[0341] <Other Examples> Although various embodiments, aspects, and examples relating to ophthalmic devices have been described above, those skilled in the art will appreciate that the present disclosure also provides embodiments, aspects, and examples relating to categories other than ophthalmic devices.

[0342] Non-limiting examples include the "method for processing ophthalmic images" according to the 38th aspect example described above, and further, a method realized by combining any of the matters described in the various embodiments, aspects, and examples of the ophthalmic device described above with the 38th aspect example.

[0343] Another non-limiting example is the "method for controlling an ophthalmic device" according to the 39th aspect example described above, and further, a method realized by combining any of the matters described in the various embodiments, aspects, and examples of the ophthalmic device described above with the 39th aspect example.

[0344] Yet another non-limiting example is a program that causes a computer to execute each step in the "method for processing ophthalmic images" according to the 38th example embodiment, and further, a program that causes a computer to execute each step in a method realized by combining any of the matters described in the various embodiments, aspects, and examples of the ophthalmic device described above with the 38th example embodiment.

[0345] Yet another non-limiting example is a program that causes a computer to execute each step in the "method for controlling an ophthalmic device" according to the 39th example embodiment, and further, a program that causes a computer to execute each step in a method realized by combining any of the matters described in the various embodiments, aspects, and examples of the ophthalmic device described above with the 39th example embodiment.

[0346] Yet another non-limiting example is a computer-readable non-transitory recording medium having recorded thereon a program that causes a computer to execute each step in the "method for processing ophthalmic images" according to the 38th embodiment, and further, a computer-readable non-transitory recording medium having recorded thereon a program that causes a computer to execute each step in a method realized by combining any of the matters described in the various embodiments, aspects, and examples of the ophthalmic device described above with the 38th embodiment.

[0347] Yet another non-limiting example is a computer-readable non-transitory recording medium having recorded thereon a program that causes a computer to execute each step in the "method for controlling an ophthalmic device" according to the 39th embodiment, and further, a computer-readable non-transitory recording medium having recorded thereon a program that causes a computer to execute each step in a method realized by combining any of the matters described in the various embodiments, aspects, and examples of the ophthalmic device described above with the 39th embodiment.

[0348] The present disclosure is merely an example of how to implement the present invention, and those who intend to implement the present invention can make any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the present invention. [Explanation of symbols]

[0349] 1 Ophthalmology equipment 1000 Image acquisition unit 1100 Image projection processing unit 1200 Blood vessel enhancement processing unit 1300 Noise removal processing unit 1400 Image synthesis processing unit

Claims

1. an image acquisition unit that acquires an optical coherence tomography angiography image of the fundus of the subject's eye; an image projection processing unit that applies projection processing to the optical coherence tomography angiography image to generate a projection image; a vessel enhancement processing unit that applies a vessel enhancement filter for enhancing a vessel image to the projection image to generate a vessel enhancement image; a noise removal processing unit that applies noise removal processing to the projection image to generate a noise-removed image; an image synthesis processing unit that applies image synthesis processing to the projection image, the blood vessel enhancement image, and the noise-removed image to generate a synthesized image; 1. An ophthalmic device comprising:

2. The vessel enhancement filter includes a multi-scale Frangi filter. The ophthalmic device of claim 1.

3. the noise removal processing unit is capable of executing a plurality of processes different from each other in the noise removal processing. The ophthalmic device of claim 1.

4. The noise removal processing unit selecting at least one process from the plurality of processes based on an angle of view of the optical coherence tomography angiography image; generating the denoised image by applying the at least one process to the projection image; the image synthesis processing unit generates the synthesized image by applying the image synthesis processing to the noise-removed image generated by applying the at least one processing to the projection image, the projection image, and the blood vessel enhancement image. The ophthalmic apparatus of claim 3.

5. The noise removal processing unit selecting at least one process from the plurality of processes based on at least one of a fixation position and a scan area for generating the optical coherence tomography angiography image; generating the denoised image by applying the at least one process to the projection image; the image synthesis processing unit generates the synthesized image by applying the image synthesis processing to the noise-removed image generated by applying the at least one processing to the projection image, the projection image, and the blood vessel enhancement image. The ophthalmic apparatus of claim 3.

6. The noise removal processing unit applying a blood vessel image extraction process to the projection image to extract blood vessel images having diameters within a first range; applying an erosion process to the image generated by the blood vessel image extraction process to generate an eroded image in which a reduced blood vessel image is depicted by reducing the diameter dimension of the blood vessel image extracted by the blood vessel image extraction process; The blood vessel emphasis processing unit applying a multi-scale Frangi filter to the projection images to generate a first vessel-enhanced image; applying a Frangi filter having a scale corresponding to a second range less than the first range to the projection image, and applying gamma correction to the image generated by the Frangi filter to increase the brightness of the blood vessel image, thereby generating a second blood vessel-enhanced image; the noise removal processing unit generates the noise-removed image based on the eroded image, the first blood vessel enhancement image, and the second blood vessel enhancement image; the image synthesis processing unit generates the synthesized image by applying the image synthesis processing to the noise-removed image generated from the eroded image, the first blood vessel-enhanced image, and the second blood vessel-enhanced image, the projection image, and the first blood vessel-enhanced image. The ophthalmic device of claim 1.

7. The noise removal processing unit identifying a first partial image in the first vessel-enhanced image, the first partial image corresponding to the reduced vessel image in the eroded image; identifying a second partial image in the second vessel-enhanced image corresponding to the reduced vessel image in the eroded image; generating the noise-removed image by selecting a larger luminance value between a luminance value of each pixel in the first partial image and a luminance value of a corresponding pixel in the second partial image; the image synthesis processing unit generates the synthesized image by applying the image synthesis processing to the noise-removed image generated from the first partial image and the second partial image, the projection image, and the first blood vessel-enhanced image. The ophthalmic device of claim 6.

8. The noise removal processing unit determining the first partial image by applying a mask process based on the reduced blood vessel image in the eroded image to the first blood vessel-enhanced image; determining the second sub-image by applying the masking process to the second vessel-enhanced image; The ophthalmic device of claim 7.

9. The noise removal processing unit applying a blood vessel image extraction process to the projection image to extract blood vessel images having diameters within a first range; Analyzing the blood vessel image extracted by the blood vessel image extraction process to determine a center line of the blood vessel image; A luminance distribution with respect to the distance from the center line is obtained; processing the image generated by the blood vessel image extraction process based on the luminance distribution to generate a processed image in which a reduced blood vessel image is depicted, the diameter dimension of which is reduced from the blood vessel image extracted by the blood vessel image extraction process; The blood vessel emphasis processing unit applying a multi-scale Frangi filter to the projection images to generate a first vessel-enhanced image; applying a Frangi filter having a scale corresponding to a second range less than the first range to the projection image, and applying gamma correction to the image generated by the Frangi filter to enhance the brightness of the blood vessel image, thereby generating a second blood vessel-enhanced image; the noise removal processing unit generates the noise-removed image based on the processed image, the first blood vessel enhancement image, and the second blood vessel enhancement image; the image synthesis processing unit generates the synthesized image by applying the image synthesis processing to the noise-removed image generated from the processed image, the first blood vessel enhancement image, and the second blood vessel enhancement image, the projection image, and the first blood vessel enhancement image. The ophthalmic device of claim 1.

10. The noise removal processing unit identifying a first partial image in the first vessel-enhanced image corresponding to the reduced vessel image in the processed image; identifying a second partial image in the second vessel-enhanced image, the second partial image corresponding to the reduced vessel image in the processed image; generating the noise-removed image by selecting a larger luminance value between a luminance value of each pixel in the first partial image and a luminance value of a corresponding pixel in the second partial image; the image synthesis processing unit generates the synthesized image by applying the image synthesis processing to the noise-removed image generated from the first partial image and the second partial image, the projection image, and the first blood vessel-enhanced image. The ophthalmic device of claim 9.

11. The image projection processing unit applying a first projection process to the optical coherence tomography angiography image to generate a first projection image; applying a second projection process different from the first projection process to the optical coherence tomography angiography image to generate a second projection image; The blood vessel emphasis processing unit applying a multi-scale Frangi filter to the first projection image to generate a first vessel-enhanced image; applying a Frangi filter having a scale corresponding to the range of diameter dimensions of capillaries to the second projection image, and applying gamma correction to the image generated by the Frangi filter to increase the brightness of the blood vessel image, thereby generating a second blood vessel-enhanced image as the noise-removed image; the image synthesis processing unit generates the synthesized image by applying the image synthesis processing to the first projection image, the first blood vessel enhancement image, and the second blood vessel enhancement image; The ophthalmic device of claim 1.

12. the first projection process is a maximum intensity projection; The second projection process is a mean value projection. The ophthalmic device of claim 11.

13. the vessel enhancement processing unit applies a multiscale Frangi filter to the projection image to generate a vessel enhancement image; The noise removal processing unit applying an avascular region identification process to the blood vessel-enhanced image to identify an avascular region image corresponding to the avascular region of the fundus; generating the noise-removed image by applying a mask process to the blood vessel-enhanced image based on the avascular region image identified by the avascular region identifying process; the image synthesis processing unit generates the synthesized image by applying the image synthesis processing to the noise-removed image generated by applying the mask processing based on the avascular region image to the blood vessel-enhanced image and the projection image. The ophthalmic device of claim 1.

14. the noise removal processing unit executes a first filtering process in the avascular region specifying process, in which a dispersion filter is applied to the blood vessel enhancement image. The ophthalmic device of claim 13.

15. The noise removal processing unit, in the avascular region specifying processing, a first brightness threshold determination process for determining a first brightness threshold based on a dispersion filtered image generated by the first filtering process; a first thresholding process applying a thresholding process using the first brightness threshold to the dispersion filtered image to generate a first mask image; To execute The ophthalmic device of claim 14.

16. the noise removal processing unit performs a second filtering process in the avascular region specifying process by applying an average filter to the projection image. The ophthalmic device of claim 15.

17. The noise removal processing unit, in the avascular region specifying processing, a second brightness threshold determination process for determining a second brightness threshold based on the average filtered image generated by the second filtering process; a second thresholding process applying a thresholding process using the second brightness threshold to the mean filtered image to generate a second mask image; To execute The ophthalmic device of claim 16.

18. The noise removal processing unit, in the avascular region specifying processing, combining the first mask image and the second mask image to generate a combined mask image; generating the avascular region image based on the composite mask image; 18. The ophthalmic device of claim 17.

19. the noise removal processing unit generates a sum image of the first mask image and the second mask image as the composite mask image in the avascular region identification processing.

19. The ophthalmic device of claim 18.

20. the noise removal processing unit generates the avascular region image by applying a Gaussian filter to the sum image in the avascular region identification processing.

20. The ophthalmic device of claim 19.

21. the vessel enhancement processing unit applies a multiscale Frangi filter to the projection image to generate a vessel enhancement image; The noise removal processing unit applying a high-density vascular region identification process to the projection image or the vessel-enhanced image to identify a high-density vascular region image corresponding to the high-density vascular region of the fundus; generating the noise-removed image by applying a mask process to the vessel-enhanced image based on the high-density vessel region image identified by the high-density vessel region identification process; the image synthesis processing unit generates the synthesized image by applying the image synthesis processing to the noise-removed image generated by applying the mask processing based on the high-density blood vessel region image to the blood vessel enhancement image and the projection image. The ophthalmic device of claim 1.

22. 1. A method for processing optical coherence tomography angiography images of the fundus of a subject's eye by a computer including a processor, a storage device, and a data input interface, comprising: an input processing step of inputting the optical coherence tomography angiography image into the computer via the data input interface; a storage processing step in which the storage device stores the input optical coherence tomography angiography image; an image projection processing step in which the processor applies a projection process to the optical coherence tomography angiography image stored in the storage device to generate a projection image; a vessel enhancement processing step in which the processor applies a vessel enhancement filter for enhancing a vessel image to the projection image to generate a vessel enhancement image; a noise reduction step in which the processor applies a noise reduction process to the projection image to generate a noise-removed image; an image synthesis processing step in which the processor applies image synthesis processing to the projection image, the blood vessel enhancement image, and the noise-removed image to generate a synthesized image; A method comprising:

23. 1. A method for controlling an ophthalmic device including a processor, a storage device, and an image capture device, comprising: an image acquisition control step of causing the image acquisition device to acquire the optical coherence tomography angiography image; a storage control step of storing the acquired optical coherence tomography angiography image in the storage device; an image projection control step of causing the processor to apply a projection process to the optical coherence tomography angiography image stored in the storage device to generate a projection image; a vessel enhancement control step of causing the processor to apply a vessel enhancement filter for enhancing a vessel image to the projection image to generate a vessel enhancement image; a noise reduction control step of causing the processor to apply a noise reduction process to the projected image to generate a noise-removed image; an image synthesis control step of causing the processor to apply image synthesis processing to the projection image, the blood vessel enhancement image, and the noise-removed image to generate a synthesized image; A method comprising:

Images