Image generation method and image generation system

Abstract

To provide an image generation method and image generation system that can generate vascular images of a subject's eye, which are useful for the early detection of eye diseases and for understanding the lesions of eye diseases. [Solution] The image generation method comprises an image data acquisition step (S01), a first image processing step (S03), a second image processing step (S04), and a vascular system image generation processing step (S05, etc.). The image data acquisition step acquires image data of the eye under examination. The first image processing step extracts the curved structure of blood vessels contained in the image data and generates image data of a vascular-enhanced image 314 that emphasizes the curved structure. The second image processing step classifies the image data of the vascular-enhanced image 314 by the similarity of the brightness intensity of the pixels, groups pixels with common similarity and divides them into superpixels to generate segmented image data of a superpixel image 321. The vascular system image generation processing step generates a vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image 321.

Description

Technical Field

[0001] The present disclosure relates to an image generation method and an image generation system.

Background Art

[0002] The image creation method described in Patent Document 1 has a first step, a second step, and a third step. The first step clusters pixels corresponding to each pixel data based on the optical feature similarity and spatial proximity of the pixel data included in the sample image data (OCT image) to generate superpixels. The second step generates pixel group data representing the superpixel based on the pixel data of each pixel included in the superpixel. The third step generates the plurality of sample cross-sectional images based on the pixel group data.

[0003] The image creation method described in this Patent Document 1 aims to clarify the fundus cross-sectional image by reshaping the superpixels (see paragraph 0062, paragraph 0063, and FIG. 7). Note that OCT is an abbreviation for "Optical Coherence Tomography".

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Previously, vascular images of the eye under examination were mainly obtained by visualizing blood vessels using fluorescein angiography with contrast agents. However, with the current high-precision OCTA imaging, it is now possible to visualize even capillaries in the vascular images of the eye under examination without the use of contrast agents. For this reason, healthcare professionals involved in eye examinations are requesting that OCTA images be processed to generate new vascular images that are useful for the early detection of eye diseases and for understanding lesions in eye diseases, by processing the original images in which the vascular structure is not clearly depicted.

[0006] Here, an OCTA image is created by taking multiple images of the same area of ​​the fundus using an OCT device and imaging only the signals that change among the multiple OCT images. It is created by depicting only the parts of the OCT image that change due to blood flow as blood flow information. OCTA is an abbreviation for "Optical Coherence Tomography Angiography".

[0007] In contrast, the image processing technology described in Patent Document 1 aims to sharpen tomographic images by reshaping OCT images representing tomographic images of the fundus of the eye into superpixels, which are created by grouping multiple pixels based on similarities in features and colors. Therefore, the image processing technology described in Patent Document 1 has the problem of not being able to meet the demand for generating new vascular images that are useful for the early detection of eye diseases and for understanding lesions in eye diseases.

[0008] This disclosure addresses the above-mentioned issues and aims to provide an image generation method and image generation system that can generate vascular images of a subject's eye that are useful for early detection of eye diseases and understanding lesions in eye diseases. [Means for solving the problem]

[0009] To solve the above problems, the present disclosure provides an image generation method for generating a vascular system image of an eye under examination, comprising an image data acquisition step, a first image processing step, a second image processing step, and a vascular system image generation processing step. The image data acquisition step acquires image data of the eye under examination. The first image processing step extracts the curved structure of blood vessels contained in the image data and generates image data of a vascular-enhanced image with the curved structure emphasized. The second image processing step classifies the image data of the vascular-enhanced image by the similarity of the brightness intensity of the pixels, groups pixels with common similarity and divides them into superpixels to generate segmented image data of a superpixel image. The vascular system image generation processing step generates the vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image.

[0010] To solve the above problems, the image generation system for generating a vascular system image of an eye under examination according to this disclosure comprises an image data acquisition unit, a first image processing unit, a second image processing unit, and a vascular system image generation processing unit. The image data acquisition unit acquires image data of the eye under examination. The first image processing unit generates image data of a vascular system-enhanced image by performing a vascular extraction process to extract the curved structure of blood vessels contained in the image data and a vascular enhancement process to emphasize the extracted vascular structure. The second image processing unit classifies the image data of the vascular system-enhanced image by the similarity of the brightness intensity of pixels and generates segmented image data of a superpixel image by performing a superpixel segmentation process to group pixels with common similarity and divide them into superpixels. The vascular system image generation processing unit generates the vascular system image by performing a brightness intensity process on each segmented image data of the superpixel image. [Effects of the Invention]

[0011] The image generation method and image generation system disclosed herein can generate vascular images of a subject's eye, which are useful for the early detection of eye diseases and for understanding lesions in eye diseases. [Brief explanation of the drawing]

[0012] [Figure 1]This is a system configuration diagram showing an example of an image generation system to which the image generation method of Embodiment 1 is applied. [Figure 2] This is an explanatory diagram of the structure of the human eyeball. [Figure 3] This flowchart shows the image processing flow for generating a vascular density map from OCTA image data of the retinal surface slab in Embodiment 1. [Figure 4] This flowchart shows the image processing flow for generating a vascular map from a vascular density map in Embodiment 1. [Figure 5] This flowchart shows the image processing flow for generating a non-blood flow area detection map from the blood vessel density map of Embodiment 1. [Figure 6] This block diagram shows the image workflow from image processing in the image processing unit of Embodiment 1 to the generation of the final image. [Figure 7] This is an explanatory diagram showing the transition of vascular images obtained by vascular extraction and vascular enhancement processing on OCTA image data of the retinal surface slab. [Figure 8] This is an image diagram illustrating the transition of vascular images obtained by superpixel segmentation processing on vascular-enhanced image data. [Figure 9] This is an image diagram illustrating the image transitions used to generate a vascular density map from superpixel images using OCTA original image data through mean / median calculation processing. [Figure 10] This is an image diagram illustrating the image transitions used to generate a vascular density map from superpixel images using OCTA noise reduction image data through mean / median calculation processing. [Figure 11] This is an image diagram illustrating the image transitions when generating a vascular map by calculating a first intensity value using a first threshold from each segmented image data of the vascular density map. [Figure 12] This is an explanatory diagram illustrating the image transitions when generating a non-blood flow region detection map by calculating a second intensity value using a second threshold from each segmented image data of the blood vessel density map. [Figure 13]It is a histogram data diagram showing an example of a superpixel number distribution graph for brightness intensity used to determine the first threshold and the second threshold. [Figure 14] It is a system configuration diagram showing an example of an image generation system to which the image generation method of Embodiment 2 is applied. [Figure 15] It is a flowchart showing the flow of image processing for generating a feature extraction map from the OCTA original image data of the outer retinal slab in Embodiment 2. [Figure 16] It is a block diagram showing the image workflow up to the final image generation by image processing in the image processing unit of Embodiment 2. [Figure 17] It is an image explanatory diagram showing the image transition when generating a feature extraction map from the OCTA original image data or OCTA noise-removed image data of the outer retinal slab.

Embodiments for Carrying Out the Invention

[0013] The embodiments for implementing the image generation method and the image generation system of the present disclosure are the image generation method and the image generation system according to Embodiment 1 and Embodiment 2, and will be described below with reference to the drawings.

[0014] The image generation method and the image generation system of Embodiment 1 and Embodiment 2 are applied to image processing when acquiring OCTA image data from an OCT device and generating a vascular system image of the subject eye useful for early detection of eye diseases and understanding of eye disease lesions from the acquired OCTA image data. Embodiment 1

[0015] [System Configuration (Fig. 1)] Fig. 1 shows an example of an image generation system to which the image generation method of Embodiment 1 is applied. As shown in Fig. 1, the image generation system 100 includes an OCTA image data creation unit 110, a user input device 120, an image processing unit 130, a display 140, and a vascular system image data storage unit 150.

[0016] The OCTA image data creation unit 110 creates OCTA image data based on OCT image data acquired from an OCT device (optical coherence tomography) that has the function of obtaining tomographic data of the fundus. OCTA image data is data obtained by taking multiple images of the same part of the fundus with an OCT device, and then imaging only the signals that have changed from the multiple OCT image data. In other words, since signal changes occur between OCT images in areas where blood is flowing, the OCTA image is a vascular image created by depicting only the signal portions that have changed from the multiple OCT images as blood flow information. The OCTA image data creation unit 110 may be located in the control unit of the OCT device, or it may be located in an external device capable of acquiring OCTA image data from the OCT device.

[0017] One advantage of OCTA imaging is its non-invasive nature, as it does not require contrast agents. It is safe, has no side effects, and requires a short examination time, making repeated examinations possible and suitable for understanding changes in the condition of eye diseases. Another advantage of OCTA imaging is that the cross-sectional image signal from the OCT device is acquired three-dimensionally. By segmenting the cross-sectional image signal, it is possible to observe the vascular structure by unfolding the fundus cross-section layer by layer. Therefore, the OCTA image data creation unit 110 can create OCTA image data (two-dimensional image data) layer by layer, for example, four layers with different depths in the cross-sectional direction, such as the retinal superficial slab, the retinal deep slab, the retinal outer slab, and the choroidal capillary plate slab.

[0018] Here, OCTA image data of the retinal surface slab is obtained by a C-scan method, in which the measurement light is rapidly scanned through the fundus using a galvanometer mirror, and a two-dimensional OCT image is constructed from the interference signal with the optical path of the reference mirror. Furthermore, OCTA image data of the retinal deep slab, retinal outer slab, choroidal capillary plate slab, etc., contain information in the depth direction from the retinal surface slab, and information of each layer located in the depth direction from the retinal surface slab can be obtained by moving the reference mirror along the optical axis.

[0019] The user input device 120 is an input operation device that performs various input operations based on input operations by the user (examiner). These various input operations include, in addition to the start and end operations of image processing, the input operation for selecting the type of layered OCTA image data created by the OCTA image data creation unit 110, and the input operation for selecting the type of vascular image generated by the image processing unit 130. In Embodiment 1, the user input device 120 selects OCTA image data of the retinal surface slab from the OCTA image data created layer by layer by the OCTA image data creation unit 110. In Embodiment 1, the user input device 120 selects one or more vascular images from the vascular density map, vascular map, and non-blood flow area detection map that can be generated by the image processing unit 130, based on the examiner's judgment.

[0020] The image processing unit 130 generates a vascular system image of the eye under examination through multiple stages of image processing based on OCTA image data acquired from the OCTA image data creation unit 110. The image processing unit 130 of Embodiment 1 is capable of generating a vascular density map, a vascular map, and a non-blood flow area detection map based on the original OCTA image data of the retinal surface slab from the OCTA image data creation unit 110.

[0021] The image processing unit 130 of Embodiment 1 includes an image data acquisition unit 131, a noise reduction processing unit 132, a first image processing unit 133, a second image processing unit 134, a third image processing unit 135, a fourth image processing unit 136, and a fifth image processing unit 137. Here, the third image processing unit 135, the fourth image processing unit 136, and the fifth image processing unit 137 correspond to vascular system image generation processing units.

[0022] The image data acquisition unit 131 acquires OCTA original image data of the retinal surface slab as image data of the eye under examination. The noise reduction processing unit 132 generates OCTA noise-reduced image data by performing a noise reduction process to remove noise from the OCTA original image data of the retinal surface slab. The first image processing unit 133 generates image data of a vascular-enhanced image by performing a vascular extraction process to extract the curved structure of blood vessels contained in the OCTA original image data or OCTA noise-reduced image data, and a vascular enhancement process to emphasize the extracted vascular structure. The second image processing unit 134 generates segmented image data of a superpixel image by performing a superpixel segmentation process that classifies the image data of the vascular-enhanced image data by the similarity of the brightness intensity of pixels, and groups pixels with common similarity into superpixels.

[0023] The third image processing unit 135 generates a vascular density map of the retinal surface slab by performing an average / median calculation process (an example of brightness intensity processing) that calculates the average or median brightness intensity for each segmented image data of the superpixel image. The fourth image processing unit 136 determines a first threshold for the vascular density map to classify brightness intensity regions that contain blood vessels with a diameter exceeding a predetermined diameter. This fourth image processing unit 136 generates a vascular map of the retinal surface slab by performing a first intensity value calculation process (an example of brightness intensity processing) that retains image data equal to or greater than the first threshold from each segmented image data of the vascular density map. The fifth image processing unit 137 determines a second threshold for the vascular density map to classify brightness intensity regions that contain capillaries. This fifth image processing unit 137 generates a non-blood flow region detection map of the retinal surface slab by performing a second intensity value calculation process (an example of brightness intensity processing) that retains image data equal to or less than the second threshold from each segmented image data of the vascular density map.

[0024] The display 140 displays the final vascular system image of the retinal surface slab selected by the user through operation on the user input device 120. The display 140 may also display the vascular system image that changes at each stage of image processing when the image processing unit 130 performs image processing. In Embodiment 1, the display 140 displays one or more vascular system images selected by the user from a vascular density map, a vascular map, and a non-blood flow area detection map as the final vascular system image of the retinal surface slab. Therefore, an examiner with specialized knowledge and experience, such as an ophthalmologist, can determine whether or not a patient has an eye disease by looking at the vascular system image displayed on the screen of the display 140, and can also determine the degree of disease progression if an eye disease is determined to be present. Thus, the examiner's determination that the patient is in the early stages of eye disease progression can be used for early detection of eye disease and early treatment of eye disease.

[0025] The vascular image data storage unit 150 stores vascular image data of the retinal surface slab generated by the image processing unit 130. Therefore, the examiner can, for example, operate the user input device 120 to display past and current vascular images of the same patient, read from the vascular image data storage unit 150, side by side on the screen of the display 140. Thus, by comparing past and current vascular images displayed side by side on the display 140, the examiner can use this to understand the progression and improvement of eye disease lesions over time.

[0026] [Detailed structure of the eyeball (Figure 2)] Since the details of the eyeball structure are necessary for understanding the image processing techniques of this disclosure, they will be explained based on Figure 2, which shows the eyeball structure.

[0027] The retina 201 of the human eyeball 200 corresponds to the film or image sensor part of a camera. The cross-sectional structure of the fundus has the retina 201 on the inner surface of the fundus, the choroid 202 outside the retina 201, and the sclera 203 outside the choroid 202. The retina 201 has the optic disc 205 where the optic nerves 204 converge at its center (not the optical center), and is in contact with the vitreous humor 206. The retina 201 has the macula 207, which has high visual acuity, in the part corresponding to the center of the visual field, and is in contact with the vitreous humor 206. When a lesion occurs in the macula 207, the central part of the visual field may appear distorted, or in some cases, objects may appear larger or smaller, or the area being looked at may become dark (central scotoma), directly affecting visual acuity. The retina 201 has the fovea 208 at the center of the macula 207, which is the part of the macula 207 with the best visual acuity, and is slightly concave than the surrounding retina 201. The fovea 208 has a special structure that allows for high visual acuity, as it contains no blood vessels other than cone cells, which are highly sensitive to vision.

[0028] The cornea 209 and lens 210 of the human eyeball 200 correspond to the lens part of a camera. The cornea 209 is the outermost part of the iris, and the lens 210 is located within it. The cornea 209 and lens 210 have very similar functions, and they both play a role in adjusting the refraction of light so that the world we see is not distorted, and projecting it onto the retina 201.

[0029] The retina 201 of the human eyeball 200 has blood vessels 211 in its superficial slab. These blood vessels 211 originate from the ophthalmic artery, which branches directly from the cervical aorta. The ophthalmic artery passes through the optic nerve 204 (central retinal artery) and then branches into four at the optic disc 205 (retinal branching arteries), distributing to the retinal nerve layer. The retinal artery returns via capillaries to four retinal veins (retinal branching veins), which then merge into a single central retinal vein at the optic disc 205, and finally return to the heart as a cervical vein. Therefore, the blood vessels 211 in the superficial slab of the retina 201 have a vascular structure that includes both large blood vessels with a wide diameter and small capillaries with a narrow diameter.

[0030] [Details of image processing (Figures 3-5)] The details of the image processing that generates the vascular density map will be explained based on the flowchart in Figure 3, which shows the flow of image processing performed by the image processing unit 130.

[0031] Step S01 is the step of acquiring the original OCTA image data of the retinal surface slab from the OCTA image data generated by the OCTA image data creation unit 110 as image data of the eye under examination. Step S01 corresponds to the image data acquisition process.

[0032] Step S02 is a step in which noise is removed from the OCTA original image data of the retinal surface slab acquired in Step S01 by noise reduction processing, thereby obtaining OCTA noise-reduced image data. Here, the noise reduction processing mainly targets the removal of speckle noise (such as spot-like noise when laser light is irradiated) and random noise caused by electronic noise inherent in the OCTA original image data. Step S02 corresponds to the noise reduction processing step.

[0033] Note that the noise reduction process in step S02 is not essential for performing the various image processing steps from step S03 onward. Therefore, the various image processing steps from step S03 onward may be performed using either the original OCTA image data or the OCTA noise-reduced image data.

[0034] Step S03 is a step in which the curved structures of blood vessels contained in the OCTA original image data or OCTA noise-reduced image data are extracted by a blood vessel extraction process, and the extracted blood vessel structures are enhanced by an enhancement process to generate image data of a blood vessel-enhanced image. Step S03 corresponds to the first image processing step.

[0035] Step S04 is a step in which the image data of the blood vessel-enhanced image is classified by the similarity of the brightness intensity of pixels, and images of the superpixel image are generated by a superpixel segmentation process in which pixels with common similarity are grouped and divided into superpixels. Here, the image data of the superpixel image becomes divided image data by superpixel regions. Step S04 corresponds to the second image processing step.

[0036] Step S05 is a step in which a vascular density map is generated by an average / median calculation process that calculates the average or median brightness intensity for each segmented image data of the superpixel image. Step S05 corresponds to the third image processing step (vascular system image generation processing step).

[0037] The details of the image processing that generates the blood vessel map will be explained based on the flowchart in Figure 4, which shows the flow of image processing performed in the image processing unit 103.

[0038] Step S06 is the step of obtaining the vascular density map generated by the flowchart shown in Figure 3.

[0039] Step S07 determines a first threshold for the vascular density map, which is used to define brightness intensity regions that include blood vessels with a diameter exceeding a predetermined diameter. Step S07 is a step in generating a vascular map by a first intensity value calculation process that retains image data equal to or greater than the first threshold from each segmented image data of the vascular density map. In other words, this first intensity value calculation process is an image processing that excludes image data below the first threshold from each segmented image data of the vascular density map, and an example of determining the first threshold is shown in Figure 13. Step S07 corresponds to the fourth image processing step (vascular system image generation processing step).

[0040] The details of the image processing that generates the non-blood flow area detection map will be explained based on the flowchart in Figure 5, which shows the flow of image processing performed by the image processing unit 103.

[0041] Step S08 is the step of obtaining the vascular density map generated by the flowchart shown in Figure 3.

[0042] Step S09 determines a second threshold for the vascular density map, which is used to distinguish brightness intensity regions containing capillaries. Step S09 is a step in which a non-blood flow region detection map is generated by a second intensity value calculation process that retains image data below the second threshold from each segmented image data of the vascular density map. In other words, this second intensity value calculation process is an image processing that excludes image data exceeding the second threshold from each segmented image data of the vascular density map, and Figure 13 shows an example of determining the second threshold (<first threshold). Step S09 corresponds to the fifth image processing step (vascular system image generation processing step).

[0043] [Details of the image workflow (Figure 6)] The details of the image workflow 300 will be explained based on the block diagram in Figure 6, which shows the image workflow from image processing in the image processing unit 103 of Embodiment 1 to the generation of the final image. The image workflow 300 is broadly divided into two image workflows: the blood vessel enhancement image flow section 310 and the map generation image flow section 320.

[0044] The blood vessel-enhanced image flow unit 310 includes an OCTA original image 311, an OCTA noise-reduced image 312, a blood vessel extraction image 313, and a blood vessel-enhanced image 314.

[0045] The OCTA original image 311 is an image obtained from an OCT device using OCTA image data of a retinal surface slab without any preprocessing such as noise reduction. The OCTA noise reduction image 312 is an image obtained from an OCT device using OCTA original image data of a retinal surface slab after preprocessing by noise reduction. The vascular extraction image 313 is an image obtained from an OCTA original image data or OCTA noise reduction image data after applying vascular extraction processing (A). The vascular enhancement image 314 is an image obtained from an image data obtained from an vascular enhancement processing (B) which adds vascular extraction image data with a specific weight to the original image data (OCTA original image data or OCTA noise reduction image data).

[0046] The map generation image flow unit 320 includes a superpixel image 321, a vascular density map 322, a vascular map 323, and a non-blood flow area detection map 324.

[0047] The superpixel image 321 is an image generated from the image data after applying superpixel segmentation processing (C) to the vascular-enhanced image data. The vascular density map 322 is an image generated from the image data after applying mean / median calculation processing (D) to each segment of the superpixel image data 321. The vascular map 323 is an image generated from the image data after applying first intensity value calculation processing (E) to each segment of the vascular density map 322. The non-blood flow area detection map 324 is an image generated from the image data after applying second intensity value calculation processing (F) to each segment of the vascular density map 322.

[0048] [Details of blood vessel extraction and blood vessel enhancement processing (Figure 7)] The details of the blood vessel extraction process (A) and the blood vessel enhancement process (B) will be explained based on Figure 7, which shows the image transitions resulting from image processing on the original OCTA image data of the retinal surface slab.

[0049] The original OCTA image 311 of the retinal surface slab transitions to the vascular extraction image 313 via vascular extraction processing (A), and further transitions to the vascular enhancement image 314 via vascular enhancement processing (B).

[0050] The vascular extraction process (A) uses a vascular extraction filter to extract vascular structures from the image data of the OCTA original image 311, where vascular structures are not clearly depicted, by applying a filter that reveals the unique shapes and patterns of blood vessels, thereby obtaining a vascular extraction image 313. Here, as the vascular extraction filter, for example, a Bar-Circular Symmetric Filter (BCOSFIRE), which is designed to detect curved structures such as blood vessels, can be used. Note that the vascular extraction process (A) may be applied to the OCTA denoised image 312 of the retinal surface slab instead of the OCTA original image 311 of the retinal surface slab.

[0051] The vascular enhancement process (B) adds vascular extraction image data with specific weights to the original image data (OCTA original image data or OCTA denoising image data) to create a vascular enhancement image 314 using image data that enhances the vascular image. Here, "specific weights" means that the image portion of the vascular extraction image data, including large vessels and capillaries, is weighted higher than the non-vascular portion, thereby further emphasizing the vascular structure in the retinal surface slab. As a result, the vascular enhancement image 314, as shown on the far right of Figure 7, not only is the large vessel clearly enhanced, but even the capillaries are clearly visible. Note that the vascular enhancement process (B) may also be a filtering process using a weighted-guided image filter or the like.

[0052] [Details of the superpixel segmentation process (Figure 8)] The details of the superpixel segmentation process (C) will be explained based on Figure 8, which shows the image transitions due to image processing on vascular-enhanced image data.

[0053] The enhanced vascular-enhanced image 314 of the retinal surface slab transitions to a superpixel image 321 via superpixel segmentation processing (C). In other words, the superpixel image 321 is obtained by applying superpixel segmentation processing (C) to the image data of the vascular-enhanced image 314.

[0054] Superpixel segmentation (C) is a process that classifies pixels based on the similarity of their brightness intensity, groups pixels with common similarities, and divides them into superpixels. In the vascular-enhanced image 314 of the retinal surface slab, the vascular structures appear brighter, while non-perfused areas appear darker compared to the surrounding tissue with blood flow. Therefore, superpixel segmentation (C) groups pixels with similar brightness intensity characteristics, dividing the image data of the vascular-enhanced image of the retinal surface slab into thousands of small pixel groups. As shown in Figure 8, the superpixel image 321 after superpixel segmentation (C) is drawn with whitish lines in the grayscale image, but in the actual color image, it appears as a raised pattern drawn with yellow lines against a dark background. These yellow lines in the superpixel image 321 are boundaries that show how the image was divided by superpixel segmentation (C).

[0055] The reason for employing superpixel segmentation (C) is to perform efficient image processing and analysis by treating image data not in the usual pixel units, but in pixel groups formed by dividing the image data into localized superpixel regions. In particular, superpixel segmentation (C) on vascular-weighted image data of retinal surface slabs is useful for efficient analysis and quantification of perfused and non-perfused regions where blood flows.

[0056] [Details of the mean / median calculation process (Figures 9 and 10)] The details of the mean / median calculation process (D) will be explained based on Figures 9 and 10, which show the image transitions due to image processing on the superpixel image 321. Note that the superpixel image 321 in Figure 9 is based on the original OCTA image data of the retinal surface slab. The superpixel image 321 in Figure 10 is based on the OCTA denoised image data of the retinal surface slab.

[0057] The superpixel image 321, based on the original OCTA image data or denoised OCTA image data of the retinal surface slab, transitions to the vascular density map 322 image via the mean / median calculation process (D). In other words, the vascular density map 322 image is obtained by applying the mean / median calculation process (D) to the superpixel region of each segmented image data of the superpixel image 321.

[0058] The mean / median calculation process (D) calculates the mean or median brightness intensity of the superpixel region of each segmented image data in the superpixel image 321. Then, the mean / median calculation process (D) replaces each segmented image data of the superpixel region with the mean or median brightness intensity to obtain the vascular density map 322 image. The vascular density map 322 image is a color image display where the vascular images are prominent. The vascular images in the vascular density map 322 are displayed in a light gray color in the grayscale images of Figures 9 and 10, but are displayed in a bright red color in the actual color image. On the other hand, areas without blood flow in the vascular density map 322 (avascular region FAZ, non-perfusion region NPA) have low brightness intensity values ​​because there is no blood flow, and are displayed in a dark gray color in the grayscale images of Figures 9 and 10, but are displayed in a dark blue color in the actual color image.

[0059] Here, the foveal avascular zone (FAZ) refers to the avascular area surrounded in a ring by retinal cilia and other tissues, centered around the fovea 208 of the macula 207 of retina 201 (see Figure 2). The non-perfusion area (NPA) refers to the avascular area in the superficial slab of retina 201 due to capillary occlusion (see Figure 2).

[0060] Therefore, the color vascular density map 322 image generated by the mean / median calculation process (D) is an image in which the vascular and non-vascular areas of the retinal surface slab are distinguished by color coding. Furthermore, the difference between the two images shown in Figure 9 and the two images shown in Figure 10 is that the vascular structure is more clearly shown in the superpixel image 321 of Figure 10, which is based on OCTA noise reduction image data, compared to the superpixel image 321 of Figure 9, which is based on OCTA original image data. For this reason, although the color vascular density map 322 is generated by color coding of vascular areas (red) and non-vascular areas (blue) by the mean / median calculation process (D), the color coding distinction in Figure 10 is more precise than that of the vascular density map 322 of Figure 9.

[0061] [Details of the first intensity value calculation process (Figures 11 and 13)] The details of the first intensity value calculation process (E) will be explained based on Figure 11, which shows the image transition due to image processing on the vascular density map 322, and Figure 13, which shows an example of the superpixel number distribution with respect to brightness intensity used to determine the first threshold.

[0062] The image of the vascular density map 322, based on the original OCTA image data of the retinal surface slab or the OCTA noise-reduced image data of the retinal surface slab, transitions to the image of the vascular map 323 via the first intensity value calculation process (E). In other words, the image of the vascular map 323 is obtained by applying the first intensity value calculation process (E) to the superpixel regions of each segmented image data of the vascular density map 322.

[0063] The first intensity value calculation process (E) determines a first threshold for each segmented image data of the blood vessel density map 322, which is used to distinguish brightness intensity regions containing blood vessels with a diameter exceeding a predetermined diameter. The first intensity value calculation process (E) is an image processing method that generates image data for the blood vessel map 323 by retaining image data that is equal to or greater than the first threshold from each segmented image data of the blood vessel density map 322.

[0064] In other words, the vascular regions of the retinal surface slab contain blood flow information and are therefore displayed brightly in OCTA images and bright red in the color vascular density map 322. Thus, the vascular map 323 can be calculated by retaining the regions with high blood flow within the vascular region using a threshold (first threshold). The first threshold can be determined using existing knowledge or histogram analysis. In the case of histogram analysis, as shown in Figure 13, a brightness intensity threshold is determined as the first threshold for classifying the brightness intensity regions with high blood flow, which retains blood vessels with a diameter exceeding a predetermined diameter and excludes capillaries with a diameter less than a predetermined diameter.

[0065] Thus, the image of the vascular map 323 generated by the first intensity value calculation process (E) can clearly display large arteries, several capillaries, and even macular neovascularization structures, as shown in the image on the right of Figure 11. For this reason, the information on the location of blood vessels in the vascular map 323 is useful for detecting non-perfusion areas (NPAs) that appear around large arteries, and for detecting an eye disease called macular neovascularization.

[0066] [Details of the second intensity value calculation process (Figures 12 and 13)] The details of the second intensity value calculation process (F) will be explained based on Figure 12, which shows the image transition due to image processing on the vascular density map 322, and Figure 13, which shows an example of the superpixel number distribution with respect to brightness intensity used to determine the second threshold.

[0067] The image of the vascular density map 322, based on the original OCTA image data or the OCTA noise-reduced image data of the retinal surface slab, transitions to the image of the non-blood flow area detection map 324 via a second intensity value calculation process (F). In other words, the image of the non-blood flow area detection map 324 is obtained by applying the second intensity value calculation process (F) to the superpixel regions of each segmented image data of the vascular density map 322.

[0068] The second intensity value calculation process (F) determines a second threshold (<first threshold) for each segmented image data of the vascular density map 322 that distinguishes the brightness intensity region containing capillaries. The second intensity value calculation process (F) is an image processing step that generates image data for the non-blood flow region detection map 324 by retaining image data below the second threshold from each segmented image data of the vascular density map 322.

[0069] In other words, since there is no blood flow information in the non-perfusion area (NPA) and the avascular area (FAZ) of the retinal surface, they appear dark in the OCTA image of the retinal surface slab and are displayed in dark blue in the color vascular density map 322 image. Each segmented image data of the non-blood flow area detection map 324 can be calculated by retaining areas with low blood flow within the vascular region using a threshold (second threshold). The second threshold can be determined using existing knowledge or histogram analysis. In the case of histogram analysis, as shown in Figure 13, a second threshold (<first threshold) is determined to separate brightness intensity areas with low blood flow, leaving capillaries with low blood flow and areas with no blood flow, and excluding other blood vessels.

[0070] Thus, the image of the non-blood-flowing area detection map 324 generated by the second intensity value calculation process (F) displays areas with little blood flow and areas with no blood flow. The image display of areas with little blood flow and areas with no blood flow can be used to detect the avascular area (FAZ) and non-perfusion area (NPA) of the cavity, which have no blood flow. Furthermore, in the image of the non-blood-flowing area detection map 324 generated by the second intensity value calculation process (F), excluded large vessels are displayed in white. These large vessels displayed in white represent the image display positions of large arteries in the vascular structure and can be used to detect the non-perfusion area (NPA). This is because the non-perfusion area (NPA) usually appears around arteries. Therefore, the non-blood-flowing area detection map 324 can be used to detect the avascular area (FAZ) and non-perfusion area (NPA) of the cavity, which have no blood flow, and can further improve the accuracy of detecting the non-perfusion area (NPA).

[0071] [Regarding the comparison of vascular density maps] The vascular density algorithm that generates existing vascular density maps takes OCTA images and scan parameters as inputs necessary for calculating vascular density.

[0072] The process for generating existing vascular density maps consists of the following steps: 1. Steps for creating OCTA surface images 2. Steps to create a binarized map based on the brightness intensity of the OCTA surface image. The creation of the binarized map (vascular segmentation map) is performed using a binarization algorithm based on local adaptive thresholding, where the brightness intensity of each pixel is compared to the average brightness intensity of its neighbors. The size of the neighboring region depends on the scan pattern. The binarized map from the binarized images is denoised to reduce noise before calculating the vascular density. 3. Steps to create a vascular density map from a binarized map Vascular density is calculated by averaging over a predefined (4x4) region based on a binarized map. Next, the calculated vascular density is smoothed using a two-dimensional averaging filter (15x15) to create a vascular density map.

[0073] This vascular density algorithm uses a binary map (a map with a data format represented by "0"s and "1"s) instead of a vascular density map for 3D disk scans displaying vascular density. In other cases, it converts the vascular density map to a color map using a lookup table for display. Therefore, existing vascular density maps rely on thresholding and averaging of small regions and are heavily affected by artifacts (such as virtual images that are introduced during scanning and do not actually exist) and noise.

[0074] In contrast, the vascular density algorithm for generating the vascular density map 322 in Embodiment 1 does not include scan parameters in the inputs required for calculating vascular density.

[0075] The process for generating the vascular density map 322 in Embodiment 1 consists of the following steps. 1. Image data acquisition process for obtaining OCTA image data 2. First image processing step: Generates image data of a vascular-enhanced image 314 from OCTA image data by performing vascular extraction processing (A) and vascular enhancement processing (B). 3. A second image processing step in which segmented image data of a superpixel image 321 is generated from blood vessel-enhanced image data by superpixel segmentation processing (C), 4. Third image processing step: Generates a vascular density map 322 by performing mean / median calculation processing (D) from segmented image data of the superpixel image 321. Furthermore, the process for generating the blood vessel map 323 includes a fourth image processing step in addition to the third image processing step, and the process for generating the non-blood flow area detection map 324 includes a fifth image processing step in addition to the third image processing step.

[0076] Thus, the vascular density map 322 generated by the image processing of Embodiment 1 employs a vascular density algorithm that uses first and second image processing, and does not include the step of creating a binarized map that is present in existing vascular density algorithms. Therefore, the vascular density map 322 can be prevented from being affected by artifacts and noise, unlike existing vascular density algorithms that include a step of creating a binarized map.

[0077] [The relationship between each of the three types of map images and eye diseases] The image generation method and image generation system of Embodiment 1 can generate vascular images of the retinal surface slab using three types of maps: a vascular density map 322, a vascular map 323, and a non-blood flow area detection map 324. The eye diseases that can be detected by the three types of map images are those that can be observed as changes from the normal vascular structure in the retinal surface slab in the OCTA image of the retinal surface slab. The relationship between each of the three types of map images and eye diseases is explained below separately for each map image.

[0078] (Vascular density map 322) As shown in Figures 9 and 10, the vascular density map 322 images are useful for measuring vascular density in specific areas of the retinal surface slab. Therefore, eye diseases closely related to detection include diabetic retinopathy, glaucoma, and retinal vein occlusion.

[0079] Diabetic retinopathy is a complication of diabetes that affects the retina (201) of the eyeball (200). It is a disease in which the capillaries spreading throughout the retina (201) are damaged when high blood sugar levels persist for a long time. Therefore, if the vascular density map (322) shows a decrease in vascular density from normal in the displayed image, it may indicate ischemia or a perfusion area, which is characteristic of diabetic retinopathy.

[0080] Glaucoma is a disease that causes abnormalities in the optic nerve 204 (the nerve that transmits information from the eye to the brain) and the visual field. Therefore, in the vascular density map 322 image, a decrease in vascular density in the area surrounding the optic nerve head 205 in the displayed image indicates damage to the optic nerve 204 and may be a sign of glaucoma.

[0081] Retinal vein occlusion is an eye disease caused by blockage of a large vein in the retina (vein 201), leading to stagnation of blood flow. Therefore, if there are areas of reduced blood flow in the vascular density map (vein 322) image, it may be due to vein occlusion.

[0082] (Vascular map 323) The image of the vascular map 323, as shown in Figure 11, helps to provide a detailed image of the blood vessels 211 by highlighting the large and small blood vessels of the retina 201. For this reason, eye diseases that are closely related to detection include diabetic retinopathy, hypertensive retinopathy, and macular degeneration.

[0083] As mentioned above, "diabetic retinopathy" is a disease in which the capillaries spreading across the retina 201 are damaged when high blood sugar levels persist for a long period. Therefore, the image of the vascular map 323 allows for the identification of microaneurysms, hemorrhages, and neovascularization by displaying detailed images of the blood vessels 211.

[0084] "Hypertensive retinopathy" refers to a disease in which high blood pressure causes damage to the retina 201 and its capillaries. Therefore, the image of the vascular map 323, with its detailed display of the blood vessels 211, can show changes in the diameter of the blood vessels 211, such as narrowing of the blood vessels 211, damage to arteries and veins, and other signs of vascular damage due to high blood pressure.

[0085] "Macular degeneration" refers to a disease in which damage occurs to the macula 207, the central part of the retina 201, making it difficult to see what you are trying to look at. Therefore, the image of the blood vessel map 323 allows for the identification of abnormal blood vessels or macular neovascularization near the macula 207 by displaying detailed images of the blood vessels 211.

[0086] (Non-blood flow area detection map 324) As shown in Figure 12, the images from the non-blood-flow area detection map 324 can highlight non-perfusion areas (NPAs) and are useful for evaluating the size and shape of the fossa avascular area (FAZ). Therefore, eye diseases closely related to detection targets include diabetic retinopathy, macular ischemia, and retinal vein occlusion.

[0087] As mentioned above, "diabetic retinopathy" is a disease in which the capillaries spread across the retina 201 are damaged when high blood sugar levels persist for a long period. Therefore, the image of the non-blood flow area detection map 324, by displaying the non-perfusion area NPA (NPA) where there is no blood flow, can be used to determine if a large non-perfusion area NPA indicates an advanced disease stage accompanied by a significant reduction in capillaries.

[0088] "Macular ischemia" is an irreversible condition commonly seen in diabetic retinopathy and can lead to blindness. Therefore, the images in the non-blood flow area detection map 324, based on the display of the cavity avascular zone (FAZ), may suggest macular ischemia, which is often associated with diabetic retinopathy or retinal vein occlusion, depending on changes in the size and irregularities of the cavity avascular zone (FAZ).

[0089] "Retinal vein occlusion," as described above, is an eye disease that develops when a large vein in the retina 201 becomes blocked, causing blood flow to stagnate. Therefore, the image of the non-blood flow area detection map 324 can be used to determine the non-perfusion and ischemic areas caused by vein occlusion by displaying the non-perfusion area (NPA) where there is no blood flow.

[0090] (summary) Images from vascular density map 322 can be used primarily to detect ocular diseases related to vascular perfusion, such as diabetic retinopathy, glaucoma, and retinal vein occlusion. Images from vascular map 323 are more detailed and can identify changes in both small and large vessels, making them useful for detecting conditions such as diabetic retinopathy, hypertensive retinopathy, and macular degeneration. Images from non-blood flow area detection map 324 are important for identifying ischemic areas and evaluating the cavity avascular area (FAZ), and can provide analytical information regarding conditions such as diabetic retinopathy, macular ischemia, and retinal vein occlusion.

[0091] [Effects of image generation methods and image generation systems] The image generation method and image generation system 100 of Embodiment 1 have the following effects.

[0092] (1) An image generation method for generating a vascular system image of an eye under examination comprises an image data acquisition step (S01), a first image processing step (S03), a second image processing step (S04), and a vascular system image generation processing step (S05, etc.). The image data acquisition step acquires image data of the eye under examination. The first image processing step extracts the curved structure of blood vessels contained in the image data and generates image data of a vascular-enhanced image 314 in which the curved structure is emphasized. The second image processing step classifies the image data of the vascular-enhanced image 314 by the similarity of the brightness intensity of pixels, groups pixels with common similarity and divides them into superpixels to generate segmented image data of a superpixel image 321. The vascular system image generation processing step generates a vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image 321. This image generation method can generate vascular images of the examined eye, which are useful for the early detection of eye diseases and for understanding lesions in eye diseases.

[0093] (2) In the image data acquisition process (S01), multiple images of the same area of ​​the fundus of the eye under examination are taken using an OCT device that can obtain tomographic data, and OCTA image data is acquired by depicting only the parts of the multiple OCT images that change due to blood flow as blood flow information. This image generation method obtains OCTA image data without the use of contrast agents as image data of the eye being examined. This method has the advantage of obtaining vascular image data that is safe, has no side effects, and is useful for examining eye diseases.

[0094] (3) The image data acquisition process (S01) acquires image data of the retinal surface slab from the OCTA image data. This image generation method obtains vascular image data depicting blood vessels 211 distributed in the superficial slab of the retina 201 by acquiring image data of the retinal surface slab from OCTA image data.

[0095] (4) The vascular system image generation process includes a third image processing step (S05) which generates a vascular density map 322 of the retinal surface slab by an average / median calculation process (D) which calculates the average or median value of brightness intensity for each segmented image data of the superpixel image. This image generation method can generate an image of a vascular density map 322 of the retinal surface slab, which is useful for detecting eye diseases that manifest in vascular density, by calculating the average / median brightness intensity for each segmented image data of the superpixel image 321 (D).

[0096] (5) The vascular system image generation process determines a first threshold for the vascular density map 322 that divides the brightness intensity region containing blood vessels with a diameter exceeding a predetermined diameter. The vascular system image generation process includes a fourth image processing step (S07) which generates a vascular map 323 of the retinal surface slab by a first intensity value calculation process (E) that leaves image data equal to or greater than the first threshold from each segmented image data of the vascular density map 322. This image generation method can generate an image of a vascular map 323 of the retinal surface slab, which has a more detailed vascular structure and can distinguish changes in both small and large blood vessels, by calculating a first intensity value of brightness intensity for each segmented image data of the vascular density map 322 (E).

[0097] (6) The vascular image generation process determines a second threshold for the vascular density map 322 that divides the brightness intensity region containing capillaries. The vascular image generation process includes a fifth image processing step (S09) which generates a non-blood flow region detection map 324 of the retinal surface slab by a second intensity value calculation process (F) that leaves image data below the second threshold from each segmented image data of the vascular density map 322. This image generation method can identify non-perfusion areas (NPAs) and evaluate cavity avascular areas (FAZs) of the retinal surface slab. Images of these non-perfusion areas (NPAs) can be generated by a second intensity value calculation process (F) for each segmented image data of the vascular density map (322).

[0098] (7) The OCTA image data of the retinal surface slab acquired in the image data acquisition step (S01) is subjected to a noise reduction process (S02) that removes at least speckle noise and random noise. This image generation method removes unwanted noise components from the image data of the vascular structure, resulting in clearer OCTA noise-reduced image data of vascular structures compared to the original OCTA image data of the retinal surface slab.

[0099] (8) The image generation system 100 for generating a vascular system image of the eye under examination comprises an image data acquisition unit 131, a first image processing unit 133, a second image processing unit 134, and a vascular system image generation processing unit (such as a third image processing unit 135). The image data acquisition unit 131 acquires image data of the eye under examination. The first image processing unit 133 generates image data of a vascular-enhanced image by performing a vascular extraction process (A) to extract the curved structure of blood vessels contained in the image data and a vascular enhancement process (B) to emphasize the extracted vascular structure. The second image processing unit 134 generates segmented image data of a superpixel image by performing a superpixel segmentation process (C) to classify the image data of the vascular-enhanced image by the similarity of the brightness intensity of pixels and group pixels with common similarity into superpixels. The vascular system image generation processing unit generates a vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image. This image generation system 100 can generate vascular images of the eye being examined, which are useful for the early detection of eye diseases and for understanding lesions in eye diseases. Embodiment 2

[0100] Embodiment 2 is an embodiment in which, unlike Embodiment 1 which generates a vascular image based on OCTA image data of the retinal surface slab, the vascular image is generated based on OCTA image data of the retinal outer layer slab, which is deeper than the retinal surface slab.

[0101] [System Configuration (Figure 14)] Figure 14 shows an example of an image generation system to which the image generation method of Embodiment 2 is applied. As shown in Figure 14, the image generation system 100' includes an OCTA image data creation unit 110, a user input device 120, an image processing unit 130', a display 140, and a vascular system image data storage unit 150. The OCTA image data creation unit 110, user input device 120, display 140, and vascular system image data storage unit 150, excluding the image processing unit 130', have the same configuration as in Embodiment 1, so their description is omitted.

[0102] The image processing unit 130' can acquire original OCTA image data of the outer retinal slab from the OCTA image data creation unit 110, and generate a first feature extraction map and a second feature extraction map of the outer retinal slab based on this original OCTA image data.

[0103] The image processing unit 130' of Embodiment 2 includes an image data acquisition unit 131', a noise reduction processing unit 132', a first image processing unit 133', a second image processing unit 134', a third image processing unit 135', and a fourth image processing unit 136'. Here, the third image processing unit 135' and the fourth image processing unit 136' correspond to vascular system image generation processing units.

[0104] The image data acquisition unit 131' acquires OCTA original image data of the outer retinal slab as image data of the eye under examination. The noise reduction processing unit 132' generates OCTA noise-reduced image data by performing a noise reduction process to remove noise from the OCTA original image data of the outer retinal slab. The first image processing unit 133' generates image data of a vascular-enhanced image by performing a vascular extraction process to extract the curved structures of blood vessels contained in the OCTA original image data or OCTA noise-reduced image data, and a vascular enhancement process to emphasize the extracted vascular structures. The second image processing unit 134' generates segmented image data of a superpixel image by performing a superpixel segmentation process that classifies the image data of the vascular-enhanced image data by the similarity of the brightness intensity of pixels, and groups pixels with common similarity into superpixels.

[0105] The third image processing unit 135' generates a first feature extraction map of the outer retinal slab (corresponding to the blood vessel density map in Embodiment 1) by performing an average / median calculation process that calculates the average or median brightness intensity for each segmented image data of the superpixel image. The fourth image processing unit 136' determines a first threshold for the first feature extraction map that divides the brightness intensity region containing blood vessels with a diameter exceeding a predetermined diameter. The fourth image processing unit 136' generates a second feature extraction map of the outer retinal slab (corresponding to the blood vessel map in Embodiment 1) by performing a first intensity value calculation process that retains image data above the first threshold from each segmented image data of the first feature extraction map. The outer retinal slab refers to the outer layer of the retina 201 that is in contact with the choroid 202, and is the layer in which photoreceptor cells that sense light are lined up.

[0106] [Details of image processing (Figure 15)] Details of the image processing that generates the vascular density map and the vascular map will be explained based on the flowchart in Figure 15, which shows the flow of image processing performed in the image processing unit 130'.

[0107] Step S21 is the step of acquiring the original OCTA image data of the outer retinal slab from the OCTA image data generated by the OCTA image data creation unit 110 as image data of the eye under examination. Step S21 corresponds to the image data acquisition process.

[0108] Step S22 is a step in which noise is removed from the OCTA original image data of the outer retinal slab acquired in step S21 by noise reduction processing, thereby obtaining OCTA noise-reduced image data. Note that the noise reduction processing in step S22 is not a necessary process for carrying out the various image processing steps from step S23 onward. Step S22 corresponds to the noise reduction processing step.

[0109] Step S23 is a step in which the curved structures of blood vessels contained in the OCTA original image data or OCTA noise-reduced image data are extracted by a blood vessel extraction process, and the extracted blood vessel structures are enhanced by an enhancement process to generate image data of a blood vessel-enhanced image. Step S23 corresponds to the first image processing step.

[0110] Step S24 is a step in which the image data of the blood vessel-enhanced image is classified by the similarity of the brightness intensity of the pixels, and images of the superpixel image are generated by a superpixel segmentation process in which pixels with common similarity are grouped and divided into superpixels. Here, the image data of the superpixel image becomes image data divided by superpixel regions. Step S24 corresponds to the second image processing step.

[0111] Step S25 is a step in which a first feature extraction map of the outer retinal slab is generated by an average / median calculation process that calculates the average or median brightness intensity for each segmented image data of the superpixel image. Step S25 corresponds to the third image processing step (vascular system image generation processing step).

[0112] Step S26 determines a first threshold for each segmented image data of the first feature extraction map, which is used to distinguish brightness intensity regions containing blood vessels with a diameter exceeding a predetermined diameter. Step S26 is a step in which a second feature extraction map of the outer retinal slab is generated by a first intensity value calculation process that retains image data equal to or greater than the first threshold from each segmented image data of the first feature extraction map. Step S26 corresponds to the fourth image processing step (vascular system image generation processing step).

[0113] [Details of the image workflow (Figure 16)] The details of the image workflow 300 will be explained based on the block diagram in Figure 16, which shows the image workflow up to the final image generation by image processing in the image processing unit 103' of Embodiment 2. The image workflow 300 is broadly divided into two image workflows: the blood vessel enhancement image flow section 310 and the map generation image flow section 320.

[0114] The blood vessel-enhanced image flow unit 310 includes an OCTA original image 315, an OCTA noise-reduced image 316, a blood vessel extraction image 317, and a blood vessel-enhanced image 318.

[0115] The OCTA original image 315 is an image obtained from an OCT device using OCTA image data of the outer retinal slab without any preprocessing such as noise reduction. The OCTA noise reduction image 316 is an image obtained from an OCT device using OCTA original image data of the outer retinal slab after preprocessing by noise reduction. The vascular extraction image 317 is an image obtained from an OCTA original image data or OCTA noise reduction image data after applying vascular extraction processing (A). The vascular enhancement image 318 is an image obtained from an image data obtained from an vascular enhancement processing (B) in which vascular extraction image data with a specific weight is added to the original image data (OCTA original image data or OCTA noise reduction image data).

[0116] The map generation image flow unit 320 includes a superpixel image 325, a first feature extraction map 326, and a second feature extraction map 327.

[0117] The superpixel image 325 is an image obtained by dividing the image data of the blood vessel-enhanced image 318 into thousands of superpixels after applying superpixel segmentation processing (C). The first feature extraction map 326 is an image obtained by applying mean / median calculation processing (D) to each segment of the superpixel image 325. The second feature extraction map 327 is an image obtained by applying first intensity value calculation processing (E) to each image data of the first feature extraction map 326.

[0118] [Details of image processing (Figure 17)] Details of the image processing will be explained based on Figure 17, which shows the transition of images generated by image processing of the OCTA original image data or OCTA denoising image data of the outer retinal slab.

[0119] The OCTA original image 315 or OCTA denoising image 316 of the outer retinal slab transitions to the superpixel image 325 via superpixel segmentation processing (C). Note that Figure 17 omits the blood vessel extraction processing (A) and blood vessel enhancement processing (B), which are image processing steps similar to those in Embodiment 1. In other words, the superpixel image 325 is obtained by applying superpixel segmentation processing (C) to the image data of the blood vessel enhancement image 318.

[0120] In the vascular-enhanced image 318 of the outer retinal slab, the vascular structures appear brighter, while non-perfusion areas appear darker. Therefore, the superpixel segmentation process (C) groups pixels with similar brightness intensity characteristics, dividing the image data of the vascular-enhanced image of the outer retinal slab into thousands of small pixel groups. The superpixel image 325 after the superpixel segmentation process (C), as shown in Figure 17, is depicted as whitish lines in the grayscale image, but in the actual color image, it appears as a raised pattern drawn as yellow lines against a dark background. These yellow lines in the superpixel image 325 are boundaries indicating how the image was divided by the superpixel segmentation process (C).

[0121] The superpixel image 325 transitions to the first feature extraction map 326 via the mean / median calculation process (D). In other words, the image of the first feature extraction map 326 of the outer retinal slab is obtained by applying the mean / median calculation process (D) to each segmented image data of the superpixel image 325, in the same manner as in Embodiment 1.

[0122] The image of the first feature extraction map 326 transitions to the image of the second feature extraction map 327 via the first intensity value calculation process (E). In other words, the image of the second feature extraction map 327 of the outer retinal slab is obtained by applying the first intensity value calculation process (E) to each segmented image data of the first feature extraction map 326, in the same manner as in Embodiment 1.

[0123] [Detection of eye diseases] The outer retinal slab is located in the lateral part of the retina 201, adjacent to the choroid 202. Unlike the superficial retinal slab, which is distributed with blood vessels, it is basically a layer without blood vessels, containing rows of photoreceptor cells that sense light. Therefore, in a normal eye, the OCTA original image 315 of the outer retinal slab will not capture any blood vessels.

[0124] In contrast, the OCTA original image 315 of the outer retinal slab in Embodiment 2 shows blood vessels in a portion of the image, as shown in Figure 17. Image processing based on this OCTA original image 315 generates a first feature extraction map 326. The first feature extraction map 326 is generated by the mean / median calculation process (D) for each image data of the superpixel image 325, and can display choroidal neovascularization as shown in the upper right image of Figure 17. The second feature extraction map 327 is generated by the first intensity value calculation process (E) for the image data of the first feature extraction map 326, and can display the presence of choroidal neovascularization as shown in the lower right image of Figure 17. Therefore, the information on the presence and location of blood vessels in the first feature extraction map 326 or the second feature extraction map 327 is useful for detecting an eye disease called choroidal neovascularization.

[0125] Choroidal neovascularization refers to a disease in which abnormal new blood vessels grow from the choroid 202, which is located outside the retina 201 and has abundant blood vessels. These abnormal new blood vessels are very fragile and prone to leaking blood and fluids, causing problems such as decreased vision and retinal edema.

[0126] [Effects of image generation methods and image generation systems] The image generation method and image generation system 100' of Embodiment 2 provide the following effects in addition to the effects of (1), (2), and (8) of Embodiment 1.

[0127] (9) The image data acquisition process (S21) acquires image data of the outer retinal slab from the OCTA image data. This image generation method obtains vascular image data, which shows blood vessels in the outer retinal slab when there is an eye disease such as choroidal neovascularization, by acquiring image data of the outer retinal slab from OCTA image data.

[0128] (10) The vascular system image generation process includes a third image processing step (S25) which generates a first feature extraction map 326 of the outer retinal slab by an average / median calculation process (D) which calculates the average or median value of brightness intensity for each segmented image data of the superpixel image 325. This image generation method can generate an image of the first feature extraction map 326 of the outer retinal slab, which is useful for detecting eye diseases, by calculating the average / median brightness intensity for each segmented image data of the superpixel image 325 (D).

[0129] (11) The vascular system image generation process determines a first threshold for the first feature extraction map 326 that divides the brightness intensity region containing blood vessels with a diameter exceeding a predetermined diameter. The vascular system image generation process includes a fourth image processing step (S26) which generates a second feature extraction map 327 of the outer retinal slab by a first intensity value calculation process (E) that leaves image data equal to or greater than the first threshold from each segmented image data of the first feature extraction map 326. This image generation method can generate an image of a second feature extraction map 327 of the outer retinal slab, which has a more detailed vascular structure and can distinguish changes in both small and large blood vessels, by performing a first intensity value calculation process (E) for brightness intensity relative to the first feature extraction map 326.

[0130] The above description is based on the drawings of the image generation method and image generation system of Embodiments 1 and 2. However, the specific configuration of the image generation method and image generation system of this disclosure is not limited to Embodiments 1 and 2, and changes or additions to the design are permitted as long as they do not deviate from the gist of the invention as described in each claim.

[0131] The algorithm disclosed herein can provide a comprehensive workflow for highlighting feature areas from OCTA images and dividing the image into small regions (superpixels) with similar features. The algorithm then outputs a vascular map, a vascular density map, a non-blood-flow area detection map, and a feature extraction map (mainly for neovascularization detection) by performing image processing to calculate the brightness intensity of each small region (superpixel). Overall, the image generation method and image generation system disclosed herein have the potential to visually highlight feature areas such as blood vessels and effectively detect areas of eye disease.

[0132] Embodiments 1 and 2 illustrate examples of applying the algorithm of this disclosure to image data of OCTA images. However, the algorithm of this disclosure may be applicable not only to image data of OCTA images, but also to image data of OCT images, fundus images, and other types of imaging.

[0133] Embodiment 1 showed an example of acquiring OCTA image data of a retinal surface slab as the image data acquisition process and image data acquisition unit. Embodiment 2 showed an example of acquiring OCTA image data of an retinal outer layer slab as the image data acquisition process and image data acquisition unit. However, the OCTA image data acquired by the image data acquisition process and image data acquisition unit is not limited to image data of the retinal surface slab or retinal outer layer slab, but can also be applied to image data of other areas or layers, such as the retinal deep slab or choroidal capillary plate slab.

[0134] Embodiment 1 shows an example in which the vascular system image generation process and vascular system image generation processing unit generate a vascular density map 322, a vascular map 323, and a non-blood flow area detection map 324 as vascular system images. Embodiment 2 shows an example in which the vascular system image generation process and vascular system image generation processing unit generate a first feature extraction map 326 and a second feature extraction map 327 as vascular system images. However, the vascular system image generation process and vascular system image generation processing unit are not limited to generating these maps. The vascular system image generation process and vascular system image generation processing unit may generate an appropriate map that can accurately detect the type of eye disease to be detected by brightness intensity processing as a vascular system image. In short, the vascular system image generation process and vascular system image generation processing unit only need to generate a vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image. [Explanation of Symbols]

[0135] 100 Image Generation Systems 130 Image Processing Units 131 Image Data Acquisition Unit 133 First Image Processing Unit 134 Second Image Processing Unit 135 Third Image Processing Unit (Vascular System Image Generation Processing Unit) 136. Image Processing Unit 4 (Vascular System Image Generation Processing Unit) 137. Image Processing Unit 5 (Vascular System Image Generation Processing Unit) S01 Image data acquisition step (image data acquisition process) S03 First image processing step (first image processing process) S04 Second image processing step (second image processing process) S05 Third image processing step (vascular system image generation process) S07 Fourth image processing step (vascular system image generation process) S09 Fifth image processing step (vascular system image generation process) 100' Image Generation System 130' Image Processing Unit 131' Image data acquisition unit 133' First Image Processing Unit 134' Second Image Processing Unit 135' Third Image Processing Unit (Vascular System Image Generation Processing Unit) 136' Fourth Image Processing Unit (Vascular System Image Generation Processing Unit) S21 Image data acquisition step (image data acquisition process) S23 First image processing step (first image processing process) S24 Second image processing step (second image processing process) S25 Third image processing step (vascular system image generation process) S26 Fourth image processing step (vascular system image generation process)

Claims

1. An image generation method for generating an image of the vascular system of an eye under examination, The process includes an image data acquisition step to acquire image data of the eye being examined, A first image processing step involves extracting the curved structure of blood vessels contained in the aforementioned image data and generating image data of a blood vessel-enhanced image in which the curved structure is emphasized. A second image processing step involves classifying the image data of the blood vessel-enhanced image by the similarity of the brightness intensity of the pixels, grouping the pixels that share the same similarity, and dividing them into superpixels to generate segmented image data of the superpixel image. The process includes a vascular system image generation step, which generates the vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image. An image generation method characterized by the following:

2. In the image generation method described in claim 1, The image data acquisition step involves taking multiple images of the same area of ​​the fundus of the eye being examined using an OCT device that can obtain tomographic data, and then acquiring OCTA image data by depicting only the parts of the multiple OCT images that change due to blood flow as blood flow information. An image generation method characterized by the following:

3. In the image generation method described in claim 2, The image data acquisition step involves acquiring image data of the retinal surface slab from the OCTA image data. An image generation method characterized by the following:

4. In the image generation method described in claim 3, The vascular system image generation process includes a third image processing step that generates a vascular density map of the retinal surface slab by performing an average / median calculation process that calculates the average or median brightness intensity for each segmented image data of the superpixel image. An image generation method characterized by the following:

5. In the image generation method described in claim 4, The vascular system image generation processing step determines a first threshold for the vascular density map that divides the brightness intensity region into areas containing blood vessels with a diameter exceeding a predetermined diameter, The fourth image processing step includes generating a vascular map of the retinal surface slab by a first intensity value calculation process that retains image data above the first threshold from each segmented image data of the vascular density map. An image generation method characterized by the following:

6. In the image generation method described in claim 4, The vascular system image generation processing step determines a second threshold for the vascular density map that divides the brightness intensity region containing capillaries, The fifth image processing step includes generating a non-blood flow region detection map of the retinal surface slab by a second intensity value calculation process that retains image data below the second threshold from each segmented image data of the blood vessel density map. An image generation method characterized by the following:

7. In the image generation method described in claim 3, The OCTA image data of the retinal surface slab acquired in the image data acquisition step includes a noise reduction process that removes at least speckle noise and random noise. An image generation method characterized by the following:

8. In the image generation method described in claim 2, The image data acquisition step involves acquiring image data of the outer retinal slab from the OCTA image data. An image generation method characterized by the following:

9. In the image generation method described in claim 8, The vascular system image generation process includes a third image processing step that generates a first feature extraction map of the outer retinal slab by an average / median calculation process that calculates the average or median brightness intensity for each segmented image data of the superpixel image. An image generation method characterized by the following:

10. In the image generation method described in claim 9, The vascular system image generation processing step determines a first threshold for the first feature extraction map that divides the brightness intensity region containing blood vessels with a diameter exceeding a predetermined diameter, This includes a fourth image processing step that generates a second feature extraction map of the outer retinal slab by a first intensity value calculation process that retains image data above a first threshold from each segmented image data of the first feature extraction map. An image generation method characterized by the following:

11. An image generation system that generates vascular images of the eye under examination, An image data acquisition unit that acquires image data of the eye to be examined, A first image processing unit generates image data of a vascular-enhanced image by performing a vascular extraction process to extract the curved structure of blood vessels contained in the aforementioned image data and a vascular enhancement process to enhance the extracted vascular structure. A second image processing unit generates segmented image data of a superpixel image by superpixel segmentation processing, which classifies the image data of the blood vessel-enhanced image based on the similarity of the brightness intensity of the pixels, and groups pixels with common similarity and divides them into superpixels. The vascular system image generation processing unit generates the vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image. An image generation system characterized by the following features.

Images