Image generation method and image generation system

Abstract

Provided are an image generation method and an image generation system with which it is possible to generate a vascular system image of a subject eye useful for early detection of an eye disease and assessment of pathological changes in eye disease. The image generation method comprises an image data acquisition step (S01), a first image processing step (S03), a second image processing step (S04), and a vascular system image generation processing step (S05, etc.). In the image data acquisition step, image data of a subject eye is acquired. In the first image processing step, a curved structure of a blood vessel included in the image data is extracted, and image data of a blood vessel emphasized image 314 in which the curved structure is emphasized is generated. In the second image processing step, segmented image data for a superpixel image 321 is generated, the superpixel image 321 being obtained by classifying the image data of the blood-vessel-highlighted image 314 according to similarity of pixel brightness intensity and grouping pixels having the same similarities to divide the blood vessel emphasized image 314 into superpixels. In the vascular system image generation processing step, brightness intensity processing is applied to each item of divided image data of the superpixel image 321 to generate a vascular system image.

Description

Image generation method and image generation system

[0001] The present disclosure relates to an image generation method and an image generation system.

[0002] The image creation method described in Patent Document 1 has a first step, a second step, and a third step. The first step clusters pixels corresponding to each pixel data based on the optical feature similarity and spatial proximity of the pixel data included in the sample image data (OCT image) to generate superpixels. The second step generates pixel group data representing the superpixel based on the pixel data of each pixel included in the superpixel. The third step generates the plurality of sample cross-sectional images based on the pixel group data.

[0003] The image creation method described in this Patent Document 1 aims to clarify the fundus cross-sectional image by reshaping the superpixels (see paragraphs 0062, paragraph 0063, and FIG. 7). Note that OCT is an abbreviation for "Optical Coherence Tomography".

[0004] Japanese Unexamined Patent Application Publication No. 2018-73138

[0005] By the way, the blood vessel images of the eyes to be examined so far have mainly been obtained by depicting blood vessels through fundus fluorescence angiography using a contrast agent. However, the current blood vessel images of the eyes to be examined have reached a situation where OCTA images have high precision and can depict capillaries as blood vessel images of the eyes to be examined without using a contrast agent. Therefore, it is required by medical staff involved in eye examinations to perform image processing on an original image in which the blood vessel structure is not clearly depicted by OCTA images and generate a new blood vessel system image useful for early detection of eye diseases and understanding of eye disease lesions.

[0006] Here, an OCTA image is created by taking multiple images of the same area of ​​the fundus using an OCT device and imaging only the signals that change among the multiple OCT images. It is created by depicting only the parts of the OCT image that change due to blood flow as blood flow information. Note that OCTA is an abbreviation for "Optical Coherence Tomography Angiography".

[0007] In contrast, the image processing technology described in Patent Document 1 aims to sharpen tomographic images by reshaping OCT images representing tomographic images of the fundus of the eye into superpixels, which are created by grouping multiple pixels based on similarities in features and colors. Therefore, the image processing technology described in Patent Document 1 has the problem of not being able to meet the demand for generating new vascular images that are useful for the early detection of eye diseases and for understanding lesions in eye diseases.

[0008] This disclosure addresses the above-mentioned issues and aims to provide an image generation method and image generation system that can generate vascular images of a subject's eye that are useful for early detection of eye diseases and understanding lesions in eye diseases.

[0009] To solve the above problems, the present disclosure provides an image generation method for generating a vascular system image of an eye under examination, comprising an image data acquisition step, a first image processing step, a second image processing step, and a vascular system image generation processing step. The image data acquisition step acquires image data of the eye under examination. The first image processing step extracts the curved structure of blood vessels contained in the image data and generates image data of a vascular-enhanced image with the curved structure emphasized. The second image processing step classifies the image data of the vascular-enhanced image by the similarity of the brightness intensity of pixels, groups pixels with common similarity and divides them into superpixels to generate segmented image data of a superpixel image. The vascular system image generation processing step generates the vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image.

[0010] To solve the above problems, the image generation system for generating a vascular system image of an eye under examination according to the present disclosure comprises an image data acquisition unit, a first image processing unit, a second image processing unit, and a vascular system image generation processing unit. The image data acquisition unit acquires image data of the eye under examination. The first image processing unit generates image data of a vascular system-enhanced image by performing a vascular extraction process to extract the curved structure of blood vessels contained in the image data and a vascular enhancement process to emphasize the extracted vascular structure. The second image processing unit classifies the image data of the vascular system-enhanced image by the similarity of the brightness intensity of pixels and generates segmented image data of a superpixel image by performing a superpixel segmentation process to group pixels with common similarity and divide them into superpixels. The vascular system image generation processing unit generates the vascular system image by performing a brightness intensity process on each segmented image data of the superpixel image.

[0011] The image generation method and image generation system disclosed herein can generate vascular images of a subject's eye, which are useful for the early detection of eye diseases and for understanding lesions in eye diseases.

[0012] This is a system configuration diagram showing an example of an image generation system to which the image generation method of Embodiment 1 is applied. This is an explanatory diagram of the structure of the human eyeball. This is a flowchart showing the image processing flow for generating a vascular density map from OCTA image data of the retinal surface slab in Embodiment 1. This is a flowchart showing the image processing flow for generating a vascular map from the vascular density map of Embodiment 1. This is a flowchart showing the image processing flow for generating a non-blood flow area detection map from the vascular density map of Embodiment 1. This is a block diagram showing the image workflow from image processing in the image processing unit of Embodiment 1 up to the generation of the final image. This is an explanatory image diagram showing the transition of vascular images due to vascular extraction processing and vascular enhancement processing on OCTA image data of the retinal surface slab. This is an explanatory image diagram showing the transition of vascular images due to superpixel segmentation processing on vascular enhancement image data. This is an explanatory image diagram showing the image transition for generating a vascular density map by mean / median calculation processing from a superpixel image using OCTA original image data. This is an explanatory image diagram showing the image transition for generating a vascular density map by mean / median calculation processing from a superpixel image using OCTA noise reduction image data. This is an explanatory diagram showing the image transitions when a vascular map is generated by a first intensity value calculation process using a first threshold from each segmented image data of the vascular density map. This is an explanatory diagram showing the image transitions when a non-blood flow area detection map is generated by a second intensity value calculation process using a second threshold from each segmented image data of the vascular density map. This is a histogram data diagram showing an example of a superpixel number distribution graph for brightness intensity used to determine the first and second thresholds. This is a system configuration diagram showing an example of an image generation system to which the image generation method of Embodiment 2 is applied. This is a flowchart showing the flow of image processing to generate a feature extraction map from OCTA original image data of the outer retinal slab in Embodiment 2. This is a block diagram showing the image workflow up to the final image generation by image processing in the image processing unit of Embodiment 2. This is an explanatory diagram showing the image transitions when a feature extraction map is generated from OCTA original image data or OCTA noise-reduced image data of the outer retinal slab.

[0013] The embodiments for implementing the image generation method and image generation system of this disclosure are the image generation method and image generation system according to Embodiment 1 and Embodiment 2, which will be described below with reference to the drawings.

[0014] The image generation method and image generation system of Embodiments 1 and 2 are applied to image processing when acquiring OCTA image data from an OCT device and generating vascular images of a subject's eye from the acquired OCTA image data, which are useful for early detection of eye diseases and understanding lesions in eye diseases. Embodiment 1

[0015] [System Configuration (Figure 1)] Figure 1 shows an example of an image generation system to which the image generation method of Embodiment 1 is applied. As shown in Figure 1, the image generation system 100 includes an OCTA image data creation unit 110, a user input device 120, an image processing unit 130, a display 140, and a vascular system image data storage unit 150.

[0016] The OCTA image data creation unit 110 creates OCTA image data based on OCT image data acquired from an OCT device (optical coherence tomography) that has the function of obtaining tomographic data of the fundus. OCTA image data is data obtained by taking multiple images of the same part of the fundus using an OCT device, and then imaging only the signals that have changed from the multiple OCT image data images. In other words, since signal changes occur between OCT images in areas where blood is flowing, the OCTA image is a vascular image created by depicting only the signal portions that have changed from the multiple OCT images as blood flow information. The OCTA image data creation unit 110 may be located in the control unit of the OCT device, or it may be located in an external device capable of acquiring OCTA image data from the OCT device.

[0017] One advantage of OCTA imaging is its non-invasive nature, as it does not require contrast agents. It is safe, has no side effects, and requires a short examination time, making repeated examinations possible and suitable for understanding changes in the condition of eye diseases. Another advantage of OCTA imaging is that the cross-sectional image signal from the OCT device is acquired three-dimensionally. By segmenting the cross-sectional image signal, it is possible to observe the vascular structure by unfolding the fundus cross-section layer by layer. Therefore, the OCTA image data creation unit 110 can create OCTA image data (two-dimensional image data) layer by layer, for example, four layers with different depths in the cross-sectional direction, such as the retinal superficial slab, the retinal deep slab, the retinal outer slab, and the choroidal capillary plate slab.

[0018] Here, OCTA image data of the retinal surface slab is obtained by a C-scan method, in which the measurement light is rapidly scanned through the fundus using a galvanometer mirror, and a two-dimensional OCT image is constructed from the interference signal with the optical path of the reference mirror. Furthermore, OCTA image data of the retinal deep slab, retinal outer slab, choroidal capillary plate slab, etc., contain information in the depth direction from the retinal surface slab, and information of each layer located in the depth direction from the retinal surface slab can be obtained by moving the reference mirror along the optical axis.

[0019] The user input device 120 is an input operation device that performs various input operations based on input operations by the user (examiner). In addition to the start and end operations of image processing, the various input operations include the input operation for selecting the type of layered OCTA image data created by the OCTA image data creation unit 110, and the input operation for selecting the type of vascular image generated by the image processing unit 130. The user input device 120 of Embodiment 1 selects the OCTA image data of the retinal surface slab from the OCTA image data created layer by layer by the OCTA image data creation unit 110. The user input device 120 of Embodiment 1 selects one or more vascular images from the vascular density map, vascular map, and non-blood flow area detection map that can be generated by the image processing unit 130, based on the examiner's judgment.

[0020] The image processing unit 130 generates a vascular system image of the eye under examination through multiple stages of image processing based on the OCTA image data acquired from the OCTA image data creation unit 110. The image processing unit 130 of Embodiment 1 is capable of generating a vascular density map, a vascular map, and a non-blood flow area detection map based on the original OCTA image data of the retinal surface slab from the OCTA image data creation unit 110.

[0021] The image processing unit 130 of Embodiment 1 includes an image data acquisition unit 131, a noise reduction processing unit 132, a first image processing unit 133, a second image processing unit 134, a third image processing unit 135, a fourth image processing unit 136, and a fifth image processing unit 137. Here, the third image processing unit 135, the fourth image processing unit 136, and the fifth image processing unit 137 correspond to vascular system image generation processing units.

[0022] The image data acquisition unit 131 acquires OCTA original image data of the retinal surface slab as image data of the eye under examination. The noise reduction processing unit 132 generates OCTA noise-reduced image data by performing a noise reduction process to remove noise from the OCTA original image data of the retinal surface slab. The first image processing unit 133 generates image data of a vascular-enhanced image by performing a vascular extraction process to extract the curved structure of blood vessels contained in the OCTA original image data or OCTA noise-reduced image data, and a vascular enhancement process to emphasize the extracted vascular structure. The second image processing unit 134 generates segmented image data of a superpixel image by performing a superpixel segmentation process that classifies the image data of the vascular-enhanced image data by the similarity of the brightness intensity of pixels, and groups pixels with common similarity into superpixels.

[0023] The third image processing unit 135 generates a vascular density map of the retinal surface slab by performing an average / median calculation process (an example of brightness intensity processing) that calculates the average or median brightness value for each segmented image data of the superpixel image. The fourth image processing unit 136 determines a first threshold for the vascular density map that classifies brightness intensity regions containing blood vessels with a diameter exceeding a predetermined diameter. The fourth image processing unit 136 generates a vascular map of the retinal surface slab by performing a first intensity value calculation process (an example of brightness intensity processing) that retains image data equal to or greater than the first threshold from each segmented image data of the vascular density map. The fifth image processing unit 137 determines a second threshold for the vascular density map that classifies brightness intensity regions containing capillaries. The fifth image processing unit 137 generates a non-blood flow region detection map of the retinal surface slab by performing a second intensity value calculation process (an example of brightness intensity processing) that retains image data equal to or less than the second threshold from each segmented image data of the vascular density map.

[0024] The display 140 displays the final vascular system image of the retinal surface slab selected by the user through operation on the user input device 120. The display 140 may also display vascular system images that change at each stage of image processing when the image processing unit 130 performs image processing. In Embodiment 1, the display 140 displays one or more vascular system images selected by the user from a vascular density map, a vascular map, and a non-blood flow area detection map as the final vascular system image of the retinal surface slab. Therefore, examiners with specialized knowledge and experience, such as ophthalmologists, can determine whether or not a patient has an eye disease by looking at the vascular system image displayed on the screen of the display 140, and can also determine the degree of disease progression if an eye disease is determined to be present. Thus, the examiner's determination that the patient is in the early stages of eye disease progression can be used for early detection of eye disease and early treatment of eye disease.

[0025] The vascular image data storage unit 150 stores vascular image data of the retinal surface slab generated by the image processing unit 130. Therefore, the examiner can, for example, operate the user input device 120 to display past and current vascular images of the same patient, read from the vascular image data storage unit 150, side by side on the screen of the display 140. Thus, by comparing past and current vascular images displayed side by side on the display 140, the examiner can use this to understand the progression and improvement of eye disease lesions over time.

[0026] [Details of the eyeball structure (Figure 2)] Details of the eyeball structure are necessary information for understanding the image processing technology of this disclosure, and will therefore be explained based on Figure 2, which shows the eyeball structure.

[0027] The retina 201 of the human eyeball 200 corresponds to the film or image sensor part of a camera. The cross-sectional structure of the fundus has the retina 201 on the inner surface of the fundus, the choroid 202 outside the retina 201, and the sclera 203 outside the choroid 202. The retina 201 has an optic disc 205 where the optic nerves 204 converge at its center (not the optical center), and is in contact with the vitreous humor 206. The retina 201 has a macula 207, which has high visual acuity, in the part corresponding to the center of the visual field, and is in contact with the vitreous humor 206. When a lesion occurs in the macula 207, the central part of the visual field may appear distorted, or depending on the condition, objects may appear larger or smaller, or the area being looked at may become dark (central scotoma), directly affecting visual acuity. The retina 201 has a fovea 208 at the center of the macula 207, which is the part of the macula 207 with the best visual acuity, and is slightly concave than the surrounding retina 201. The fovea 208 has a special structure that allows for high visual acuity, as it contains no blood vessels other than cone cells, which are highly sensitive to vision.

[0028] The cornea 209 and lens 210 of the human eyeball 200 correspond to the lens part of a camera. The cornea 209 is the outermost part of the iris, and the lens 210 is located within it. The cornea 209 and lens 210 have very similar functions, both of which adjust the refraction of light so that the visible world is not distorted, and project it onto the retina 201.

[0029] The retina 201 of the human eyeball 200 has blood vessels 211 in its superficial slab. These blood vessels 211 originate from the ophthalmic artery, which branches directly from the cervical aorta, passes through the optic nerve 204 (central retinal artery), and then branches into four at the optic disc 205 (retinal branching arteries) to supply the retinal nerve layer. The retinal arteries return via capillaries to become four retinal veins (retinal branching veins), which then become one central retinal vein at the optic disc 205, and finally return to the heart as cervical veins. Therefore, the blood vessels 211 in the superficial slab of the retina 201 have a vascular structure that includes both large blood vessels with a wide diameter and small capillaries with a narrow diameter.

[0030] [Details of Image Processing (Figures 3-5)] Details of the image processing for generating the blood vessel density map will be explained based on the flowchart in Figure 3, which shows the flow of image processing performed by the image processing unit 130.

[0031] Step S01 is the step of acquiring the original OCTA image data of the retinal surface slab from the OCTA image data generated by the OCTA image data creation unit 110 as image data of the eye under examination. Step S01 corresponds to the image data acquisition step.

[0032] Step S02 is a step in which noise is removed from the original OCTA image data of the retinal surface slab acquired in step S01 by noise reduction processing to obtain OCTA noise-reduced image data. Here, the noise reduction processing mainly targets the removal of speckle noise (such as spot-like noise when laser light is irradiated) and random noise caused by electronic noise inherent in the original OCTA image data. Step S02 corresponds to the noise reduction processing step.

[0033] Note that the noise reduction process in step S02 is not essential for performing the various image processing steps from step S03 onward. Therefore, the various image processing steps from step S03 onward may be performed using either the OCTA original image data or the OCTA noise reduction image data.

[0034] Step S03 is a step in which the curved structure of blood vessels contained in the OCTA original image data or OCTA noise-reduced image data is extracted by a blood vessel extraction process, and the extracted blood vessel structure is enhanced by an enhancement process to generate image data of a blood vessel-enhanced image. Step S03 corresponds to the first image processing step.

[0035] Step S04 is a step in which the image data of the blood vessel-enhanced image is classified by the similarity of the brightness intensity of the pixels, and pixels with common similarity are grouped and divided into super pixels, thereby generating image data of a super pixel image. Here, the image data of the super pixel image becomes image data divided by super pixel regions. Step S04 corresponds to the second image processing step.

[0036] Step S05 is a step in which a blood vessel density map is generated by an average / median calculation process that calculates the average or median value of brightness intensity for each segmented image data of the superpixel image. Step S05 corresponds to the third image processing step (vascular system image generation processing step).

[0037] The details of the image processing that generates the blood vessel map will be explained based on the flowchart in Figure 4, which shows the flow of image processing performed by the image processing unit 103.

[0038] Step S06 is the step of obtaining a vascular density map generated by the flowchart shown in Figure 3.

[0039] Step S07 is a step in which a first threshold is set for the vascular density map to classify brightness intensity regions that include blood vessels with a diameter exceeding a predetermined diameter. Step S07 is a step in which a vascular map is generated by a first intensity value calculation process that retains image data of the first threshold or higher from each segmented image data of the vascular density map. In other words, this first intensity value calculation process is an image processing that excludes image data below the first threshold from each segmented image data of the vascular density map, and an example of determining the first threshold is shown in Figure 13. Step S07 corresponds to the fourth image processing step (vascular system image generation processing step).

[0040] Details of the image processing that generates the non-blood flow area detection map will be explained based on the flowchart in Figure 5, which shows the flow of image processing performed by the image processing unit 103.

[0041] Step S08 is the step of obtaining a vascular density map generated by the flowchart shown in Figure 3.

[0042] Step S09 determines a second threshold for the vascular density map, which is used to distinguish brightness intensity regions containing capillaries. Step S09 is a step in which a non-blood flow region detection map is generated by a second intensity value calculation process that retains image data below the second threshold from each segmented image data of the vascular density map. In other words, this second intensity value calculation process is an image processing that excludes image data exceeding the second threshold from each segmented image data of the vascular density map, and Figure 13 shows an example of determining the second threshold (< first threshold). Step S09 corresponds to the fifth image processing step (vascular system image generation processing step).

[0043] [Details of the Image Workflow (Figure 6)] Details of the image workflow 300 will be explained based on the block diagram in Figure 6, which shows the image workflow up to the final image generation by image processing in the image processing unit 103 of Embodiment 1. The image workflow 300 is broadly divided into two image workflows: the blood vessel enhancement image flow section 310 and the map generation image flow section 320.

[0044] The image flow section 310 for vessel enhancement has an OCTA original image 311, an OCTA noise-removed image 312, a vessel-extracted image 313, and a vessel-enhanced image 314.

[0045] The OCTA original image 311 is an image formed by image data obtained from OCTA image data of the retinal surface slab acquired from an OCT device without performing preprocessing such as noise removal. The OCTA noise-removed image 312 is an image formed by image data after performing preprocessing by noise removal on the OCTA original image data of the retinal surface slab acquired from the OCT device. The vessel-extracted image 313 is an image formed by image data after performing a vessel extraction process (A) on the OCTA original image data or the OCTA noise-removed image data. The vessel-enhanced image 314 is an image formed by image data after performing a vessel enhancement process (B) of adding vessel-extracted image data with specific weighting to the original image data (OCTA original image data or OCTA noise-removed image data).

[0046] The image flow section 320 for map generation has a superpixel image 321, a vessel density map 322, a vessel map 323, and a non-blood flow region detection map 324.

[0047] The superpixel image 321 is an image generated by image data after performing superpixel segmentation processing (C) on the vessel-enhanced image data. The vessel density map 322 is an image generated by image data after performing average value / median value calculation processing (D) on each divided image data of the superpixel image 321. The vessel map 323 is an image generated by image data after performing first intensity value calculation processing (E) on each divided image data of the vessel density map 322. The non-blood flow region detection map 324 is an image generated by image data after performing second intensity value calculation processing (F) on each divided image data of the vessel density map 322.

[0048] [Details of blood vessel extraction and blood vessel enhancement processing (Figure 7)] Details of blood vessel extraction processing (A) and blood vessel enhancement processing (B) will be explained based on Figure 7, which shows the image transition due to image processing on the original OCTA image data of the retinal surface slab.

[0049] The OCTA original image 311 of the retinal surface slab transitions to a vascular extraction image 313 via vascular extraction processing (A), and further transitions to a vascular enhancement image 314 via vascular enhancement processing (B).

[0050] The blood vessel extraction process (A) uses a blood vessel extraction filter to extract the blood vessel structure from the image data of the OCTA original image 311, in which the blood vessel structure is not clearly depicted, by applying a filter that reveals the shape and pattern unique to blood vessels, thereby obtaining a blood vessel extraction image 313. Here, as the blood vessel extraction filter, for example, a bar-circular symmetric filter (BCOSFIRE), which is designed to detect curved structures such as blood vessels, can be used. Note that the blood vessel extraction process (A) may be applied to the OCTA denoised image 312 of the retinal surface slab instead of the OCTA original image 311 of the retinal surface slab.

[0051] The vascular enhancement process (B) adds vascular extraction image data with specific weights to the original image data (OCTA original image data or OCTA noise reduction image data) to create a vascular enhancement image 314 using image data that enhances the vascular image. Here, "specific weights" means that the image portion of the vascular extraction image data is weighted more highly in the brightness intensity range of the vascular portion, including large vessels and capillaries, than in the non-vascular portion, thereby further emphasizing the vascular structure in the retinal surface slab. As a result, the vascular enhancement image 314, as shown at the right end of Figure 7, not only is the large vessel clearly enhanced, but even the capillaries are clearly visible. Note that the vascular enhancement process (B) may also be a filtering process using a weighted-guided image filter or the like.

[0052] [Details of Super Pixel Segmentation Processing (Figure 8)] Details of the super pixel segmentation processing (C) will be explained based on Figure 8, which shows the image transitions due to image processing on vascular-enhanced image data.

[0053] The enhanced vascular-enhanced image 314 of the retinal surface slab transitions to a superpixel image 321 via superpixel segmentation processing (C). In other words, the superpixel image 321 is obtained by applying superpixel segmentation processing (C) to the image data of the vascular-enhanced image 314.

[0054] Superpixel segmentation (C) is a process that classifies pixels based on the similarity of their brightness intensity, groups pixels with common similarities, and divides them into superpixels. In the vascular-enhanced image 314 of the retinal surface slab, the vascular structures appear brighter, while non-perfused areas appear darker compared to the surrounding tissue with blood flow. Therefore, superpixel segmentation (C) groups pixels with similar brightness intensity characteristics, dividing the image data of the vascular-enhanced image of the retinal surface slab into thousands of small pixel groups. As shown in Figure 8, the superpixel image 321 after superpixel segmentation (C) is drawn with whitish lines in the grayscale image, but in the actual color image, it becomes a raised pattern drawn with yellow lines against a dark background. These yellow lines in the superpixel image 321 are boundaries that show how the image was divided by superpixel segmentation (C).

[0055] The reason for employing superpixel segmentation (C) is to perform efficient image processing and analysis by treating image data not in the usual pixel units, but in pixel groups formed by dividing the image data into localized superpixel regions. In particular, superpixel segmentation (C) for vascular-weighted image data of retinal surface slabs is useful for efficient analysis and quantification of perfused and non-perfused regions where blood flows.

[0056] [Details of Mean / Median Calculation Process (Figures 9 and 10)] Details of the mean / median calculation process (D) will be explained based on Figures 9 and 10, which show the image transitions due to image processing on the superpixel image 321. Note that the superpixel image 321 in Figure 9 is an image based on the OCTA original image data of the retinal surface slab. The superpixel image 321 in Figure 10 is an image based on the OCTA noise-reduced image data of the retinal surface slab.

[0057] The superpixel image 321, based on the original OCTA image data or OCTA noise-reduced image data of the retinal surface slab, transitions to the vascular density map 322 image via an average / median calculation process (D). In other words, the vascular density map 322 image is obtained by applying the average / median calculation process (D) to the superpixel region of each segmented image data of the superpixel image 321.

[0058] The mean / median calculation process (D) calculates the mean or median brightness intensity of the superpixel region of each segmented image data in the superpixel image 321. Then, the mean / median calculation process (D) replaces each segmented image data of the superpixel region with the mean or median brightness intensity to obtain the vascular density map 322 image. The vascular density map 322 image is a color image display where the vascular images are prominent. The vascular images in the vascular density map 322 are displayed in a light gray color in the grayscale images of Figures 9 and 10, but are displayed in a bright red color in the actual color image. On the other hand, areas of the vascular density map 322 without blood flow (avascular region FAZ, non-perfusion region NPA) have low brightness intensity values ​​because there is no blood flow, and are displayed in a dark gray color in the grayscale images of Figures 9 and 10, but are displayed in a dark blue color in the actual color image.

[0059] Here, the foveal avascular zone (FAZ) refers to the avascular area surrounded in a ring by retinal ciliary tissue, etc., centered around the fovea 208 of the macula 207 of the retina 201 (see Figure 2). The non-perfusion area (NPA) refers to the avascular area in the superficial slab of the retina 201 due to capillary occlusion (see Figure 2).

[0060] Therefore, the color vascular density map 322 image generated by the mean / median calculation process (D) is an image in which the vascular regions and non-vascular regions of the retinal surface slab are distinguished by color coding. Furthermore, the difference between the two images shown in Figure 9 and the two images shown in Figure 10 is that the vascular structure is more clearly shown in the superpixel image 321 of Figure 10, which is based on OCTA noise reduction image data, compared to the superpixel image 321 of Figure 9, which is based on OCTA original image data. For this reason, although the color vascular density map 322 is generated by color coding of vascular regions (red) and non-vascular regions (blue) by the mean / median calculation process (D), the color coding distinction in Figure 10 is more precise than that of the vascular density map 322 of Figure 9.

[0061] [Details of the First Intensity Value Calculation Process (Figures 11 and 13)] The details of the first intensity value calculation process (E) will be explained based on Figure 11, which shows the image transition due to image processing on the blood vessel density map 322, and Figure 13, which shows an example of the superpixel number distribution with respect to brightness intensity used to determine the first threshold.

[0062] The image of the vascular density map 322, based on the original OCTA image data or the OCTA noise-reduced image data of the retinal surface slab, transitions to the image of the vascular map 323 via a first intensity value calculation process (E). In other words, the image of the vascular map 323 is obtained by applying the first intensity value calculation process (E) to the superpixel regions of each segmented image data of the vascular density map 322.

[0063] The first intensity value calculation process (E) determines a first threshold value for each segmented image data of the blood vessel density map 322, which is used to distinguish brightness intensity regions that include blood vessels with a diameter exceeding a predetermined diameter. The first intensity value calculation process (E) is an image processing method that generates image data for the blood vessel map 323 by retaining image data that is equal to or greater than the first threshold value from each segmented image data of the blood vessel density map 322.

[0064] In other words, the vascular region of the retinal surface slab contains blood flow information, so it is displayed brightly in the OCTA image and bright red in the color vascular density map 322. Therefore, the vascular map 323 can be calculated by retaining the regions with high blood flow within the vascular region using a threshold (first threshold). The first threshold can be determined by existing knowledge or histogram analysis. In the case of histogram analysis, as shown in Figure 13, a brightness intensity threshold is determined as the first threshold for classifying the brightness intensity region with high blood flow, which retains blood vessels with a diameter exceeding a predetermined diameter and excludes capillaries with a diameter less than a predetermined diameter.

[0065] Thus, the image of the vascular map 323 generated by the first intensity value calculation process (E) can clearly display large arteries, several capillaries, and even macular neovascularization structures, as shown in the image on the right of Figure 11. For this reason, the information on the location of blood vessels in the vascular map 323 is useful for detecting non-perfusion areas (NPAs) that appear around large arteries, and for detecting an eye disease called macular neovascularization.

[0066] [Details of the second intensity value calculation process (Figures 12 and 13)] The details of the second intensity value calculation process (F) will be explained based on Figure 12, which shows the image transition due to image processing on the vascular density map 322, and Figure 13, which shows an example of the superpixel number distribution with respect to brightness intensity used to determine the second threshold.

[0067] The image of the vascular density map 322, based on the original OCTA image data or OCTA noise-reduced image data of the retinal surface slab, transitions to the image of the non-blood flow region detection map 324 via a second intensity value calculation process (F). In other words, the image of the non-blood flow region detection map 324 is obtained by applying the second intensity value calculation process (F) to the superpixel region of each segmented image data of the vascular density map 322.

[0068] The second intensity value calculation process (F) determines a second threshold (<first threshold) for each segmented image data of the blood vessel density map 322 that classifies the brightness intensity region containing capillaries. The second intensity value calculation process (F) is an image processing that generates image data for the non-blood flow region detection map 324 by retaining image data below the second threshold from each segmented image data of the blood vessel density map 322.

[0069] In other words, since there is no blood flow information in the non-perfusion area (NPA) and the avascular area (FAZ) of the cavity, they appear dark in the OCTA image of the retinal surface slab and are displayed in dark blue in the color vascular density map 322 image. Each segmented image data of the non-blood flow area detection map 324 can be calculated by retaining areas with low blood flow within the vascular region using a threshold (second threshold). The second threshold can be determined using existing knowledge or histogram analysis. In the case of histogram analysis, as shown in Figure 13, a second threshold (<first threshold) is determined to separate the brightness intensity areas with low blood flow, leaving capillaries with low blood flow and areas with no blood flow, and excluding other blood vessels.

[0070] Thus, the image of the non-blood-flowing area detection map 324 generated by the second intensity value calculation process (F) displays areas with little blood flow and areas with no blood flow. The image display of areas with little blood flow and areas with no blood flow can be used to detect the avascular zone (FAZ) and non-perfusion zone (NPA) of the cavity where there is no blood flow. Furthermore, in the image of the non-blood-flowing area detection map 324 generated by the second intensity value calculation process (F), excluded large vessels are displayed in white. These large vessels displayed in white represent the image display positions of large arteries in the vascular structure and can be used to detect the non-perfusion zone (NPA). This is because the non-perfusion zone (NPA) usually appears around arteries. Therefore, the non-blood-flowing area detection map 324 can be used to detect the avascular zone (FAZ) and non-perfusion zone (NPA) of the cavity where there is no blood flow, and the accuracy of detecting the non-perfusion zone (NPA) can be further improved.

[0071] [Regarding the comparison of vascular density maps] Existing vascular density algorithms that generate vascular density maps use OCTA images and scan parameters as inputs necessary for calculating vascular density.

[0072] The process for generating existing vascular density maps consists of the following steps: 1. Step of creating OCTA surface images. 2. Step of creating a binarized map based on the brightness intensity of the OCTA surface images. The binarized map (vascular segmentation map) is created using a binarization algorithm based on the local adaptive thresholding method, where the brightness intensity of each pixel is compared with the average brightness intensity of its neighbors. The size of the neighboring region depends on the scan pattern. The binarized map from the binarized images is denoised to reduce noise before calculating the vascular density. 3. Step of creating a vascular density map from the binarized map. The vascular density is calculated by averaging the binarized map over a predefined (4x4) region. The calculated vascular density is then smoothed using a two-dimensional averaging filter (15x15) to create a vascular density map.

[0073] This vascular density algorithm uses a binary map (a map with a data format represented by "0"s and "1"s) instead of a vascular density map for 3D disk scans displaying vascular density. In other cases, it converts the vascular density map to a color map using a lookup table for display. Therefore, existing vascular density maps rely on thresholding and averaging of small regions and are heavily affected by artifacts (such as virtual images that are introduced during the scan and do not actually exist) and noise.

[0074] In contrast, the vascular density algorithm for generating the vascular density map 322 in Embodiment 1 does not include scan parameters in the inputs required for calculating vascular density.

[0075] The process for generating the vascular density map 322 in Embodiment 1 consists of the following steps: 1. Image data acquisition step to acquire OCTA image data; 2. First image processing step to generate image data of a vascular-enhanced image 314 from OCTA image data by performing a vascular extraction process (A) and a vascular enhancement process (B); 3. Second image processing step to generate segmented image data of a superpixel image 321 from the vascular-enhanced image data by performing a superpixel segmentation process (C); 4. Third image processing step to generate the vascular density map 322 from the segmented image data of the superpixel image 321 by performing an average / median calculation process (D).

[0076] Thus, the vascular density map 322 generated by the image processing of Embodiment 1 employs a vascular density algorithm that uses first and second image processing, and does not include the step of creating a binarized map that is present in existing vascular density algorithms. Therefore, the vascular density map 322 can be prevented from being affected by artifacts and noise, unlike existing vascular density algorithms that include a step of creating a binarized map.

[0077] [Relationship between each of the three types of map images and eye diseases] The image generation method and image generation system of Embodiment 1 can generate vascular images of the retinal surface slab using three types of maps: a vascular density map 322, a vascular map 323, and a non-blood flow area detection map 324. Eye diseases that can be detected by the three types of map images are diseases that can be observed as changes from the normal vascular structure in the retinal surface slab in the octave-coherent octa-analyte (OCTA) image of the retinal surface slab. The relationship between each of the three types of map images and eye diseases is explained below separately for each map image.

[0078] (Vascular Density Map 322) The vascular density map 322 image, as shown in Figures 9 and 10, is useful for measuring the vascular density of specific areas of the retinal surface slab. Therefore, eye diseases closely related to detection include diabetic retinopathy, glaucoma, and retinal vein occlusion.

[0079] Diabetic retinopathy is a complication of diabetes that affects the retina (201) of the eyeball (200). It is a disease in which the capillaries spreading throughout the retina (201) are damaged when high blood sugar levels persist for a long time. Therefore, if the vascular density map (322) shows a decrease in vascular density from normal in the displayed image, it may indicate ischemia or a perfusion area, which is characteristic of diabetic retinopathy.

[0080] Glaucoma is a disease that causes abnormalities in the optic nerve 204 (the nerve that transmits information from the eye to the brain) and the visual field. Therefore, in the vascular density map 322 image, a decrease in vascular density in the area surrounding the optic nerve head 205 in the displayed image indicates damage to the optic nerve 204 and may be a sign of glaucoma.

[0081] Retinal vein occlusion is an eye disease caused by blockage of a large vein in the retina (vein 201), resulting in stagnation of blood flow. Therefore, if there are areas of reduced blood flow in the vascular density map (vein 322) image, it may be due to vein occlusion.

[0082] (Vascular map 323) As shown in Figure 11, the image of the vascular map 323 helps to provide a detailed image of the blood vessels 211 by highlighting the large and small blood vessels of the retina 201. For this reason, eye diseases that are closely related to detection include diabetic retinopathy, hypertensive retinopathy, and macular degeneration.

[0083] As mentioned above, "diabetic retinopathy" is a disease in which the capillaries spreading across the retina 201 are damaged when high blood sugar levels persist for a long period. Therefore, the image of the blood vessel map 323 allows for the identification of microaneurysms, hemorrhages, and neovascularization by displaying detailed images of the blood vessels 211.

[0084] "Hypertensive retinopathy" refers to a disease in which high blood pressure causes damage to the retina 201 and its capillaries. Therefore, the image of the vascular map 323, by displaying a detailed image of the blood vessels 211, can show changes in the diameter of the blood vessels 211, such as narrowing of the blood vessels 211, damage to arteries and veins, and other signs of vascular damage due to high blood pressure.

[0085] "Macular degeneration" refers to a disease in which damage occurs to the macula 207, the central part of the retina 201, making it difficult to see what you are trying to look at. Therefore, the image of the blood vessel map 323 allows for the identification of abnormal blood vessels or macular neovascularization near the macula 207 by displaying detailed images of the blood vessels 211.

[0086] (Non-blood-flowing area detection map 324) As shown in Figure 12, the image of the non-blood-flowing area detection map 324 can highlight non-perfusion areas (NPAs) where there is no blood flow, and is useful for evaluating the size and shape of the cavity avascular area (FAZ). For this reason, eye diseases that are closely related to detection targets include diabetic retinopathy, macular ischemia, and retinal vein occlusion.

[0087] As mentioned above, "diabetic retinopathy" is a disease in which the capillaries spread across the retina 201 are damaged when high blood sugar levels persist for a long period. Therefore, the image of the non-perfusion area detection map 324, by displaying the non-perfusion area (NPA) where there is no blood flow, can be used to determine if a large non-perfusion area (NPA) indicates an advanced disease stage accompanied by a significant decrease in capillaries.

[0088] "Macular ischemia" is an irreversible condition commonly seen in diabetic retinopathy and can lead to blindness. Therefore, the image of the non-blood flow area detection map 324, based on the display image of the cavity avascular area FAZ, may suggest macular ischemia, which is often associated with diabetic retinopathy or retinal vein occlusion, depending on changes in the size and irregularities of the cavity avascular area FAZ.

[0089] "Retinal vein occlusion," as described above, is an eye disease that develops when a large vein in the retina 201 becomes blocked, causing blood flow to stagnate. Therefore, the image of the non-blood flow area detection map 324 can be used to determine the non-perfusion and ischemic areas caused by vein occlusion by displaying the non-perfusion area (NPA) where there is no blood flow.

[0090] (Summary) Images from the vascular density map 322 can be used primarily to detect ocular diseases related to vascular perfusion, such as diabetic retinopathy, glaucoma, and retinal vein occlusion. Images from the vascular map 323 are more detailed and can identify changes in both small and large vessels, making them useful for detecting conditions such as diabetic retinopathy, hypertensive retinopathy, and macular degeneration. Images from the non-blood flow area detection map 324 are important for identifying ischemic areas and evaluating the cavity avascular area (FAZ), and can provide analytical information regarding conditions such as diabetic retinopathy, macular ischemia, and retinal vein occlusion.

[0091] [Effects of the Image Generation Method and Image Generation System] The image generation method and image generation system 100 of Embodiment 1 have the following effects.

[0092] (1) The image generation method for generating a vascular system image of an eye under examination comprises an image data acquisition step (S01), a first image processing step (S03), a second image processing step (S04), and a vascular system image generation processing step (S05, etc.). The image data acquisition step acquires image data of the eye under examination. The first image processing step extracts the curved structure of blood vessels contained in the image data and generates image data of a vascular-enhanced image 314 in which the curved structure is emphasized. The second image processing step classifies the image data of the vascular-enhanced image 314 by the similarity of the brightness intensity of pixels, groups pixels with common similarity and divides them into superpixels to generate segmented image data of a superpixel image 321. The vascular system image generation processing step generates a vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image 321. This image generation method can generate a vascular system image of an eye under examination that is useful for the early detection of eye diseases and for understanding lesions of eye diseases.

[0093] (2) In the image data acquisition step (S01), multiple images of the same area of ​​the fundus of the eye under examination are taken using an OCT device that can obtain tomographic data, and OCTA image data is obtained by depicting only the part of the multiple OCT images that changes due to blood flow as blood flow information. This image generation method has the advantage of obtaining vascular image data that is safe and has no side effects and is useful for examining eye diseases by obtaining OCTA image data without using contrast agents as image data of the eye under examination.

[0094] (3) The image data acquisition step (S01) acquires image data of the retinal surface slab from the OCTA image data. This image generation method acquires vascular image data depicting the blood vessels 211 distributed in the surface slab of the retina 201 by acquiring image data of the retinal surface slab from the OCTA image data.

[0095] (4) The vascular system image generation process includes a third image processing step (S05) which generates a vascular density map 322 of the retinal surface slab by performing an average / median calculation process (D) which calculates the average or median brightness intensity for each segmented image data of the superpixel image. This image generation method can generate an image of a vascular density map 322 of the retinal surface slab, which is useful for detecting eye diseases that appear in vascular density, by performing an average / median brightness intensity calculation process (D) for each segmented image data of the superpixel image 321.

[0096] (5) The vascular system image generation process determines a first threshold for the vascular density map 322 that divides the brightness intensity region into areas containing blood vessels with a diameter exceeding a predetermined diameter. The vascular system image generation process includes a fourth image processing step (S07) which generates a vascular map 323 of the retinal surface slab by a first intensity value calculation process (E) that leaves image data of the vascular density map 322 that is equal to or greater than the first threshold. This image generation method can generate an image of the vascular map 323 of the retinal surface slab, in which the vascular structure is more detailed and changes in both small and large blood vessels can be identified, by a first intensity value calculation process (E) of brightness intensity for each segmented image data of the vascular density map 322.

[0097] (6) The vascular system image generation process determines a second threshold for the vascular density map 322 that divides the brightness intensity region containing capillaries. The vascular system image generation process includes a fifth image processing step (S09) which generates a non-blood-flow region detection map 324 of the retinal surface slab by a second intensity value calculation process (F) that leaves image data below the second threshold from each segmented image data of the vascular density map 322. This image generation method can generate an image of a non-blood-flow region detection map 324 of the retinal surface slab that can identify non-perfusion regions (NPAs) and evaluate cavity avascular regions (FAZs) by a second intensity value calculation process (F) of brightness intensity for each segmented image data of the vascular density map 322.

[0098] (7) The method includes a noise reduction process (S02) to remove at least speckle noise and random noise from the OCTA image data of the retinal surface slab acquired in the image data acquisition process (S01). This image generation method removes unwanted noise components from the image data of the vascular structure, thereby enabling the acquisition of OCTA noise-reduced image data with clearer vascular images compared to the original OCTA image data of the retinal surface slab.

[0099] (8) The image generation system 100 for generating a vascular system image of the eye under examination comprises an image data acquisition unit 131, a first image processing unit 133, a second image processing unit 134, and a vascular system image generation processing unit (such as a third image processing unit 135). The image data acquisition unit 131 acquires image data of the eye under examination. The first image processing unit 133 generates image data of a vascular-enhanced image by performing a vascular extraction process (A) to extract the curved structure of blood vessels contained in the image data and a vascular enhancement process (B) to emphasize the extracted vascular structure. The second image processing unit 134 generates segmented image data of a superpixel image by performing a superpixel segmentation process (C) to classify the image data of the vascular-enhanced image by the similarity of the brightness intensity of pixels and group pixels with common similarity into superpixels. The vascular system image generation processing unit generates a vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image. This image generation system 100 can generate a vascular system image of the eye under examination that is useful for the early detection of eye diseases and for understanding lesions of eye diseases. Embodiment 2

[0100] Embodiment 2 is an embodiment in which, unlike Embodiment 1 which generates a vascular image based on OCTA image data of the retinal surface slab, the vascular image is generated based on OCTA image data of the retinal outer layer slab, which is deeper than the retinal surface slab.

[0101] [System Configuration (Figure 14)] Figure 14 shows an example of an image generation system to which the image generation method of Embodiment 2 is applied. As shown in Figure 14, the image generation system 100' includes an OCTA image data creation unit 110, a user input device 120, an image processing unit 130', a display 140, and a vascular system image data storage unit 150. The OCTA image data creation unit 110, user input device 120, display 140, and vascular system image data storage unit 150, excluding the image processing unit 130', have the same configuration as in Embodiment 1, so their description is omitted.

[0102] The image processing unit 130' can acquire original OCTA image data of the outer retinal slab from the OCTA image data creation unit 110, and generate a first feature extraction map and a second feature extraction map of the outer retinal slab based on this original OCTA image data.

[0103] The image processing unit 130' of Embodiment 2 includes an image data acquisition unit 131', a noise reduction processing unit 132', a first image processing unit 133', a second image processing unit 134', a third image processing unit 135', and a fourth image processing unit 136'. Here, the third image processing unit 135' and the fourth image processing unit 136' correspond to vascular system image generation processing units.

[0104] The image data acquisition unit 131' acquires OCTA original image data of the outer retinal slab as image data of the eye under examination. The noise reduction processing unit 132' generates OCTA noise-reduced image data by performing a noise reduction process to remove noise from the OCTA original image data of the outer retinal slab. The first image processing unit 133' generates image data of a vascular-enhanced image by performing a vascular extraction process to extract the curved structure of blood vessels contained in the OCTA original image data or the OCTA noise-reduced image data, and a vascular enhancement process to emphasize the extracted vascular structure. The second image processing unit 134' generates segmented image data of a superpixel image by performing a superpixel segmentation process that classifies the image data of the vascular-enhanced image data by the similarity of the brightness intensity of pixels, and groups pixels with common similarity into superpixels.

[0105] The third image processing unit 135' generates a first feature extraction map of the outer retinal slab (corresponding to the blood vessel density map in Embodiment 1) by performing an average / median calculation process that calculates the average or median brightness intensity for each segmented image data of the superpixel image. The fourth image processing unit 136' determines a first threshold for the first feature extraction map that divides the brightness intensity region containing blood vessels with a diameter exceeding a predetermined diameter. The fourth image processing unit 136' generates a second feature extraction map of the outer retinal slab (corresponding to the blood vessel map in Embodiment 1) by performing a first intensity value calculation process that retains image data equal to or greater than the first threshold from each segmented image data of the first feature extraction map. The outer retinal slab refers to the outer layer of the retina 201 that is in contact with the choroid 202, and is the layer in which photoreceptor cells that sense light are lined up.

[0106] [Details of Image Processing (Figure 15)] Details of the image processing that generates the vascular density map and the vascular map will be explained based on the flowchart in Figure 15, which shows the flow of image processing performed by the image processing unit 130'.

[0107] Step S21 is the step of acquiring the original OCTA image data of the outer retinal slab from the OCTA image data generated by the OCTA image data creation unit 110 as image data of the eye under examination. Step S21 corresponds to the image data acquisition step.

[0108] Step S22 is a step in which noise is removed from the original OCTA image data of the outer retinal slab acquired in step S21 by noise reduction processing to obtain OCTA noise-reduced image data. Note that the noise reduction processing in step S22 is not an essential process for carrying out the various image processing steps from step S23 onward. Step S22 corresponds to the noise reduction processing step.

[0109] Step S23 is a step in which the curved structure of blood vessels contained in the OCTA original image data or OCTA noise-reduced image data is extracted by a blood vessel extraction process, and the extracted blood vessel structure is enhanced by an enhancement process to generate image data of a blood vessel-enhanced image. Step S23 corresponds to the first image processing step.

[0110] Step S24 is a step in which the image data of the blood vessel-enhanced image is classified by the similarity of the brightness intensity of the pixels, and pixels with common similarity are grouped and divided into super pixels, thereby generating image data of a super pixel image. Here, the image data of the super pixel image becomes image data divided by super pixel regions. Step S24 corresponds to the second image processing step.

[0111] Step S25 is a step in which a first feature extraction map of the outer retinal slab is generated by an average / median calculation process that calculates the average or median value of brightness intensity for each segmented image data of the superpixel image. Step S25 corresponds to the third image processing step (vascular system image generation processing step).

[0112] Step S26 is a step in which a first threshold is set for each segmented image data of the first feature extraction map to classify brightness intensity regions that contain blood vessels with a diameter exceeding a predetermined diameter. Step S26 is a step in which a second feature extraction map of the outer retinal slab is generated by a first intensity value calculation process that retains image data of the first threshold or higher from each segmented image data of the first feature extraction map. Step S26 corresponds to the fourth image processing step (vascular system image generation processing step).

[0113] [Details of the Image Workflow (Figure 16)] Details of the image workflow 300 will be explained based on the block diagram in Figure 16, which shows the image workflow up to the final image generation by image processing in the image processing unit 103' of Embodiment 2. The image workflow 300 is broadly divided into two image workflows: the blood vessel enhancement image flow section 310 and the map generation image flow section 320.

[0114] The blood vessel-enhanced image flow unit 310 includes an OCTA original image 315, an OCTA noise-reduced image 316, a blood vessel extraction image 317, and a blood vessel-enhanced image 318.

[0115] The OCTA original image 315 is an image obtained from an OCT device using OCTA image data of the outer retinal slab without any preprocessing such as noise reduction. The OCTA noise reduction image 316 is an image obtained from an OCT device using OCTA original image data of the outer retinal slab after preprocessing by noise reduction. The vascular extraction image 317 is an image obtained from an OCTA original image data or OCTA noise reduction image data after applying vascular extraction processing (A). The vascular enhancement image 318 is an image obtained from an vascular enhancement processing (B) which adds vascular extraction image data with a specific weight to the original image data (OCTA original image data or OCTA noise reduction image data).

[0116] The map generation image flow unit 320 includes a superpixel image 325, a first feature extraction map 326, and a second feature extraction map 327.

[0117] The superpixel image 325 is an image obtained by dividing the image data of the blood vessel-enhanced image 318 into thousands of superpixels after applying superpixel segmentation processing (C). The first feature extraction map 326 is an image obtained by applying mean / median calculation processing (D) to each segment of the superpixel image 325. The second feature extraction map 327 is an image obtained by applying first intensity value calculation processing (E) to each image data of the first feature extraction map 326.

[0118] [Details of Image Processing (Figure 17)] Details of the image processing will be explained based on Figure 17, which shows the transition of images generated by image processing of the OCTA original image data or OCTA denoising image data of the outer retinal slab.

[0119] The OCTA original image 315 or OCTA denoising image 316 of the outer retinal slab transitions to a superpixel image 325 via superpixel segmentation processing (C). Note that Figure 17 omits the blood vessel extraction processing (A) and blood vessel enhancement processing (B), which are image processing steps similar to those in Embodiment 1. In other words, the superpixel image 325 is obtained by applying superpixel segmentation processing (C) to the image data of the blood vessel enhancement image 318.

[0120] In the vascular-enhanced image 318 of the outer retinal slab, the vascular structures appear brighter, while non-perfusion areas appear darker. Therefore, the superpixel segmentation process (C) groups pixels with similar brightness intensity characteristics, dividing the image data of the vascular-enhanced image of the retinal surface slab into thousands of small pixel groups. The superpixel image 325 after the superpixel segmentation process (C), as shown in Figure 17, is depicted as whitish lines in the grayscale image, but in the actual color image, it appears as a raised pattern drawn as yellow lines against a dark background. These yellow lines in the superpixel image 325 are boundaries indicating how the image was divided by the superpixel segmentation process (C).

[0121] The superpixel image 325 transitions to the first feature extraction map 326 via the mean / median calculation process (D). In other words, the image of the first feature extraction map 326 of the outer retinal slab is obtained by applying the mean / median calculation process (D) to each segmented image data of the superpixel image 325, in the same manner as in Embodiment 1.

[0122] The image of the first feature extraction map 326 transitions to the image of the second feature extraction map 327 via the first intensity value calculation process (E). In other words, the image of the second feature extraction map 327 of the outer retinal slab is obtained by applying the first intensity value calculation process (E) to each segmented image data of the first feature extraction map 326, in the same manner as in Embodiment 1.

[0123] [Detection of eye diseases] The outer retinal slab is located in the outer part of the retina 201, adjacent to the choroid 202. Unlike the superficial retinal slab, which is distributed with blood vessels, it is basically a layer without blood vessels, and is lined with photoreceptor cells that sense light. For this reason, the OCTA original image 315 of the outer retinal slab will basically not show blood vessels in a normal eye.

[0124] In contrast, the OCTA original image 315 of the outer retinal slab in Embodiment 2, as shown in Figure 17, captures blood vessels in a portion of the image, and a first feature extraction map 326 is generated by image processing based on this OCTA original image 315. The first feature extraction map 326 is generated by the mean / median calculation process (D) for each image data of the superpixel image 325, and can display choroidal neovascularization as shown in the upper right image of Figure 17. Furthermore, the second feature extraction map 327 is generated by the first intensity value calculation process (E) for the image data of the first feature extraction map 326, and can display the presence of choroidal neovascularization as shown in the lower right image of Figure 17. For this reason, the information on the presence and location of blood vessels in the first feature extraction map 326 or the second feature extraction map 327 is useful for detecting an eye disease called choroidal neovascularization.

[0125] Choroidal neovascularization refers to a disease in which abnormal new blood vessels grow from the choroid 202, which is located outside the retina 201 and has abundant blood vessels. These abnormal new blood vessels are very fragile and prone to leaking blood and fluids, causing problems such as decreased vision and retinal edema.

[0126] [Effects of the Image Generation Method and Image Generation System] The image generation method and image generation system 100' of Embodiment 2 provide the following effects in addition to the effects of (1), (2), and (8) of Embodiment 1.

[0127] (9) The image data acquisition step (S21) acquires image data of the outer retinal slab from the OCTA image data. This image generation method acquires vascular image data in which blood vessels are depicted in the outer retinal slab when there is an eye disease such as choroidal neovascularization, by acquiring image data of the outer retinal slab from the OCTA image data.

[0128] (10) The vascular system image generation process includes a third image processing step (S25) which generates a first feature extraction map 326 of the outer retinal slab by performing an average / median calculation process (D) which calculates the average or median value of brightness intensity for each segmented image data of the superpixel image 325. This image generation method can generate an image of the first feature extraction map 326 of the outer retinal slab, which is useful for detecting eye diseases, by performing an average / median calculation process (D) of brightness intensity for each segmented image data of the superpixel image 325.

[0129] (11) The vascular system image generation process determines a first threshold for the first feature extraction map 326 that divides the brightness intensity region containing blood vessels with a diameter exceeding a predetermined diameter. The vascular system image generation process includes a fourth image processing step (S26) which generates a second feature extraction map 327 of the outer retinal slab by a first intensity value calculation process (E) that leaves image data equal to or greater than the first threshold from each segmented image data of the first feature extraction map 326. This image generation method can generate an image of the second feature extraction map 327 of the outer retinal slab, which has a more detailed vascular structure and can distinguish changes in both small and large blood vessels, by a first intensity value calculation process (E) of brightness intensity for the first feature extraction map 326.

[0130] The above description is based on the drawings of the image generation method and image generation system of Embodiments 1 and 2. However, the specific configuration of the image generation method and image generation system of this disclosure is not limited to Embodiments 1 and 2, and changes or additions to the design are permitted as long as they do not depart from the gist of the invention as described in each claim.

[0131] The algorithm disclosed herein can provide a comprehensive workflow for highlighting feature regions from OCTA images and dividing the image into small regions (superpixels) with similar features. The algorithm then performs image processing to calculate the brightness intensity of each small region (superpixel), thereby outputting a vascular map, a vascular density map, a non-blood-flow area detection map, and a feature extraction map (primarily for neovascularization detection). Overall, the image generation method and system disclosed herein have the potential to visually highlight feature regions such as blood vessels and effectively detect areas of eye disease.

[0132] Embodiments 1 and 2 illustrate examples of applying the algorithm of the present disclosure to image data of OCTA images. However, the algorithm of the present disclosure may be applicable not only to image data of OCTA images, but also to image data of OCT images, fundus images, and other types of imaging.

[0133] Embodiment 1 shows an example in which the image data acquisition process and image data acquisition unit acquire image data of an OCTA image of the retinal surface slab. Embodiment 2 shows an example in which the image data acquisition process and image data acquisition unit acquire image data of an OCTA image of the retinal outer layer slab. However, the image data of the OCTA image acquired by the image data acquisition process and image data acquisition unit is not limited to image data of the retinal surface slab or the retinal outer layer slab, but can also be applied to image data of other areas or layers, such as the retinal deep slab or choroidal capillary plate slab.

[0134] Embodiment 1 shows an example in which the vascular system image generation process and vascular system image generation processing unit generate a vascular density map 322, a vascular map 323, and a non-blood flow area detection map 324 as vascular system images. Embodiment 2 shows an example in which the vascular system image generation process and vascular system image generation processing unit generate a first feature extraction map 326 and a second feature extraction map 327 as vascular system images. However, the vascular system image generation process and vascular system image generation processing unit are not limited to generating these maps. The vascular system image generation process and vascular system image generation processing unit may generate an appropriate map that can accurately detect the type of eye disease to be detected by brightness intensity processing as a vascular system image. In short, the vascular system image generation process and vascular system image generation processing unit only need to generate a vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image.

[0135] The image generation method and image generation system of Embodiments 1 and 2 may be appropriately combined in part with other parts without contradiction. The image generation method and image generation system of Embodiments 1 and 2 may be described in part or in whole as follows. However, the content is not limited to the appendix described below. [1] An image generation method for generating a vascular system image of an eye under examination, comprising: an image data acquisition step of acquiring image data of the eye under examination; a first image processing step of extracting the curved structure of blood vessels contained in the image data and generating image data of a vascular-enhanced image in which the curved structure is emphasized; a second image processing step of classifying the image data of the vascular-enhanced image by the similarity of the brightness intensity of pixels, grouping pixels with common similarity and dividing them into superpixels to generate segmented image data of a superpixel image; and a vascular system image generation processing step of generating the vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image. [2] The image generation method described in [1], wherein the image data acquisition step is characterized in that multiple images of the same part of the fundus of the eye under examination are taken using an OCT device that can obtain tomographic data, and OCTA image data is obtained in which only the part of the multiple OCT images in which the image changes due to blood flow is depicted as blood flow information. [3] The image generation method described in [2], wherein the image data acquisition step is characterized in that image data of the retinal surface slab is obtained from the OCTA image data. [4] The image generation method described in [3], wherein the vascular system image generation processing step is characterized in that a third image processing step is performed to generate a vascular density map of the retinal surface slab by an average value / median calculation process which calculates the average value or median value of brightness intensity for each segmented image data of the superpixel image.[5] An image generation method according to [4], wherein the vascular system image generation processing step includes a fourth image processing step of determining a first threshold for the vascular density map to classify brightness intensity regions containing blood vessels with a diameter exceeding a predetermined diameter, and generating a vascular map of the retinal surface slab by a first intensity value calculation process that leaves image data equal to or greater than the first threshold from each segmented image data of the vascular density map. [6] An image generation method according to [4], wherein the vascular system image generation processing step includes a fifth image processing step of determining a second threshold for the vascular density map to classify brightness intensity regions containing capillaries, and generating a non-blood flow region detection map of the retinal surface slab by a second intensity value calculation process that leaves image data equal to or less than the second threshold from each segmented image data of the vascular density map. [7] An image generation method according to any one of [3] to [6], wherein the image generation method includes a noise reduction processing step of removing at least speckle noise and random noise from the OCTA image data of the retinal surface slab acquired in the image data acquisition step. [8] An image generation method according to [2], characterized in that the image data acquisition step acquires image data of the outer retinal slab from the OCTA image data. [9] An image generation method according to [8], characterized in that the vascular system image generation processing step includes a third image processing step of generating a first feature extraction map of the outer retinal slab by an average value / median calculation process that calculates the average value or median value of brightness intensity for each segmented image data of the superpixel image.

[10] An image generation method according to [9], characterized in that the vascular system image generation processing step includes a fourth image processing step of generating a second feature extraction map of the outer retinal slab by determining a first threshold for the first feature extraction map that divides a brightness intensity region containing blood vessels with a diameter exceeding a predetermined diameter, and leaving image data equal to or greater than the first threshold from each segmented image data of the first feature extraction map.

[11] An image generation system for generating a vascular system image of an eye under examination, comprising: an image data acquisition unit that acquires image data of the eye under examination; a first image processing unit that generates image data of a vascular-enhanced image by performing a vascular extraction process to extract the curved structure of blood vessels contained in the image data and a vascular enhancement process to enhance the extracted vascular structure; a second image processing unit that generates segmented image data of a superpixel image by performing a superpixel segmentation process to classify the image data of the vascular-enhanced image by the similarity of the brightness intensity of pixels and to group pixels with common similarity and divide them into superpixels; and a vascular system image generation processing unit that generates the vascular system image by performing a brightness intensity process on each segmented image data of the superpixel image. Cross-reference of related applications

[0136] This application claims priority based on Japanese Patent Application No. 2024-162278, filed with the Japan Patent Office on 19 September 2024, all of which disclosures are incorporated herein by reference in their entirety.

Claims

1. An image generation method for generating a vascular system image of an eye under examination, comprising: an image data acquisition step of acquiring image data of the eye under examination; a first image processing step of extracting the curved structure of blood vessels contained in the image data and generating image data of a vascular system-enhanced image in which the curved structure is emphasized; a second image processing step of classifying the image data of the vascular system-enhanced image by the similarity of the brightness intensity of pixels, grouping pixels with common similarity and dividing them into superpixels to generate segmented image data of a superpixel image; and a vascular system image generation processing step of generating the vascular system image by applying brightness intensity processing to each segmented image data of the superpixel image.

2. The image generation method according to claim 1, wherein the image data acquisition step is characterized in that multiple images of the same part of the fundus of the eye under examination are taken using an OCT device that can obtain tomographic data, and OCTA image data is obtained in which only the part of the multiple OCT images in which the image changes due to blood flow is depicted as blood flow information.

3. The image generation method according to claim 2, wherein the image data acquisition step is characterized in that image data of the retinal surface slab is acquired from the OCTA image data.

4. The image generation method according to claim 3, wherein the vascular system image generation processing step includes a third image processing step of generating a vascular density map of the retinal surface slab by an average / median calculation process that calculates the average or median value of brightness intensity for each segmented image data of the superpixel image.

5. The image generation method according to claim 4, wherein the vascular system image generation processing step includes a fourth image processing step of determining a first threshold for the vascular density map to divide a brightness intensity region containing blood vessels with a diameter exceeding a predetermined diameter, and generating a vascular map of the retinal surface slab by a first intensity value calculation process that leaves image data equal to or greater than the first threshold from each segmented image data of the vascular density map.

6. The image generation method according to claim 4, wherein the vascular system image generation processing step includes a fifth image processing step of determining a second threshold for classifying brightness intensity regions containing capillaries in the vascular density map, and generating a non-blood flow region detection map of the retinal surface slab by a second intensity value calculation process that leaves image data below the second threshold from each segmented image data of the vascular density map.

7. An image generation method according to claim 3, characterized in that it includes a noise reduction step to remove at least speckle noise and random noise from the OCTA image data of the retinal surface slab acquired in the image data acquisition step.

8. The image generation method according to claim 2, wherein the image data acquisition step is characterized in that image data of the outer retinal slab is acquired from the OCTA image data.

9. An image generation method according to claim 8, wherein the vascular system image generation processing step includes a third image processing step of generating a first feature extraction map of the outer retinal slab by an average / median calculation process that calculates the average or median value of brightness intensity for each segmented image data of the superpixel image.

10. An image generation method according to claim 9, wherein the vascular system image generation processing step includes a fourth image processing step of determining a first threshold for the first feature extraction map that divides a brightness intensity region containing blood vessels with a diameter exceeding a predetermined diameter, and generating a second feature extraction map of the outer retinal slab by a first intensity value calculation process that leaves image data equal to or greater than the first threshold from each segmented image data of the first feature extraction map.

11. An image generation system for generating a vascular system image of an eye under examination, comprising: an image data acquisition unit that acquires image data of the eye under examination; a first image processing unit that generates image data of a vascular-enhanced image by performing a vascular extraction process to extract the curved structure of blood vessels contained in the image data and a vascular enhancement process to enhance the extracted vascular structure; a second image processing unit that generates segmented image data of a superpixel image by performing a superpixel segmentation process to classify the image data of the vascular-enhanced image by the similarity of the brightness intensity of pixels and grouping pixels with common similarity into superpixels; and a vascular system image generation processing unit that generates the vascular system image by performing a brightness intensity process on each segmented image data of the superpixel image.

Images