Image processing method, program, image processing device, and ophthalmic system

Abstract

To enable understanding of choroidal asymmetry. [Solution] In choroidal vascular images, the direction of blood vessel course is determined for each of several pairs of analysis points that are symmetrical with respect to the line connecting the macula and the optic nerve head. The asymmetry of each pair of analysis points is analyzed from the direction of blood vessel course of each analysis point, and the pair of analysis points with asymmetry is highlighted by enclosing them in a frame.

Description

[Technical Field]

[0001] The technologies disclosed herein relate to image processing methods, programs, image processing devices, and ophthalmic systems. [Background technology]

[0002] Japanese Patent Publication No. 2015-202236 discloses a technique for extracting vascular regions and measuring vascular diameter. Conventionally, there has been a need to analyze fundus images and measure vascular diameter. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2015-202236 [Overview of the project]

[0004] An image processing method according to a first aspect of the technology of this disclosure includes the steps of: setting a first analysis point and a second analysis point symmetrical with respect to a reference line in a fundus image; determining a first blood vessel direction at the first analysis point and a second blood vessel direction at the second analysis point; and analyzing the asymmetry between the first blood vessel direction and the second blood vessel direction.

[0005] An image processing method according to a second aspect of the technology of the present disclosure includes the steps of: setting a plurality of first analysis points in a first region and setting a plurality of second analysis points in a second region in a fundus image; determining a first blood vessel direction for each of the plurality of first analysis points and a second blood vessel direction for each of the plurality of second analysis points; defining a plurality of combinations of first and second analysis points that are symmetrical with respect to the plurality of first and second analysis points, and determining a symmetry index that shows the symmetry between the first blood vessel direction and the second blood vessel direction for each of the plurality of defined combinations.

[0006] A program of a third aspect of the technology of this disclosure causes a computer to execute an image processing method of the first or second aspect.

[0007] An image processing apparatus according to a fourth aspect of the technology of this disclosure comprises a storage device for storing a program for causing the processing apparatus to execute an image processing method, and a processing apparatus that executes the image processing method by executing the program stored in the storage device, wherein the image processing method is an image processing method according to the first or second aspect.

[0008] A fifth aspect of the technology of this disclosure is an ophthalmic system comprising an image processing device of the fourth aspect and an ophthalmic device for capturing the fundus image. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram of the ophthalmology system 100. [Figure 2] This is a schematic diagram showing the overall configuration of the ophthalmic device 110. [Figure 3] This is a block diagram of the electrical system configuration of management server 140. [Figure 4] This is a block diagram of the functions of CPU 162 on management server 140. [Figure 5] This is a flowchart of an image processing program. [Figure 6] Figure 5 is a flowchart of the blood vessel direction analysis processing program in step 210. [Figure 7] Figure 5 is a flowchart of the analysis program for the symmetry of the blood vessel direction in step 212. [Figure 8A] This is a diagram showing choroidal vascular images. [Figure 8B] This figure shows multiple analysis points set on a choroidal vascular image. [Figure 9] This figure shows histograms of the gradient direction of analysis points 242 and 246, which are arranged symmetrically with respect to the line LIN, in a choroidal vascular image rotated so that the line LIN connecting the macula M and the optic nerve head is horizontal. [Figure 10] It is a diagram showing the positional relationship between the straight line LIN, each analysis point, and each histogram. [Figure 11] It is a diagram showing the display screen 300 in the choroidal vascular analysis mode. [Figure 12] It is a display screen that appears when the symmetry icon 334 is clicked on the display screen of FIG. 11. [Figure 13] It is a display screen that appears when the asymmetry histogram display icon 346 is clicked on the display screen of FIG. 12. [Figure 14] It is a display screen that appears when the asymmetry color display icon 348 is clicked on the display screen of FIG. 12.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. For convenience of explanation, a scanning laser ophthalmoscope will be referred to as "SLO" hereinafter.

[0011] Referring to FIG. 1, the configuration of the ophthalmic system 100 will be described. As shown in FIG. 1, the ophthalmic system 100 includes an ophthalmic device 110, an axial length measuring device 120, a management server device (hereinafter referred to as "management server") 140, and an image display device (hereinafter referred to as "image viewer") 150. The ophthalmic device 110 acquires fundus images. The axial length measuring device 120 measures the axial length of a patient's eye. The management server 140 stores a plurality of fundus images and axial lengths obtained by photographing the fundus of a plurality of patients by the ophthalmic device 110 in correspondence with the patient IDs.

[0012] The ophthalmic device 110, the axial length measuring device 120, the management server 140, and the image viewer 150 are interconnected via a network 130.

[0013] Note that a diagnostic support device that performs image analysis using other ophthalmic devices (examination devices such as OCT (Optical Coherence Tomography) measurement, visual field measurement, intraocular pressure measurement, etc.) and artificial intelligence may be connected to the ophthalmic device 110, the axial length measuring device 120, the management server 140, and the image viewer 150 via the network 130.

[0014] Next, the configuration of the ophthalmic device 110 will be described with reference to FIG. 2. As shown in FIG. 2, the ophthalmic device 110 includes a control unit 20, a display / operation unit 30, and an SLO unit 40, and photographs the posterior segment (fundus) of the test eye 12. Further, it may include an OCT unit (not shown) that acquires OCT data of the fundus.

[0015] The control unit 20 includes a CPU 22, a memory 24, a communication interface (I / F) 26, and the like. The display / operation unit 30 is a graphic user interface that displays the captured image and receives various instructions including an instruction for photographing, and includes an input / instruction device 34 such as a display 32 and a touch panel.

[0016] The SLO unit 40 includes a light source 42 for G light (green light: wavelength 530 nm), a light source 44 for R light (red light: wavelength 650 nm), and a light source 46 for IR light (infrared (near-infrared light): wavelength 800 nm). The light sources 42, 44, and 46 emit each light according to an instruction from the control unit 20. The SLO unit 40 includes optical systems 50, 52, 54, and 56 that reflect or transmit the light from the light sources 42, 44, and 46 and guide it to one optical path. The optical systems 50 and 56 are mirrors, and the optical systems 52 and 54 are beam splitters. The G light is reflected by the optical systems 50 and 54, the R light is transmitted through the optical systems 52 and 54, and the IR light is reflected by the optical systems 52 and 56 and guided to one optical path, respectively.

[0017] The SLO unit 40 includes a wide-angle optical system 80 that scans light from light sources 42, 44, and 46 in a two-dimensional manner across the posterior segment (fundus) of the eye under examination 12. The SLO unit 40 includes a beam splitter 58 that reflects green light and transmits other light from the light coming from the posterior segment (fundus) of the eye under examination 12. The SLO unit 40 includes a beam splitter 60 that reflects red light and transmits other light from the light transmitted through the beam splitter 58. The SLO unit 40 includes a beam splitter 62 that reflects infrared light from the light transmitted through the beam splitter 60. The SLO unit 40 includes a green light detection element 72 that detects green light reflected by the beam splitter 58, a red light detection element 74 that detects red light reflected by the beam splitter 60, and an infrared light detection element 76 that detects infrared light reflected by the beam splitter 62.

[0018] The wide-angle optical system 80 includes an X-direction scanning device 82 composed of polygon mirrors that scan light from light sources 42, 44, and 46 in the X direction, a Y-direction scanning device 84 composed of galvanometer mirrors that scan light in the Y direction, and an optical system 86 that widens the scanned light, including a slit mirror and an elliptical mirror (not shown). The optical system 86 makes the field of view (FOV) of the fundus an ultra-wide field at the periphery of the fundus, allowing for imaging of a wide area of ​​the fundus. Specifically, it is possible to image a wide area of ​​the fundus with an external light illumination angle of approximately 120 degrees from outside the eye 12 (approximately 200 degrees as the internal light illumination angle that is substantially captureable when the fundus of the eye 12 is illuminated by scanning light, with the center O of the eyeball of the eye 12 as the reference position). The optical system 86 may also be configured using multiple lens groups instead of the slit mirror and elliptical mirror. Each of the X-direction scanning device 82 and the Y-direction scanning device 84 may use a two-dimensional scanner configured with a MEMS mirror.

[0019] When using a system including a slit mirror and an elliptical mirror as the optical system 86, a configuration using an elliptical mirror as described in international applications PCT / JP2014 / 084619 and PCT / JP2014 / 084630 is also acceptable. The disclosures of international application PCT / JP2014 / 084619 (International Publication WO2016 / 103484), filed internationally on 26 December 2014, and international application PCT / JP2014 / 084630 (International Publication WO2016 / 103489), filed internationally on 26 December 2014, are incorporated herein by reference in their entirety.

[0020] When the ophthalmic device 110 is placed on a horizontal plane, the horizontal direction is defined as the "X direction," the direction perpendicular to the horizontal plane is defined as the "Y direction," and the direction connecting the center of the pupil of the anterior segment of the eye under examination 12 to the center of the eyeball is defined as the "Z direction." Therefore, the X, Y, and Z directions are perpendicular to each other.

[0021] A color fundus image is obtained by simultaneously capturing the fundus of the eye under examination 12 with green light and red light. More specifically, the control unit 20 controls the light sources 42 and 44 to emit light simultaneously, and the wide-angle optical system 80 scans the fundus of the eye under examination 12 with green light and red light. The green light reflected from the fundus of the eye under examination 12 is detected by the green light detection element 72, and image data of the second fundus image (green color fundus image) is generated by the CPU 22 of the ophthalmic device 110. Similarly, the red light reflected from the fundus of the eye under examination 12 is detected by the red light detection element 74, and image data of the first fundus image (red color fundus image) is generated by the CPU 22 of the ophthalmic device 110. Furthermore, if IR light is irradiated, the IR light reflected from the fundus of the eye under examination 12 is detected by the IR light detection element 76, and image data of the IR fundus image is generated by the CPU 22 of the ophthalmic device 110.

[0022] The structure of the eye consists of the vitreous humor surrounded by multiple layers with different structures. These layers, from the innermost to the outermost layer on the vitreous side, include the retina, choroid, and sclera. Red light passes through the retina and reaches the choroid. Therefore, the first fundus image (red color fundus image) contains information about the blood vessels present in the retina (retinal blood vessels) and the blood vessels present in the choroid (choroidal blood vessels). In contrast, green light only reaches the retina. Therefore, the second fundus image (green color fundus image) contains information about the blood vessels present in the retina (retinal blood vessels).

[0023] The CPU 22 of the ophthalmic device 110 mixes the first fundus image (red-color fundus image) and the second fundus image (green-color fundus image) in a predetermined ratio and displays the resulting color fundus image on the display 32. Alternatively, instead of a color fundus image, the first fundus image (red-color fundus image), the second fundus image (green-color fundus image), or an IR fundus image may be displayed.

[0024] Image data of the first fundus image (R-color fundus image), the second fundus image (G-color fundus image), and the IR fundus image are sent from the ophthalmic device 110 to the management server 140 via the communication IF26 and stored in the memory 164 described later.

[0025] Since the fundus of the eye 12 being examined is captured simultaneously with G-light and R-light in this manner, each position in the first fundus image (R-color fundus image) and the corresponding position in the second fundus image (G-color fundus image) are the same position in the fundus.

[0026] The axial length measuring device 120 in Figure 1 has two modes: a first mode and a second mode, for measuring the axial length of the eye 12 in the axial direction (Z direction). In the first mode, light from a light source (not shown) is guided to the eye 12, and the device receives the interference light of the reflected light from the fundus and the reflected light from the cornea. The axial length is measured based on the interference signal indicating the received interference light. The second mode is a mode that measures the axial length using ultrasound (not shown). The axial length measuring device 120 transmits the axial length measured by the first mode or the second mode to the management server 140. The axial length may be measured using both the first mode and the second mode; in this case, the average of the axial lengths measured in both modes is transmitted to the management server 140 as the axial length.

[0027] The axial length of the eye is stored in the management server 140 as patient information in memory 164, and is also used for fundus image analysis.

[0028] Next, the configuration of the management server 140 will be described with reference to Figure 3. As shown in Figure 3, the management server 140 comprises a control unit 160 and a display / operation unit 170. The control unit 160 comprises a computer including a CPU 162, a memory 164 which is a storage device, and a communication interface (I / F) 166, etc. The memory 164 stores an image processing program. The display / operation unit 170 is a graphic user interface that displays images and accepts various instructions, and comprises a display 172 and an input / instruction device 174 such as a touch panel. The management server 140 is an example of an "image processing device" of the technology of this disclosure.

[0029] The configuration of the image viewer 150 is the same as that of the management server 140, so its explanation will be omitted.

[0030] Next, referring to Figure 4, various functions realized by the CPU 162 of the management server 140 executing the image processing program will be explained. The image processing program includes image processing functions, display control functions, and processing functions. When the CPU 162 executes the image processing program having these functions, the CPU 162 functions as an image processing unit 182, a display control unit 184, and a processing unit 186, as shown in Figure 4.

[0031] Next, we will explain in detail the image processing performed by the management server 140 using Figure 5. The CPU 162 of the management server 140 executes the image processing program, thereby realizing the image processing shown in the flowchart of Figure 5.

[0032] The image processing program is executed when the management server 140 generates a choroidal vascular image based on the fundus image data captured by the ophthalmic device 110.

[0033] Choroidal vascular images are generated as follows: The image processing unit 182 of the management server 140 extracts retinal blood vessels from the second fundus image (G-color fundus image) by applying a black hat filter to the second fundus image (G-color fundus image). Next, the image processing unit 182 removes retinal blood vessels from the first fundus image (R-color fundus image) by inpainting using the retinal blood vessels extracted from the second fundus image (G-color fundus image). In other words, it uses the positional information of the retinal blood vessels extracted from the second fundus image (G-color fundus image) to fill in the retinal vascular structure of the first fundus image (R-color fundus image) with the same value as the surrounding pixels. Then, the image processing unit 182 emphasizes the choroidal blood vessels in the first fundus image (R-color fundus image) by applying Contrast Limited Adaptive Histogram Equalization to the image data of the first fundus image (R-color fundus image) from which the retinal blood vessels have been removed. This yields the choroidal vascular image shown in Figure 8A. The generated choroidal vascular image is stored in memory 164. The choroidal vascular image is an example of a "fundus image" of the technology of this disclosure. Furthermore, although a choroidal vascular image is generated from the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image), the image processing unit 182 may then generate a choroidal vascular image using the first fundus image (R-color fundus image) or an IR fundus image captured with IR light. The disclosure of a method for generating a choroidal fundus image in Japanese Patent Application No. 2018-052246, filed on March 20, 2018, is incorporated herein by reference in its entirety.

[0034] When the image processing program starts, in step 202 of Figure 5, the processing unit 186 reads the choroidal vascular image (see Figure 8A) and the G-color fundus image from memory 164. The G-color fundus image clearly captures the macula and optic nerve head, making it easier to distinguish the macula and optic nerve head in image processing compared to the choroidal vascular image. Therefore, the G-color fundus image is used for detecting the positions of the macula and optic nerve head, as described below.

[0035] In step 204, the image processing unit 182 detects the optic nerve head (ONH) (see also Figure 9) from the G-color fundus image. Since G-color (green) laser light is reflected by the retinal layer, it is preferable to use a G-color fundus image captured with G-color laser light in order to extract retinal structures. The image processing unit 182 detects the optic nerve head (ONH) as the region of a predetermined number of pixels with the largest pixel value in the G-color fundus image, since the optic nerve head (ONH) is the brightest region in the G-color fundus image. The central position of the region containing the brightest pixels is calculated as the coordinates where the optic nerve head (ONH) is located and stored in memory 164.

[0036] In step 206, the image processing unit 182 detects the macula M (see also Figure 9) from the G-color fundus image. Specifically, since the macula is a dark region in the choroidal vascular image, the image processing unit 182 detects the region of a predetermined number of pixels with the smallest pixel value in the read choroidal vascular image as the macula M. The central position of the region containing the darkest pixels is calculated as the coordinates where the macula M is located and stored in the memory 164.

[0037] In step 208, the image processing unit 182 reads the coordinates of the macula M and the optic nerve head ONH calculated from the G-color fundus image, as shown in Figures 8B and 9. The image processing unit 182 sets the read coordinates on the choroidal vascular image and sets a straight line LIN connecting the macula M and the optic nerve head ONH on the choroidal vascular image. Here, since the choroidal vascular image is generated from the G-color fundus image and the R-color fundus image, the coordinates of the macula M and the optic nerve head ONH detected in the G-color fundus image coincide with the positions of the macula M and the optic nerve head ONH in the choroidal vascular image. Then, the image processing unit 182 rotates the choroidal vascular image so that this straight line LIN is horizontal.

[0038] In step 210, the image processing unit 182 analyzes the direction of the choroidal vessels; in step 212, the image processing unit 182 analyzes the symmetry of the direction of the choroidal vessels; and in step 214, the image processing unit 182 saves the analysis results to the memory 164. Details of steps 210 and 212 will be described later.

[0039] Next, the analysis process of the blood vessel course in step 210 will be explained with reference to Figures 6, 8B, and 9. In step 222 of Figure 6, the image processing unit 182 sets the analysis points as follows.

[0040] As shown in Figure 8B, in the choroidal vascular image, a linear line LIN defines a first region 274 and a second region 272. Specifically, the first region is located above the linear line LIN, and the second region is located below the linear line LIN.

[0041] The image processing unit 182 arranges the analysis points 240KU in the first region 274 in a grid pattern with M (natural number) rows vertically and N (natural number) columns horizontally, at equal intervals. In Figure 8B, the number of analysis points in the first region 264 is M(3) × N(7) (=L:21). Since the choroidal vessel image is displayed according to conformal projection, the analysis points are located in a grid pattern. However, if the choroidal vessel image is displayed using a different projection method, the image processing unit 182 will arrange the analysis points in a pattern that matches that other projection method. The image processing unit 182 places the analysis point 240KD in the second region 272 at a position that is symmetrical to the analysis point 240KU placed in the first region 274 with respect to the line LIN.

[0042] Furthermore, the analysis points 240KU and 240KD only need to be located in positions that are symmetrical with respect to the line LIN between the first region 274 and the second region 272. Therefore, they are not limited to being located in an equally spaced grid pattern, and may not be equally spaced or in a grid pattern. The sizes of the first region 274 and the second region 272 can be varied according to the axial length of the eye. The number of L, M, and N can also be set to various values, not limited to the example above. Increasing the number will increase the resolution.

[0043] In step 224, the image processing unit 182 calculates the direction of the choroidal blood vessels at each analysis point. Specifically, the image processing unit 182 repeats the following process for each of the analysis points. That is, as shown in Figure 9, the image processing unit 182 sets a region (cell) 244 for the central pixel corresponding to the analysis point 242, which is composed of multiple surrounding pixels centered on that central pixel. In Figures 8B and 9, region 244 is shown upside down. This is to facilitate comparison with region 248, which includes the upper set of analysis points 246.

[0044] The image processing unit 182 then calculates the gradient direction of brightness for each pixel in cell 244 (indicated as an angle between 0 degrees and less than 180 degrees; 0 degrees is defined as the direction of the straight line LIN (horizontal line)) based on the brightness values ​​of the pixels surrounding the pixel to be calculated. This gradient direction calculation is performed for all pixels in cell 244.

[0045] Next, the image processing unit 182 creates a histogram 242H with nine bins (each bin width 20 degrees) whose gradient directions are 0, 20, 40, 60, 80, 100, 120, 140, and 160 degrees relative to the angle reference line. To do this, it counts the number of pixels in cell 244 corresponding to the gradient direction of each bin. The angle reference line is a straight line LIN. The width of one bin in the histogram corresponds to 20 degrees. The 0-degree bin is set with the number of pixels (count values) in cell 244 that have gradient directions of 0 to less than 10 degrees and 170 to less than 180 degrees. The 20-degree bin is set with the number of pixels (count values) in cell 244 that have gradient directions of 10 to less than 30 degrees. Similarly, the count values ​​for the 40, 60, 80, 100, 120, 140, and 160-degree bins are also set. Since the histogram 242 has 9 bins, the direction of the blood vessel course at analysis point 242 is defined by one of 9 different directions. Note that the resolution of the blood vessel course can be increased by narrowing the bin width and increasing the number of bins.

[0046] The count values ​​in each bin (vertical axis of histogram 242H) are normalized, and histogram 242H is created for the 242 analysis points shown in Figure 9.

[0047] Next, the image processing unit 182 identifies the direction of the blood vessel at the analysis point from the histogram 242H. Specifically, it identifies the angle with the smallest count value, which in the example shown in Figure 9 is 60 degrees, and identifies 60 degrees, which is the gradient direction of the identified bin, as the direction of the blood vessel at the analysis point 242. The reason why the gradient direction with the fewest counts is determined to be the direction of the blood vessel is as follows: The brightness gradient is small in the direction of the blood vessel, while the brightness gradient is large in other directions (for example, there is a large difference in brightness between blood vessels and non-blood vessels). Therefore, when a histogram of the brightness gradient of each pixel is created, the count value of the bin corresponding to the direction of the blood vessel will be small.

[0048] Similarly, cell 248 is set for analysis point 246 and histogram 246H is created. Among the bins in histogram 246H, the bin with the smallest count value, 160 degrees, is identified. Therefore, the direction of the blood vessel at analysis point 246 is determined to be 160 degrees. Histograms 242H and 246H are examples of the “first histogram” and “second histogram” of the technology of this disclosure.

[0049] By performing the above process for all analysis points in both the first and second regions, the direction of vessel course at each analysis point set in the choroidal vessel image is determined. That is, as shown in Figure 10, a histogram for each analysis point is obtained. In Figure 10, the histograms of the second region below the line LIN are displayed in a different order. This is because in Figure 10, the histogram corresponding to analysis point U1 is histogram U1H, and the histogram corresponding to its target analysis point D1 is histogram D1H. The order of the histograms in the first and second regions is consistent (the histograms of the second region are arranged in the same order as those of the first region).

[0050] In step 226, the image processing unit 182 saves the following data: the position of the macula M, the position of the optic nerve head ONH, the rotation angle obtained by rotating the choroidal vascular image so that the line LIN is horizontal, the position (XY coordinates) of each of the analysis points (L), combination information of analysis points that are symmetric with respect to the line LIN (combination of numbers of the first and second region analysis points), the direction of vascular course of each analysis point, and the histogram of each analysis point, and saves these in memory 164.

[0051] Next, referring to Figure 7, the analysis process for the symmetry of the blood vessel course in step 212 of Figure 5 will be explained. In step 232 of Figure 7, the image processing unit 182 reads out each analysis point in the upper and lower (first and second regions) and the blood vessel course of that point. Specifically, for each pair of analysis points that are symmetrical with respect to the line LIN, the image processing unit 182 reads out each analysis point and the blood vessel course of that point.

[0052] In step 234, the image processing unit 182 calculates a value indicating asymmetry for each pair of analysis points that are symmetrical with respect to the straight line LIN. The value indicating asymmetry is the difference in the direction of blood vessel course, and this difference is obtained from the histogram of each analysis point in each pair. The difference in frequency Δh in each bin of the histogram of the pair is found, and Δh is squared. Then Δh of each bin 2 The sum of ΣΔh 2 It is obtained by calculating ΣΔh. 2 If the value is large, the histogram shapes will be significantly different, resulting in greater asymmetry. Conversely, if the value is small, the histogram shapes will be similar, resulting in less asymmetry. The histograms of each analysis point in each set are examples of the "first vessel direction" and "second vessel direction" of the technology disclosed herein. Furthermore, the value indicating asymmetry is not limited to the sum of the squared errors of the histograms of each analysis point in each set. Alternatively, a representative angle may be determined from the histogram of each analysis point in each set, and the absolute difference between them may be calculated.

[0053] In step 236, the image processing unit 182 detects asymmetric analysis point pairs. Specifically, the image processing unit 182 detects a pair as an asymmetric analysis point if the value indicating the asymmetry of each pair is greater than or equal to a threshold. The threshold is a predetermined constant value, but it may also be the overall average value of the values ​​indicating the asymmetry of each pair. Figure 10 shows the analysis results from step 236. Analysis point U1 in the first region (upper region 274) and analysis point D1 in the second region (lower region 272) are symmetrical and form a pair, and similarly analysis point U11 and analysis point D11 are symmetrical and form a pair. As a result of the analysis in step 234, these pairs are determined to have a value indicating asymmetry that is above a threshold and are identified as pairs with asymmetrical analysis points. Arrow UA1 indicates the direction of blood vessel course of analysis point U1 and points in the direction of 160 degrees. Similarly, arrow DA1 indicates the direction of blood vessel course of analysis point D1 and points in the direction of 60 degrees. Arrow UA11 indicates the direction of blood vessel course of analysis point U11 and points in the direction of 160 degrees. Similarly, arrow DA11 indicates the direction of blood vessel course of analysis point D11 and points in the direction of 40 degrees.

[0054] In step 238, the image processing unit 182 stores the following data in memory 164: the asymmetry value for each set, a flag indicating whether the asymmetry value is above a threshold (i.e., whether it is asymmetrical or not), and the angle of the blood vessel direction at the analysis point for each set, all stored in memory 164.

[0055] Next, the display screen for the choroidal vascular analysis mode will be described. The memory 164 of the management server 140 contains data for creating the following choroidal vascular analysis mode display screen, or content data to be displayed on said display screen.

[0056] Specifically, the data is as follows: Image data of fundus images (first fundus image (R-color fundus image) and second fundus image (G-color fundus image)) is transmitted from the ophthalmic device 110 to the management server 140, and the management server 140 has the image data of fundus images (first fundus image (R-color fundus image) and second fundus image (G-color fundus image)). The management server 140 also has image data of choroidal vascular images (see Figure 8A). The management server 140 has the position of the macula M, the position of the optic nerve head ONH, the rotation angle obtained by rotating the choroidal vascular image so that the line LIN is horizontal, the position of each analysis point (L points), the set of analysis points that are symmetric with respect to the line LIN, and the histogram and angle indicating the direction of travel, which are feature quantities of each analysis point. The management server 140 has a value indicating the asymmetry of the set of analysis points and a flag indicating whether the value indicating asymmetry is above a threshold (whether it is asymmetric or not).

[0057] Furthermore, when a patient's fundus is photographed, the ophthalmic device 110 receives the patient's personal information. This personal information includes the patient's ID, name, age, and visual acuity. When the fundus is photographed, information indicating whether the eye being photographed is the right or left eye is also entered. Additionally, the date and time of the photograph are entered. The ophthalmic device 110 transmits the personal information, right / left eye information, and the date and time of the photograph to the management server 140. The management server 140 holds the personal information, right / left eye information, and the date and time of the photograph. The management server 140 also holds the axial length data.

[0058] As described above, the management server 140 has the data necessary to create the display screen for the choroidal vascular analysis mode described above.

[0059] When an ophthalmologist diagnoses a patient, they perform the diagnosis while viewing the choroidal vascular analysis mode display screen on the image viewer 150. In this case, the ophthalmologist sends a request to display the choroidal vascular analysis mode screen to the management server 140 via the image viewer 150 through a menu screen (not shown). Upon receiving this request, the display control unit 184 of the management server 140 creates the choroidal vascular analysis mode display screen using the content data of the specified patient ID, and the processing unit 186 sends the image data of the display screen to the image viewer 150. The processing unit 186 is an example of an "output unit" in the technology of this disclosure. The image viewer 150, having received data from the choroidal vascular analysis mode display screen, displays the choroidal vascular analysis mode display screen 300 shown in Figure 11 on the display 172 based on the data from the choroidal vascular analysis mode display screen.

[0060] Here, we will explain the display screen 300 of the choroidal vascular analysis mode shown in Figure 11. As shown in Figure 11, the display screen 300 of the choroidal vascular analysis mode has a personal information display area 302 for displaying the patient's personal information, an image display area 320, and a choroidal analysis tool display area 330.

[0061] The personal information display area 302 includes a patient ID display area 304, a patient name display area 306, an age display area 308, an axial length display area 310, a visual acuity display area 312, and a patient selection icon 314. The patient ID display area 304, patient name display area 306, age display area 308, axial length display area 310, and visual acuity display area 312 display the respective information. When the patient selection icon 314 is clicked, a list of patients is displayed on the display 172 of the image viewer 150, and the user (such as an ophthalmologist) is allowed to select the patient to be analyzed.

[0062] The image display area 320 includes the date of capture display areas 322N1 to 322N3, the right eye information display area 324R, the left eye information display area 324L, the RG image display area 326, the choroidal vascular image display area 328, and the information display area 342. The RG image is obtained by combining the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image) with the size of each pixel value in a predetermined ratio (for example, 1:1).

[0063] The choroidal analysis tool display area 330 is a field where icons for selecting multiple choroidal analyses are displayed. It includes vortex vein position icons 332, symmetry icons 334, vessel diameter icons 336, vortex vein / macular / papillary icons 338, and choroidal analysis report icons 340. The vortex vein location icon 332 indicates that the vortex vein locations should be displayed. The symmetry icon 334 indicates that the symmetry of the analysis points should be displayed. The vessel diameter icon 336 indicates that the analysis results regarding the diameter of the choroidal vessels should be displayed. The vortex vein / macular / optic disc icon 338 indicates that the analysis results regarding the positions between the vortex veins, macula, and optic disc should be displayed. The choroidal analysis report icon 340 indicates that the choroidal analysis report should be displayed.

[0064] The display screen of the image viewer 150, as described later, displays icons and buttons for instructing the generation of images, as described later. When a user of the viewer 150 (such as an ophthalmologist) clicks an icon, etc., the image viewer 150 sends an instruction signal corresponding to the clicked icon, etc. to the management server 140. Upon receiving the instruction signal from the image viewer 150, the management server 140 generates an image corresponding to the instruction signal and sends the image data of the generated image to the image viewer 150. Upon receiving the image data from the management server 140, the image viewer 150 displays the image on the display 172 based on the received image data. The display screen generation process on the management server 140 is performed by a display screen generation program running on the CPU 162.

[0065] Figure 11 shows the screen displaying the RG image and choroidal vascular image of the right eye fundus of the patient identified by patient ID: 123456 (the 324R icon is lit) when the date of acquisition (322N1) is clicked, and the acquisition date was January 1, 2016.

[0066] When the symmetry icon 334 in the choroidal analysis tool display area 330 of Figure 11 is clicked, the display screen changes to show the analysis points shown in Figure 12. As shown in Figure 12, the image viewer 150 displays each analysis point of each set as a dot on the choroidal vessel image displayed in the choroidal vessel image display area 328. Note that the image viewer 150 is not limited to displaying each analysis point of each set as a dot, but may also display arrows UA1, UA11, DA1, DA11 (see Figure 10) that indicate the direction of vessel course, visualizing the characteristics of each analysis point, or display marks such as ellipses in place of or along with the arrows. The image display area 320 of the display screen in Figure 12 is provided with an asymmetry histogram display icon 346 and an asymmetry color display icon 348.

[0067] When the asymmetry histogram display icon 346 in the image display area 320 of the display screen in Figure 12 is clicked, a screen showing the asymmetry is displayed. Specifically, as shown in Figure 13, the image viewer 150 displays a value (ΣΔh) indicating the asymmetry for each analysis point of each set shown in the choroidal vessel image in the choroidal vessel image display area 328. 2The set of analysis points U1, D1, U11, and D11 whose values ​​are greater than or equal to a predetermined value are highlighted, for example, by adding a frame. The frames are the same color for the same set, but different colors for other sets. For example, the frames surrounding analysis point U1 and analysis point D1 are both the first color (e.g., red), and the frames surrounding analysis point U11 and analysis point D11 are both the second color (e.g., orange). Furthermore, the image viewer 150 displays arrows UA1, DA1, UA11, and DA11 on the choroidal vessel image, along the angles of 60 degrees, 160 degrees, 50 degrees, and 150 degrees in the direction of vessel course for each of the analysis points U1, D1, U11, and D11 in the above set. Arrows UA1, UA11 and UDA1, DA11 are examples of the "first indicator" ("first arrow") and "second indicator" (second arrow) of the technology of this disclosure.

[0068] Furthermore, the image viewer 150 displays the histogram of each analysis point in the histogram display area 350 instead of the RG image display area 326, and displays a value (ΣΔh) indicating asymmetry. 2 The system highlights histogram pairs where the value is greater than or equal to a predetermined value. The information display area 342 displays the analysis point numbers for asymmetric pairs.

[0069] In Figure 13, analysis point U1 and analysis point D1 are defined as asymmetric pairs, and analysis point U11 and analysis point D11 are defined as asymmetric pairs. Therefore, in the choroidal vessel image display area 328, the image viewer 150 displays frames of the same color for pairs that are the same, such as the frames surrounding analysis point U1 and analysis point D1 being the first color (e.g., red), and the frames surrounding analysis point U11 and analysis point D11 being the second color (e.g., orange), while frames of different colors are used for pairs that are the same.

[0070] In the histogram display column 350, the image viewer 150 displays the frames enclosing the histogram U1H of the analysis point U1 and the histogram D1H of the analysis point D1 in the same first color (e.g., red), and the frames enclosing the histogram U11H of the analysis point U11 and the histogram D11H of the analysis point D11 in the same second color (e.g., orange). For the same group, frames of the same color are used, and frames of different colors are used for other groups. The color of the frame enclosing the analysis point in the choroidal vascular image display column 328 and the color of the frame enclosing the histogram in the histogram display column 350 are the same if the numbers of the analysis points are the same, which enhances visibility.

[0071] In the information display column 342, the image viewer 150 displays the text that the analysis points U1 and D1, and the analysis points U11 and D11 are an asymmetric pair, specifically, "U1 and D1 are asymmetric" and "U11 and D11 are asymmetric".

[0072] When the asymmetry color display icon 348 in the image display column 320 of the display screen in FIG. 12 is clicked, the display screen shown in FIG. 14 may be displayed instead of the display screen in FIG. 13. In each of the display screens in FIGS. 13 and 14, the histogram display column 350 is displayed in FIG. 13, and in FIG. 14, instead of the histogram display column 350, only the color map display column 360 color-coded according to the value indicating asymmetry (ΣΔh 2 ) is different. Hereinafter, only the color map display column 360 will be described.

[0073] First, in the memory 164 of the image viewer 150, colors are pre-associated and stored according to the magnitude of the value indicating asymmetry (ΣΔh 2 ). For example, the larger the magnitude of the value indicating asymmetry (ΣΔh 2 ), the darker the associated color. In addition, in the color map display column 360, rectangular regions are defined according to the number and position of each analysis point.

[0074] The image viewer 150 determines the magnitude of the value indicating asymmetry (ΣΔh 2 ) corresponding to each analysis point and the value indicating asymmetry (ΣΔh 2Based on the predetermined colors according to the size of each analysis point, a value (ΣΔh) indicating the asymmetry corresponding to each analysis point is applied to the rectangular region corresponding to each analysis point. 2 Display the color corresponding to the size of the element.

[0075] Furthermore, in the histogram display area 360, the image viewer 150 displays a frame of the same color but a different color from other sets of analysis points in the rectangular areas corresponding to asymmetrical sets of analysis points. For example, the rectangular areas RU1 and RD1 corresponding to analysis points U1 and D1 will display a frame of the first color (e.g., red), while the rectangular areas RU11 and RD11 corresponding to analysis points U11 and D11 will display a frame of the second color (e.g., orange).

[0076] As described above, in this embodiment, choroidal vascular images are analyzed, and the asymmetry of sets of analysis points that are symmetrical with respect to the straight line LIN connecting the macula M and the optic nerve head is analyzed, and the sets of analysis points exhibiting asymmetry are highlighted. Therefore, the asymmetry of the direction of course of choroidal blood vessels can be grasped. Furthermore, by visualizing the asymmetry of the direction of course of choroidal blood vessels, it is possible to support ophthalmologists in diagnosing the fundus. Furthermore, an SLO unit using a wide-angle optical system can obtain ultra-wide-angle UWF-SLO images covering a range of more than 200 degrees from the center of the eyeball. By using UWF-SLO images, it is possible to analyze the symmetry over a wide area, including the peripheral part of the fundus.

[0077] Next, various modifications of the technology of this disclosure will be described. <First variation> In the above embodiment, the choroidal vascular image is divided into a first region and a second region by a straight line LIN connecting the macula and the optic nerve head, and the analysis points are positioned in the first and second regions at positions symmetrical with respect to the straight line LIN. The technology of this disclosure is not limited thereto. For example, the choroidal vascular image may be divided into an temporal region and a nasal region by a line perpendicular to the line LIN (orthogonal line) with respect to the center of the macula and the optic nerve head. The analysis points may then be positioned at positions symmetrical with respect to the orthogonal line. Furthermore, the choroidal vascular image may be divided by a line intersecting the line LIN at a predetermined angle, for example, 45 degrees or 135 degrees (intersecting line) with respect to the center of the macula and the optic nerve head, and the analysis points may be positioned at positions symmetrical with respect to the intersecting line.

[0078] <Second variation> In the above embodiment, the vascular course direction of the choroidal vessels at each analysis point is created. The technology of this disclosure is not limited thereto. For example, the three-dimensional position of each pixel in the choroidal vessel image may be identified, and the vascular course direction of the choroidal vessels may be calculated in terms of direction in three-dimensional space. The three-dimensional position and direction in three-dimensional space are calculated using OCT volume data obtained using an OCT (Optical Coherence Tomography) unit provided in an ophthalmic device 110 (not shown).

[0079] <Third variation> In the above embodiment, the management server 140 has already executed the image processing program shown in Figure 5, but the technology of this disclosure is not limited thereto. When the symmetry icon 334 shown in Figure 11 is clicked, the image viewer 150 sends an image processing command to the management server 140. In response, the management server 140 may execute the image processing program shown in Figure 5.

[0080] <Fourth variation> In the above embodiment, an example was described in which a fundus image with an internal light illumination angle of approximately 200 degrees is acquired using an ophthalmic device 110. The technology of this disclosure is not limited to this, and the technology of this disclosure may also be applied to fundus images taken with an ophthalmic device with an internal illumination angle of 100 degrees or less, or to montage images obtained by combining multiple fundus images.

[0081] <Fifth variation> In the above embodiment, fundus images are captured using an ophthalmic device 110 equipped with an SLO imaging unit. However, fundus images obtained using a fundus camera capable of capturing choroidal blood vessels may also be used, or the technology of this disclosure may be applied to images obtained by OCT angiography.

[0082] <Sixth variation> In the above embodiment, the asymmetry is analyzed from the direction of course of the choroidal blood vessels, but the technology of this disclosure may also be applied to analyze the asymmetry from the direction of course of the retinal blood vessels.

[0083] <Seventh variation> In the above embodiment, the management server 140 executes the image processing program. The technology of this disclosure is not limited thereto. For example, the ophthalmic device 110 or the image viewer 150 may execute the image processing program.

[0084] <Variation 8> In the above embodiment, an ophthalmic system 100 comprising an ophthalmic device 110, an axial length measuring instrument 120, a management server 140, and an image viewer 150 was described as an example, but the technology of this disclosure is not limited thereto. For example, as a first example, the axial length measuring instrument 120 may be omitted, and the ophthalmic device 110 may further have the functions of the axial length measuring instrument 120. As a second example, the ophthalmic device 110 may further have the functions of at least one of the management server 140 and the image viewer 150. For example, if the ophthalmic device 110 has the functions of the management server 140, the management server 140 can be omitted. In this case, the image processing program is executed by either the ophthalmic device 110 or the image viewer 150. Also, if the ophthalmic device 110 has the functions of the image viewer 150, the image viewer 150 can be omitted. As a third example, the management server 140 may be omitted, and the image viewer 150 may perform the functions of the management server 140.

[0085] <Other variations> The data processing described in the above embodiment is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose. Furthermore, while the above embodiments illustrate cases where data processing is realized by a software configuration using a computer, the technology of this disclosure is not limited thereto. For example, instead of a software configuration using a computer, data processing may be performed solely by a hardware configuration such as an FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit). Alternatively, some of the data processing may be performed by a software configuration, and the remaining processing may be performed by a hardware configuration.

Claims

[Claim 1] The steps include setting a first analysis point and a second analysis point that are symmetrical with respect to the reference line in the fundus image, The steps include determining the first blood vessel direction at the first analysis point and the second blood vessel direction at the second analysis point, A step of analyzing the asymmetry between the first blood vessel direction and the second blood vessel direction, Image processing methods including [specific details omitted].

Images