Image processing apparatus, image processing method, program, and ophthalmic system

Abstract

To provide an image processing device that detects a position of vortex vein from a fundus image, and to provide a method for processing an image.SOLUTION: A method for processing an image includes the steps of: analyzing a choroid vascular structure from a fundus image; and detecting a position of vortex vein on the basis of the choroid vascular structure.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The technology of the present disclosure relates to an image processing apparatus, an image processing method, a program, and an ophthalmic system.

Background Art

[0002] Japanese Patent Application Laid-Open No. 8-71045 discloses a technique for displaying arteries and veins of the choroidal blood vessels in different colors.

Summary of the Invention

[0003] An image processing method according to a first aspect of the technology of the present disclosure includes a step of analyzing a choroidal blood vessel structure from a fundus image, and a step of detecting a vortex vein position based on the blood vessel structure.

[0004] An image processing apparatus according to a second aspect of the technology of the present disclosure has an image processing unit that analyzes a choroidal blood vessel structure from a fundus image and detects a vortex vein position based on the blood vessel structure.

[0005] A program according to a third aspect of the technology of the present disclosure causes a computer to execute the image processing method according to the first aspect.

[0006] An ophthalmic system according to a fourth aspect of the technology of the present disclosure includes a server having an image processing unit that analyzes a choroidal blood vessel structure from a fundus image and detects a vortex vein position based on the blood vessel structure, and a viewer that displays a vortex vein position superimposed fundus image in which a mark indicating the vortex vein position is superimposed on the fundus image.

[0007] An image processing method according to a fifth aspect of the technology of the present disclosure includes a step of generating a choroidal blood vessel image, and a step of analyzing the choroidal blood vessel image and detecting the position of a vortex vein.

[0008] The image processing apparatus according to the sixth aspect of the technology of the present disclosure includes an image processing unit that generates a choroidal vascular image, analyzes the choroidal vascular image, and detects the position of the vortex vein, a display control unit that generates a vortex vein position superimposed fundus image in which a mark indicating the position of the vortex vein is superimposed on the choroidal vascular image, and an output unit that outputs the vortex vein position superimposed fundus image.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14A

Figure 14B

Figure 15A

Figure 15B

Figure 16A

Figure 16B

Figure 16C

Figure 17

Figure 18

Figure 19

Figure 20

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following, for convenience of explanation, a scanning laser ophthalmoscope is referred to as "SLO".

[0011] Referring to FIG. 1, the configuration of the ophthalmic system 100 will be described. As shown in FIG. 1, the ophthalmic system 100 includes an ophthalmic device 110, an axial length measuring device 120, a management server device (hereinafter referred to as "management server") 140, and an image display device (hereinafter referred to as "image viewer") 150. The ophthalmic device 110 acquires fundus images. The axial length measuring device 120 measures the axial length of a patient's eye. The management server 140 stores a plurality of fundus images and axial lengths obtained by photographing the fundi of a plurality of patients by the ophthalmic device 110 in correspondence with the patient IDs. The image viewer 150 displays the fundus images acquired by the management server 140. The management server 140 is an example of the "server" of the technology of the present disclosure. The image viewer 150 is an example of the "viewer" of the technology of the present disclosure.

[0012] The ophthalmic device 110, the axial length measuring device 120, the management server 140, and the image viewer 150 are interconnected via a network 130. Note that other ophthalmic devices (examination devices such as OCT (Optical Coherence Tomography) measurement, visual field measurement, intraocular pressure measurement, etc.) and a diagnostic support device that performs image analysis using artificial intelligence may be connected to the ophthalmic device 110, the axial length measuring device 120, the management server 140, and the image viewer 150 via the network 130.

[0013] Next, referring to FIG. 2, the configuration of the ophthalmic device 110 will be described. As shown in FIG. 2, the ophthalmic device 110 includes a control unit 20, a display / operation unit 30, and an SLO unit 40, and photographs the posterior segment (fundus) of the eye to be examined 12. Further, it may include an OCT unit (not shown) that acquires OCT data of the fundus.

[0014] The control unit 20 includes a CPU 22, a memory 24, a communication interface (I / F) 26, etc. The display / operation unit 30 is a graphic user interface that displays the images obtained by photographing and receives various instructions including instructions for photographing, and includes an input / instruction device 34 such as a display 32 and a touch panel. The SLO unit 40 includes a light source 42 for G light (green light: wavelength 530 nm), a light source 44 for R light (red light: wavelength 650 nm), and a light source 4 6 for IR light (infrared light (near-infrared light): wavelength 800 nm). The light sources 42, 44, and 46 emit respective lights according to commands from the control unit 20.

[0015] The SLO unit 40 includes optical systems 50, 52, 54, and 56 that reflect or transmit light from the light sources 42, 44, and 46 and guide it into one optical path. The optical systems 50 and 56 are mirrors, and the optical systems 52 and 54 are beam splitters. The G light is reflected by the optical systems 50 and 54, the R light is transmitted through the optical systems 52 and 54, and the IR light is reflected by the optical systems 52 and 56 and is respectively guided into one optical path.

[0016] The SLO unit 40 includes a wide-angle optical system 80 that two-dimensionally scans the light from the light sources 42, 44, and 46 across the posterior eye part (fundus) of the eye to be examined 12. The SLO unit 40 includes a beam splitter 58 that reflects the G light and transmits light other than the G light among the light from the posterior eye part (fundus) of the eye to be examined 12. The SLO unit 40 includes a beam splitter 60 that reflects the R light and transmits light other than the R light among the light transmitted through the beam splitter 58. The SLO unit 40 includes a beam splitter 62 that reflects the IR light among the light transmitted through the beam splitter 60. The SLO unit 40 includes a G light detection element 72 that detects the G light reflected by the beam splitter 58, an R light detection element 74 that detects the R light reflected by the beam splitter 60, and an IR light detection element 76 that detects the IR light reflected by the beam splitter 62.

[0017] The wide-angle optical system 80 includes an X-direction scanning device 82 composed of a polygon mirror that scans light from light sources 42, 44, and 46 in the X direction, a Y-direction scanning device 84 composed of a galvanometer mirror that scans light in the Y direction, and a slit mirror and an elliptical mirror (not shown), and is provided with an optical system 86 that widens the scanned light. With the optical system 86, the field of view (FOV) of the fundus can be made larger than that of the conventional technology, and a wider fundus area than the conventional technology can be photographed. Specifically, a wide fundus area of about 120 degrees at the external light irradiation angle from the outside of the eye to be examined 12 (about 200 degrees at the internal light irradiation angle that can be substantially photographed by irradiating the fundus of the eye to be examined 12 with scanning light, with the center O of the eyeball of the eye to be examined 12 as the reference position) can be photographed. The optical system 86 may be configured using a plurality of lens groups instead of the slit mirror and the elliptical mirror. Each of the scanning devices of the X-direction scanning device 82 and the Y-direction scanning device 84 may use a two-dimensional scanner configured using a MEMS mirror.

[0018] When a system including a slit mirror and an elliptical mirror is used as the optical system 86, a configuration using a system with an elliptical mirror described in International Application PCT / JP2014 / 084619 or International Application PCT / JP2014 / 084630 may be adopted. The disclosures of International Application PCT / JP2014 / 084619 (International Publication WO2016 / 103484), which was filed internationally on December 26, 2014, and International Application PCT / JP2014 / 084630 (International Publication WO2016 / 103489), which was filed internationally on December 26, 2014, are each incorporated herein by reference in their entirety.

[0019] Note that when the ophthalmic device 110 is installed on a horizontal plane, the horizontal direction is the "X direction", the vertical direction with respect to the horizontal plane is the "Y direction", and the direction connecting the center of the pupil of the anterior eye part of the eye to be examined 12 and the center of the eyeball is the "Z direction". Therefore, the X direction, the Y direction, and the Z direction are perpendicular to each other.

[0020] The color fundus image is obtained by photographing the fundus of the eye 12 simultaneously with G light and R light. More specifically, the control unit 20 controls the light sources 42 and 44 to emit light simultaneously, and the G light and R light are scanned by the wide-angle optical system 80 across the fundus of the eye 12. Then, the G light reflected from the fundus of the eye 12 is detected by the G light detection element 72, and the image data of the second fundus image (G-color fundus image) is generated by the CPU 22 of the ophthalmic device 110. Similarly, the R light reflected from the fundus of the eye 12 is detected by the R light detection element 74, and the image data of the first fundus image (R-color fundus image) is generated by the CPU 22 of the ophthalmic device 110. Also, when the IR light is irradiated, the IR light reflected from the fundus of the eye 12 is detected by the IR light detection element 76, and the image data of the IR fundus image is generated by the CPU 22 of the ophthalmic device 110.

[0021] The CPU 22 of the ophthalmic device 110 mixes the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image) at a predetermined ratio and displays them on the display 32 as a color fundus image. Note that instead of the color fundus image, the first fundus image (R-color fundus image), the second fundus image (G-color fundus image), or the IR fundus image may be displayed. The image data of the first fundus image (R-color fundus image), the image data of the second fundus image (G-color fundus image), and the image data of the IR fundus image are sent from the ophthalmic device 110 to the management server 140 via the communication IF 166. Various fundus images are used for generating choroidal vascular images.

[0022] The axial length measuring device 120 in FIG. 1 has two modes: a first mode and a second mode for measuring the axial length, which is the length of the eye 12 in the axial direction (Z direction) of the eye to be examined. In the first mode, after guiding light from a light source (not shown) to the eye to be examined 12, the interference light between the reflected light from the fundus and the reflected light from the cornea is received, and the axial length is measured based on the interference signal indicating the received interference light. The second mode is a mode for measuring the axial length using ultrasonic waves (not shown). The axial length measuring device 120 transmits the axial length measured in the first mode or the second mode to the management server 140. The axial length may be measured in both the first mode and the second mode. In this case, the average of the axial lengths measured in both modes is transmitted to the management server 140 as the axial length. The axial length is stored as patient information in the management server 140 as one of the patient's data and is also used for fundus image analysis.

[0023] Next, referring to FIG. 3, the configuration of the management server 140 will be described. As shown in FIG. 3, the management server 140 includes a control unit 160 and a display / operation unit 170. The control unit 160 includes a computer including a CPU 162, a memory 164 which is a storage device, a communication interface (I / F) 166, and the like. Note that an image processing program is stored in the memory 164. The display / operation unit 170 is a graphic user interface for displaying images and receiving various instructions, and includes a display 172 and an input / instruction device 174 such as a touch panel.

[0024] The configuration of the image viewer 150 is the same as that of the management server 140, so the description thereof will be omitted.

[0025] Next, referring to FIG. 4, various functions realized by the CPU 162 of the management server 140 executing the image processing program will be described. The image processing program includes an image processing function, a display control function, and a processing function. By the CPU 162 executing the image processing program having these functions, the CPU 162 functions as an image processing unit 182, a display control unit 184, and a processing unit 186 as shown in FIG. 4.

[0026] Next, with reference to FIG. 5, the image processing by the management server 140 will be described in detail. By the CPU 162 of the management server 140 executing an image processing program, the image processing method shown in the flowchart of FIG. 5 is realized.

[0027] The image processing program is executed when the management server 140 generates a choroidal vascular image based on the image data of the fundus image captured by the ophthalmic device 110. The choroidal vascular image is generated as follows.

[0028] First, the information included in the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image) will be described. The information will be described.

[0029] The structure of the eye is such that the vitreous body is covered by a plurality of layers with different structures. The plurality of layers include, from the innermost to the outermost on the vitreous body side, the retina, the choroid, and the sclera. R light passes through the retina and reaches the choroid. Therefore, the first fundus image (R-color fundus image) includes information on the blood vessels present in the retina (retinal blood vessels) and information on the blood vessels present in the choroid (choroidal blood vessels). On the other hand, G light only reaches the retina. Therefore, the second fundus image (G-color fundus image) includes only information on the blood vessels present in the retina (retinal blood vessels).

[0030] The image processing unit 182 of the management server 140 extracts retinal blood vessels from the second fundus image (G-color fundus image) by performing black hat filter processing on the second fundus image (G-color fundus image). Next, the image processing unit 182 removes the retinal blood vessels from the first fundus image (R-color fundus image) by performing inpainting processing using the retinal blood vessels extracted from the second fundus image (G-color fundus image). That is, a process of filling the retinal blood vessel structure of the first fundus image (R-color fundus image) with the same value as the surrounding pixels is performed using the position information of the retinal blood vessels extracted from the second fundus image (G-color fundus image). Then, the image processing unit 182 performs adaptive histogram equalization processing (CLAHE, Contrast Limited Adaptive Histogram Equalization) on the image data of the first fundus image (R-color fundus image) from which the retinal blood vessels have been removed, thereby emphasizing the choroidal blood vessels in the first fundus image (R-color fundus image). Thereby, the choroidal blood vessel image shown in FIG. 8 is obtained. The generated choroidal blood vessel image is stored in the memory 164. Also, although the choroidal blood vessel image is generated from the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image), the image processing unit 182 may generate the choroidal blood vessel image using the first fundus image (R-color fundus image) or the IR fundus image taken with IR light. Regarding the method of generating the choroidal fundus image, the disclosure of Japanese Patent Application No. 2018-052246 filed on March 20, 2018 is incorporated herein by reference in its entirety.

[0031] When the image processing program starts, in step 202 of FIG. 5, the image processing unit 182 reads out the choroidal blood vessel image (see FIG. 8) from the memory 164.

[0032] In step 204, the image processing unit 182 detects vortex vein (hereinafter referred to as "VV") candidates in the choroidal blood vessel image. The details of the processing in step 204 will be described later. Here, the vortex vein VV is an outflow path of the blood flow flowing into the choroid, and there are 4 to 6 of them near the posterior pole of the equator of the eyeball.

[0033] In step 206, the image processing unit 182 performs VV identification processing (details will be described later) to calculate an identification probability indicating whether the VV candidate is a VV and set an identification flag (VV flag / non-VV flag) for the VV candidate. The details of the processing in step 206 will be described later. The details will be described later.

[0034] In step 208, the image processing unit 182 identifies the number of VVs and identifies a VV arrangement pattern (arrangement of multiple VVs). The VV arrangement pattern is information indicating the positions of multiple VV positions on the fundus. When there are four VVs, in the choroidal vascular image, the VVs often exist at the four corners as shown in FIG. 9. In FIG. 9, 246N1, 246N2, 246N3, and 246N4 indicate frames for identifying VV positions.

[0035] In step 210, the processing unit 186 stores data including the number of VVs, VV position information (coordinates indicating the VV positions in the choroidal vascular image, and the coordinates for each VV are stored), VV arrangement pattern, identification flag (VV flag / non-VV flag), and identification probability in the memory 1 64. These data are used for creating a display screen in the choroidal analysis mode described later. They are used.

[0036] Next, the details of the processing in step 204 will be described. As the processing in step 204, the VVs detected by the first VV candidate detection processing shown in FIG. 6 are used as VV candidates. FIG. 6 shows a flowchart of the first VV candidate detection processing program.

[0037] In step 224, the image processing unit 182 determines the blood vessel running direction of each pixel in the choroid blood vessel image. Specifically, the image processing unit 182 repeats the following process for all pixels. That is, the image processing unit 182 sets a region (cell) composed of a plurality of pixels around the pixel. Then, the gradient direction of the luminance at each pixel in the cell (indicated by an angle of 0 degrees or more and less than 180 degrees. Note that 0 degrees is defined as the direction of a straight line (horizontal line).) is calculated based on the luminance values of the pixels around the pixel to be calculated. This calculation of the gradient direction is performed for all pixels in the cell.

[0038] Next, in order to create a histogram with nine bins in which the gradient directions are 0 degrees, 20 degrees, 40 degrees, 60 degrees, 80 degrees, 100 degrees, 120 degrees, 140 degrees, and 160 degrees (the width of each bin is 20 degrees), the number of pixels in the cell in the gradient direction corresponding to each bin is counted. The width of one bin in the histogram corresponds to 20 degrees, and for the 0-degree bin, the number of pixels (count value) in the cell having a gradient direction of 0 degrees or more and less than 10 degrees and 170 degrees or more and less than 180 degrees is set. For the 20-degree bin, the number of pixels (count value) in the cell having a gradient direction of 10 degrees or more and less than 30 degrees is set. Similarly, the count values for the 40-degree, 60-degree, 80-degree, 100-degree, 120-degree, 140-degree, and 160-degree bins are also set. Since the number of bins in the histogram is 9, the blood vessel running direction of the pixel is defined as any one of nine types of directions. Note that by narrowing the width of the bin and increasing the number of bins, the resolution of the blood vessel running direction can be increased. The count value (vertical axis of the histogram) in each bin is normalized, and a histogram for the analysis point is created.

[0039] Next, the image processing unit 182 identifies the blood vessel running direction of the analysis points from the histogram. Specifically, it identifies the bin with the smallest count value (assumed to be 60 degrees), and identifies 60 degrees, which is the gradient direction of the identified bin, as the blood vessel running direction of the pixel. The reason why the gradient direction with the fewest counts is the blood vessel running direction is as follows. The luminance gradient is small in the blood vessel running direction, while the luminance gradient is large in other directions (for example, the luminance difference between blood vessels and other things is large). Therefore, when creating a histogram of the luminance gradients of each pixel, the count value of the bin corresponding to the blood vessel running direction becomes small. Similarly, a histogram is created for each pixel in the choroidal blood vessel image, and the blood vessel running direction of each pixel is calculated. The calculated blood vessel running direction of each pixel is stored in the memory 164. Note that the blood vessel running direction is an example of the "choroidal blood vessel structure" of the technology of the present disclosure.

[0040] In step 226, the image processing unit 182 sets the initial positions of a total of L virtual particles, with M in the vertical direction and N in the horizontal direction, at equal intervals on the choroidal blood vessel image. For example, M = 10 and N = 50, and a total of L = 500 initial positions are set.

[0041] In step 228, the image processing unit 182 acquires the blood vessel running direction of the first position (any of the L positions), moves the virtual particle a predetermined distance along the acquired blood vessel running direction, acquires the blood vessel running direction again at the moved position, and moves the virtual particle a predetermined distance along the acquired blood vessel running direction. This process of moving a predetermined distance along the blood vessel running direction is repeated the preset number of times of movement. The above processing is executed at all L positions. When the preset number of times of movement is performed for all L virtual particles, the points where a certain number or more of virtual particles are gathered are set as VV candidates. The VV candidate positions are stored in the memory 164 as the first VV candidates.

[0042] Instead of the first VV candidate detection process described with reference to FIG. 6, the second VV candidate detection process shown in FIG. 7 may be used. FIG. 7 shows a flowchart of the second VV candidate detection process program.

[0043] In step 234, the image processing unit 182 binarizes the choroidal blood vessel image at a predetermined threshold value to create a binarized image shown in FIG. 10. In step 236, the image processing unit 182 performs a thinning process on the binarized image to convert it into a line image with a width of 1 pixel shown in FIG. 11 and eliminate the thickness information.

[0044] In step 238, in the line image, as shown in FIG. 12, the image processing unit 182 identifies blood vessel intersection points where lines intersect, blood vessel branch points where lines branch, and blood vessel feature points having characteristic patterns. FIG. 12 is a white dot distribution diagram, and the blood vessel intersection points, blood vessel branch points, and blood vessel feature points are displayed as white dots. These white dots are set as VV candidate positions. Note that the blood vessel intersection points, blood vessel branch points, and blood vessel feature points are examples of the "choroidal blood vessel structure" of the technology of the present disclosure.

[0045] Next, with reference to FIG. 13, the VV discrimination process in step 206 of FIG. 5 will be described. The VV discrimination process is a process for confirming whether the VV candidate detected in step 204 of FIG. 5 is a VV. In step 252 of FIG. 13, the image processing unit 182 sets the identification number n for identifying each of the plurality of VV candidates to 1, and in step 254, the image processing unit 182 selects the VV candidate identified by the identification number n.

[0046] In step 256, for the VV candidate identified by the identification number n, the image processing unit 182 calculates the feature amount of the choroidal blood vessel image around the VV candidate position using Log-Polar transformation. Specifically, first, the image data of a predetermined region including the VV candidate n position is extracted from the choroidal blood vessel image. An image of a predetermined region centered on the pixel corresponding to the VV candidate position is extracted, and Log-Polar transformation is performed on the extracted image.

[0047] If the VV candidate n is the true VV, in the image of the predetermined region including the VV candidate position, the choroidal blood vessels run radially centering on the VV candidate position. That is, as shown in FIG. 14A, the blood vessels converge at a predetermined position (VV candidate position). When an image in which the blood vessels run radially like this is subjected to Log-Polar transformation, a single stripe pattern Z1 is formed as shown in FIG. 14B (the pixel values in the region of the stripe pattern indicate brighter values than other regions). The width of the appearance region of the stripe pattern (the width in the θ direction of the stripe pattern) is L1, and the position on the θ axis at the center of the stripe pattern is θ1. The characteristics of such a stripe pattern are defined as unimodality shown in FIG. 16A. On the other hand, if the VV candidate is not the true VV, as shown in FIG. 15A, the image of the predetermined region including the VV candidate position is an image with a plurality of diagonal lines, and the blood vessels do not converge. When an image composed of such diagonal lines is subjected to Log-Polar transformation, two stripe patterns Z2 and Z3 are formed as shown in FIG. 15B. The width of the appearance region of the stripe pattern Z2 (the width in the θ direction) is L2, and similarly, the width of the appearance region of the stripe pattern Z3 is L3 (L2 < L1, L3 < L1). The positions on the θ axis at the center positions of the stripe patterns are θ2 and θ3, respectively. The characteristics of such a stripe pattern are defined as multimodality shown in FIG. 16C. Note that there is also a case of bimodality shown in FIG. 16B. In this step 256, the Log-Polar transformed image is analyzed to calculate a feature amount indicating whether it is unimodal or multimodal. A process of integrating the pixel values in the R direction for each θ direction in FIG. 14A or FIG. 15A is performed, and as shown in FIGS. 16A to 16C, a graph is created with the horizontal axis being θ and the vertical axis being the integrated pixel value. A physical quantity representing the shape of this graph is used as the feature amount. The case where the number of peaks is 1 as in FIG. 16A is called unimodality, the case where the number of peaks is 2 as in FIG. 16B is called bimodality, and the case where the number of peaks is 3 or more as in FIG. 16C is called multimodality. For example, the number of peaks, the position on the θ axis corresponding to the peak of each peak, the width of each peak, etc. are used as the feature amounts.

[0048]

[0049]

[0050] In step 258, the image processing unit 182 calculates the discrimination probability of the VV candidate based on this feature amount. The discrimination probability of the VV candidate is determined by the above feature amount. For example, assuming that the number of peaks is n, the position on the θ axis corresponding to the peak of each peak is θ, and the width of each peak is W, the discrimination probability P of the VV candidate is represented by a function f(n, θ, W) of n, θ, and W. That is, P = f(n, θ, W) is the case.

[0051] As described above, an image in which choroidal blood vessels run radially from the center position as shown in FIG. 14A shows unimodality by Log-Polar transformation. Therefore, when the feature amount shows unimodality, the discrimination probability becomes high. In addition, an image composed of diagonal lines as shown in FIG. 15A shows multimodality by Log-Polar transformation, so the discrimination probability becomes low. Also, even with the same unimodality, if the shape of the peak is sharp (the width of the peak is narrow), the discrimination probability is higher than that of a gentle peak shape (the width of the peak is wide). Therefore, the discrimination probability can be obtained by analyzing the graph shapes of FIGS. 16A to 16C, which are the feature amounts.

[0052] In step 260, the image processing unit 182 determines whether the obtained discrimination probability is greater than a reference probability (a probability indicating a threshold value, for example, a probability of 50%).

[0053] When it is determined that the discrimination probability is greater than the reference probability, in step 262, the image processing unit 182 discriminates that the VV candidate discriminated by the discrimination number n is a VV, and assigns VV flag information indicating that it is a VV corresponding to n. When it is determined that the discrimination probability is less than the reference probability, in step 264, the image processing unit 182 discriminates that the VV candidate discriminated by the discrimination number n is not a VV, and assigns non-VV flag information indicating that it is not a VV corresponding to n.

[0054] In step 266, the image processing unit 182 determines whether the identification number n is equal to the total number N of VV candidates, thereby determining whether the above processing (steps 254 to 266) has been completed for all VV candidates. If it is determined that the above processing (steps 254 to 266) has not been completed for all VV candidates, in step 268, the image processing unit 182 increments the identification number n by 1. Thereafter, the VV identification process returns to step 254. If it is determined that the above processing (steps 254 to 266) has been completed for all VV candidates, the VV identification process ends.

[0055] (Choroidal Vessel Analysis Mode by Image Viewer 150) Next, the data of the display screen in the choroidal vessel analysis mode will be described. The management server 140 has content data (image data and various data) to be displayed on the following choroidal vessel analysis mode screen.

[0056] First, as described above, image data of fundus images (the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image)) is transmitted from the ophthalmic device 110 to the management server 140, and the management server 140 has image data of fundus images (the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image)). The management server 140 has image data of a choroidal vessel image (see FIG. 8), image data of an image in which the positions of the VVs verified as VVs are superimposed and displayed on the fundus image (see FIG. 10), the positions of the VVs, the number of VVs, and data of the VV arrangement pattern.

[0057] Also, when the fundus of the patient is photographed, personal information of the patient is input to the ophthalmic device 110. The personal information includes the patient's ID, name, age, and visual acuity, etc. Also, when the fundus of the patient is photographed, information indicating whether the eye for which the fundus is photographed is the right eye or the left eye is also input. Furthermore, when the fundus of the patient is photographed, the photographing date and time are also input. Data of personal information, right eye / left eye information, and photographing date and time are transmitted from the ophthalmic device 110 to the management server 140. The management server 140 has data of personal information, right eye / left eye information, and photographing date and time.

[0058] Furthermore, the axial length of the patient is measured by the axial length measuring device 120, and data on the axial length of the patient corresponding to the patient ID is also transmitted from the axial length measuring device 120 to the management server 140, and the management server 140 has the data on the axial length.

[0059] As described above, the management server 140 has content data for display on the above choroidal vascular analysis mode screen.

[0060] When an ophthalmologist diagnoses a patient, the diagnosis is made while viewing the display screen in the choroidal vascular analysis mode displayed on the image viewer 150. In that case, the ophthalmologist transmits a display request for the choroidal vascular analysis mode screen to the management server 140 through a menu screen (not shown) via the image viewer 150. The display control unit 184 of the management server 140 that has received the request creates a display screen in the choroidal vascular analysis mode using the content data of the specified patient ID on the image viewer 150, and the processing unit 186 transmits the image data of the display screen. Note that the processing unit 186 is an example of the "output unit" of the technology of the present disclosure. The image viewer 150 that has received the image data of the display screen in the choroidal vascular analysis mode displays the display screen 300 in the choroidal vascular analysis mode shown in FIG. 17 on the display of the image viewer 150.

[0061] Here, the display screen 300 in the choroidal vascular analysis mode shown in FIG. 17 will be described. As shown in FIG. 17, the display screen 300 in the choroidal vascular analysis mode has a personal information display column 302 for displaying the personal information of the patient, an image display column 320, and a choroidal analysis tool display column 330.

[0062] The personal information display column 302 has a patient ID display column 304, a patient name display column 306, an age display column 308, an axial length display column 310, a visual acuity display column 312, and a patient selection icon 314. Each piece of information is displayed in the patient ID display column 304, the patient name display column 306, the age display column 308, the axial length display column 310, and the visual acuity display column 312. When the patient selection icon 314 is clicked, a list of patients is displayed on the display 172 of the image viewer 150, allowing the user (such as an ophthalmologist) to select the patient to be analyzed.

[0063] The image display column 320 has a shooting date display column 322N1, a right-eye information display column 324R, a left-eye information display column 324L, an RG image display column 326, a choroidal vascular image display column 328, and an information display column 342. The RG image is an image obtained by synthesizing a first fundus image (R-color fundus image) and a second fundus image (G-color fundus image) at a predetermined ratio (for example, 1:1) of the magnitudes of the respective pixel values.

[0064] The choroidal analysis tool display column 330 has a plurality of choroidal analysis tools for instructing the image viewer 150 to perform processing, for example, a vortex vein position icon 332, a symmetry icon 334, a vessel diameter icon 336, a vortex vein / macula / optic disc icon 338, and a choroidal analysis report icon 340. The vortex vein position icon 332 instructs to identify the position of the vortex vein. The symmetry icon 334 instructs to analyze the symmetry of the vortex veins. The vessel diameter icon 336 instructs to execute a tool for analyzing the diameter of the choroidal blood vessels. The vortex vein / macula / optic disc icon 338 instructs to analyze the position among the vortex veins, the macula, and the optic nerve head. The choroidal analysis report icon 340 instructs to display the choroidal analysis report.

[0065] On the display screen of the image viewer 150, which will be described later, icons and buttons for instructing the generation of the images, which will also be described later, are displayed. When a user (such as an ophthalmologist) of the viewer 150 clicks on an icon or the like, an instruction signal corresponding to the clicked icon or the like is transmitted from the image viewer 150 to the management server 140. The management server 140 that has received the instruction signal from the image viewer 150 generates an image corresponding to the instruction signal and transmits the image data of the generated image to the image viewer 150. The image viewer 150 that has received the image data from the management server 140 displays the image on the display 172 based on the received image data. The generation process of the display screen in the management server 140 is performed by a display screen generation program operating on the CPU 162.

[0066] The example shown in FIG. 17 shows the RG image and the choroidal vascular image when the fundus of the right eye of the patient identified by the patient ID: 123456 (the icon of 324R is lit) was taken on January 1, 2016.

[0067] When the vortex vein position icon 332 in the choroidal analysis tool display column 330 in FIG. 17 is clicked, the display screen changes to a display screen that displays information about the vortex vein shown in FIG. 18. As shown in FIG. 18, based on the position of VV, the image viewer 150 displays squares (rectangles) 326S and 328S centered on the position of VV on the RG image in the RG image display column 326 and the choroidal vascular image in the choroidal vascular image display column 328. In FIG. 17, only the RG image or the choroidal vascular image may be displayed, and based on the position of VV, the square (rectangle) may be displayed on the displayed image. The choroidal vascular image on which the squares (rectangles) 326S and 328S are displayed in the choroidal vascular image display column 328 is an example of the "vortex vein position superimposed fundus image" of the technology of the present disclosure, and the squares (rectangles) 326S and 328S are examples of the "mark" of the technology of the present disclosure. In addition, based on the number of VVs and the VV position pattern, the image viewer 150 displays in the information display column 342 that the number of VVs is 3 and the VV arrangement pattern is type A. In the image display column 320, an identification probability display icon 346 and a magnification icon 348 are displayed. The VV arrangement pattern may be inferred considering not only the actually detected number and position of the vortex veins (VVs), but also the parts that are not photographed and hidden by the eyelids.

[0068] When the identification probability display icon 346 is clicked, the display screen is changed to a display screen in which the identification probability is reflected for each VV shown in FIG. 19. As shown in FIG. 19, the image viewer 150 displays in the □ (rectangle) 326S, 328S the identification probability that the VV candidate is a VV (for example, 95%, 80%, 50%). The image viewer 150 additionally displays in the information display column 342 that the number of VVs is 3 and the text "Calculate the identification probability based on the Log-Polar processing result".

[0069] On the display screen of FIG. 18, when the magnification icon 348 is clicked, a display screen in which the area including the VV is enlarged is displayed. As shown in FIG. 20, when the □ (rectangle) 328S is clicked, the image viewer 150 displays an enlarged image of the clicked part of the VV instead of in the RG image display column 326.

[0070] As described above, in the present embodiment, the position of the vortex vein (VV) is detected from the choroidal vascular image, and a mark indicating the vortex vein position is superimposed and displayed on the choroidal vascular image.

[0071] Also, by using an SLO unit with a wide-angle optical system, an ultra-wide-field UWF-SLO image in a range of 200 degrees or more from the center of the eyeball can be obtained. By using the UWF-SLO image, the vortex veins existing near the equator of the eyeball can be detected.

[0072] Next, various modifications of the technology of the present disclosure will be described. <First Modification> In the first VV candidate detection process shown in FIG. 6 and the second VV candidate detection process shown in FIG. 7 in the above embodiment, in order to reduce the computational load, instead of the entire choroidal blood vessel image, an area with a high probability of the presence of the vortex vein (VV) statistically may be set as the detection target area.

[0073] <Second Modification> The calculation of the feature amount (step 256 in FIG. 13) in the above embodiment may be determined by AI (Artificial Intelligence). A method such as deep learning may be used to identify a stripe pattern that is not explicit in the form of a hidden layer structure or the like.

[0074] <Third Modification> In the above embodiment, the management server 140 executes the image processing program shown in FIG. 5 in advance, but the technology of the present disclosure is not limited to this. For example, the following may be done. When the vortex vein position icon 332 shown in FIG. 17 is clicked, the image viewer 150 transmits an instruction for vortex vein position detection to the management server 140. In response to this, the management server 140 executes steps 202 and 204 of the image processing program in FIG. 5 to display the display screen in FIG. 18. Further, when the identification probability display icon 346 is clicked, the image viewer 150 transmits an instruction for calculating the identification probability to the management server 140. In response to this, the management server 140 executes steps 206 and 208 in FIG. 5 to display the display screen in FIG. 19.

[0075] <Fourth Modification> In the above embodiment, an example in which the ophthalmic device 110 acquires a fundus image with an internal light irradiation angle of about 200 degrees has been described. The technology of the present disclosure is not limited to this, and a fundus image taken by an ophthalmic device with an internal irradiation angle of 100 degrees or less may be used, or the technology of the present disclosure may be applied to a montage image obtained by synthesizing a plurality of fundus images. <Fifth Modification> In the above embodiment, the fundus image is captured by the ophthalmic apparatus 110 including the SLO imaging unit. However, a fundus image captured by a fundus camera capable of imaging choroidal blood vessels may also be used, or the technique of the present disclosure may be applied to an image obtained by OCT angiography.

[0076] <Sixth Modification Example> In the above embodiment, the management server 140 executes the image processing program. The technique of the present disclosure is not limited to this. For example, the ophthalmic apparatus 110 or the image viewer 150 may execute the image processing program.

[0077] <Seventh Modification Example> In the above embodiment, the ophthalmic system 100 including the ophthalmic apparatus 110, the axial length measuring device 120, the management server 140, and the image viewer 150 has been described as an example. However, the technique of the present disclosure is not limited to this. For example, as a first example, the axial length measuring device 120 may be omitted, and the ophthalmic apparatus 110 may further have the function of the axial length measuring device 120. Further, as a second example, the ophthalmic apparatus 110 may further have at least one of the functions of the management server 140 and the image viewer 150. For example, when the ophthalmic apparatus 110 has the function of the management server 140, the management server 140 can be omitted. In this case, the image processing program is executed by the ophthalmic apparatus 110 or the image viewer 150. Also, when the ophthalmic apparatus 110 has the function of the image viewer 150, the image viewer 150 can be omitted. As a third example, the management server 140 may be omitted, and the image viewer 150 may execute the function of the management server 140.

[0078] <Other Modification Examples> The data processing described in the above embodiment is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed within the scope not departing from the gist. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed within the scope not departing from the gist. In addition, in the above-described embodiment, a case where data processing is realized by a software configuration using a computer has been exemplified, but the technology of the present disclosure is not limited thereto. For example, instead of a software configuration using a computer, data processing may be executed only by a hardware configuration such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). Also, a part of the data processing may be executed by a software configuration and the remaining processing may be executed by a hardware configuration.

Claims

1. an analysis unit that analyzes the running direction of choroidal blood vessels in a fundus image; A detection unit that detects the predetermined position of the choroidal blood vessels running radially around the predetermined position as a vortex vein position. Image processing device.

2. an analysis unit that analyzes the running direction of choroidal blood vessels in a fundus image; A detection unit that detects a position where the plurality of radially running choroidal blood vessels converge as a vortex vein position. Image processing device.

3. The analysis unit is determining the running direction based on a luminance gradient of the fundus image; 3. The image processing device according to claim 1 or 2.

4. The analysis unit is determining the running direction by calculating a gradient direction of brightness of each pixel in the fundus image; 4. The image processing device according to claim 1.

5. The detection unit is extracting an image of a predetermined region centered on the position detected as the vortex vein position, and calculating a discrimination probability indicating whether the vortex vein position is a vortex vein or not based on a running direction of the choroidal blood vessels in the image of the predetermined region; The image processing device according to any one of claims 1 to 4.

6. The detection unit is extracting an image of a predetermined region centered on the position detected as the vortex vein position, and calculating the classification probability by analyzing whether or not the choroidal blood vessels in the image of the predetermined region run radially from the position; The image processing device according to claim 5 .

7. The detection unit is creating a vortex vein position superimposed fundus image by superimposing a mark indicating the vortex vein position on the fundus image; The image processing device according to any one of claims 1 to 6.

8. An output unit that outputs an image signal based on the fundus image on which the vortex vein position is superimposed. The image processing device according to claim 7.

9. The analysis unit analyzes a running direction of the choroidal blood vessels in the choroidal blood vessel image in which the choroidal blood vessels are enhanced. The image processing device according to any one of claims 1 to 8.

10. The detection unit identifies the number of detected vortex veins. The image processing device according to any one of claims 1 to 9.

11. Further comprising a choroidal blood vessel image generating unit that generates a choroidal blood vessel image from the fundus image. The image processing device according to any one of claims 1 to 10.

12. Analyzing the running direction of choroidal blood vessels in a fundus image; and detecting the predetermined position of the choroidal blood vessels radially running from the predetermined position as a vortex vein position. Image processing methods.

13. Analyzing the running direction of choroidal blood vessels in a fundus image; and detecting a position where the plurality of radially running choroidal blood vessels converge as a vortex vein position. Image processing methods.

14. The analyzing step includes: determining the running direction by calculating a gradient direction of brightness of each pixel in the fundus image; The image processing method according to claim 12 or 13.

15. The detecting step includes: extracting an image of a predetermined region centered on the position detected as the vortex vein position, and calculating a discrimination probability indicating whether the vortex vein position is a vortex vein or not based on a running direction of the choroidal blood vessels in the image of the predetermined region. The image processing method according to any one of claims 12 to 14.

16. A program for causing a computer to execute the image processing method according to any one of claims 12 to 15.

17. A server including an image processing unit that analyzes the running direction of choroidal blood vessels in a fundus image and detects a predetermined position of the choroidal blood vessels running radially from a center as a vortex vein position; a viewer for displaying a vortex vein position superimposed fundus image in which a mark indicating the vortex vein position is superimposed on the fundus image; An ophthalmology system equipped with

18. Further, an ophthalmologic apparatus for acquiring a fundus image of the subject; 20. The ophthalmic system of claim 17, comprising:

19. The server and the viewer are connected via a network. An ophthalmic system according to claim 17 or 18.

20. the server, the viewer, and the ophthalmic apparatus are connected via a network; 20. An ophthalmic system according to claim 18.

Images