Image processing method, program, and image processing apparatus

Abstract

To provide an image for supporting diagnosis of a fundus by an ophthalmologist.SOLUTION: Provided is an image processing method including: a step (208) of analyzing a choroidal blood vessel image and estimating the position of a vortex vein; a step (212) of calculating a feature value indicating the positional relationship between the position of the vortex vein and the position of a specific portion of the fundus oculi; and a step of superimposing and displaying the feature value on the fundus oculi image.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The technology of the present disclosure relates to an image processing method, a program, and an image processing device. [Background technology]

[0002] Japanese Patent Laid-Open Publication No. 7-136122 discloses a technique for detecting the positions of the optic disc and the macula from a fundus image. There has been a demand for analyzing fundus diseases using fundus images. Summary of the Invention

[0003] The image processing method according to the first aspect of the disclosed technique includes the steps of analyzing a choroidal blood vessel image to estimate the position of a vortex vein, and calculating a feature quantity indicating the positional relationship between the position of the vortex vein and the position of a specific part of the fundus.

[0004] A program according to a second aspect of the technique of the present disclosure causes a computer to execute the image processing method according to the first aspect.

[0005] An image processing device of a third aspect of the technology disclosed herein is an ophthalmic device comprising a storage device that stores a program for causing a processing device to execute an image processing method, and a processing device that executes the image processing method by executing the program stored in the storage device, wherein the image processing method is the image processing method of the first aspect.

[0006] An image processing device according to a fourth aspect of the disclosed technology is an image processing device that includes a processing device that executes an image processing method, and the processing device executes the steps of analyzing a choroidal vascular image to estimate a vortex vein position, and calculating a feature value that indicates the positional relationship between the vortex vein position and the position of a specific part of the fundus. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram of an ophthalmology system 100. [Figure 2]1 is a schematic diagram showing the overall configuration of an ophthalmologic apparatus 110. FIG. [Figure 3] FIG. 2 is a block diagram of the electrical configuration of the management server 140. [Figure 4] FIG. 2 is a functional block diagram of a CPU 162 of a management server 140. [Figure 5] 10 is a flowchart of an image processing program. [Figure 6A] 6 is a flowchart of a VV distance calculation processing program in the VV feature amount calculation processing in step 212 of FIG. 5. [Figure 6B] 6 is a flowchart of a VV angle calculation processing program in the VV feature amount calculation processing in step 212 of FIG. 5. [Figure 7] FIG. 10 is a diagram showing an image of choroidal blood vessels. [Figure 8] This is a diagram showing the position of the macula M, the position of the optic disc ONH, and the position of the VV. [Figure 9] FIG. 1 is a diagram illustrating a four-quadrant arctangent function. [Figure 10] This is a diagram showing that the position M of the macula, the position O of the optic disc ONH, and the position V of the VV are located on the surface of a sphere centered on the center C of the eyeball. [Figure 11] FIG. 3 is a diagram showing a display screen 300 in a choroidal vessel analysis mode. [Figure 12] FIG. 3 shows a display screen 300 in which the vortex vein-macula / optic disc icon 338 is clicked, and the position between the vortex vein, macula, and optic disc, etc., is displayed. [Figure 13] 10 shows a display screen 300 in which a follow-up graph icon 346 has been clicked to display a follow-up graph 350. FIG. [Figure 14] FIG. 10 shows display screen 300 on which scatter plot display icon 348 is clicked and scatter plot 352A is displayed. [Figure 15] 13 shows display screen 300 in which scatter plot display icon 348 has been clicked to display another scatter plot 352B. DETAILED DESCRIPTION OF THE INVENTION

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. For convenience of explanation, a scanning laser ophthalmoscope will be referred to as an "SLO" below.

[0009] The configuration of an ophthalmologic system 100 will be described with reference to Fig. 1. As shown in Fig. 1, the ophthalmologic system 100 includes an ophthalmologic apparatus 110, an axial length measurement apparatus 120, a management server apparatus (hereinafter referred to as "management server") 140, and an image display apparatus (hereinafter referred to as "image viewer") 150. The ophthalmologic apparatus 110 acquires fundus images. The axial length measurement apparatus 120 measures the axial length of a patient. The management server 140 stores, in association with the patient IDs, a plurality of fundus images and axial lengths obtained by photographing the funduses of a plurality of patients using the ophthalmologic apparatus 110.

[0010] The ophthalmologic apparatus 110, the axial length measurement apparatus 120, the management server 140, and the image viewer 150 are connected to one another via a network .

[0011] In addition, other ophthalmic devices (examination devices such as OCT (Optical Coherence Tomography) measurement, visual field measurement, and intraocular pressure measurement) and diagnostic support devices that perform image analysis using artificial intelligence may be connected to the ophthalmic device 110, axial length measurement device 120, management server 140, and image viewer 150 via network 130.

[0012] Next, the configuration of the ophthalmologic apparatus 110 will be described with reference to Fig. 2. As shown in Fig. 2, the ophthalmologic apparatus 110 includes a control unit 20, a display / operation unit 30, and an SLO unit 40, and captures an image of the posterior segment (fundus) of the subject's eye 12. The apparatus may further include an OCT unit (not shown) that acquires OCT data of the fundus.

[0013] The control unit 20 includes a CPU 22, a memory 24, and a communication interface (I / F) 26. The display / operation unit 30 is a graphic user interface that displays captured images and receives various instructions including instructions for capturing images, and includes a display 32 and an input / instruction device 34 such as a touch panel.

[0014] The SLO unit 40 includes a light source 42 of G light (green light: wavelength 530 nm), a light source 44 of R light (red light: wavelength 650 nm), and a light source 46 of IR light (infrared light (near-infrared light): wavelength 800 nm). The light sources 42, 44, and 46 emit their respective lights in response to commands from the control unit 20. The SLO unit 40 includes optical systems 50, 52, 54, and 56 that reflect or transmit light from the light sources 42, 44, and 46 and guide it to a single optical path. The optical systems 50 and 56 are mirrors, and the optical systems 52 and 54 are beam splitters. G light is reflected by the optical systems 50 and 54, R light is transmitted through the optical systems 52 and 54, and IR light is reflected by the optical systems 52 and 56 and each is guided to a single optical path.

[0015] The SLO unit 40 includes a wide-angle optical system 80 that two-dimensionally scans light from the light sources 42, 44, and 46 over the posterior segment (fundus) of the subject's eye 12. The SLO unit 40 includes a beam splitter 58 that reflects G light from the posterior segment (fundus) of the subject's eye 12 and transmits all light other than G light. The SLO unit 40 includes a beam splitter 60 that reflects R light from the light that has passed through the beam splitter 58 and transmits all light other than R light. The SLO unit 40 includes a beam splitter 62 that reflects IR light from the light that has passed through the beam splitter 60. The SLO unit 40 includes a G light detecting element 72 that detects G light reflected by the beam splitter 58, an R light detecting element 74 that detects R light reflected by the beam splitter 60, and an IR light detecting element 76 that detects IR light reflected by the beam splitter 62.

[0016] The wide-angle optical system 80 includes an X-direction scanning device 82 composed of a polygon mirror that scans light from the light sources 42, 44, and 46 in the X direction, a Y-direction scanning device 84 composed of a galvanometer mirror that scans light in the Y direction, and an optical system 86 including a slit mirror and an elliptical mirror (not shown) that widens the angle of the scanned light. The optical system 86 increases the field of view (FOV) of the fundus to a larger angle than conventional techniques, enabling a wider fundus area to be imaged than conventional techniques. Specifically, it is possible to image a wider fundus area with an external light irradiation angle of approximately 120 degrees from outside the subject's eye 12 (approximately 200 degrees, with the center O of the eyeball of the subject's eye 12 as the reference position, and the internal light irradiation angle that can be substantially imaged by irradiating the fundus of the subject's eye 12 with scanning light). The optical system 86 may be configured using a group of multiple lenses instead of a slit mirror and an elliptical mirror. Each of the scanning devices, the X-direction scanning device 82 and the Y-direction scanning device 84, may be a two-dimensional scanner configured using MEMS mirrors.

[0017] When a system including a slit mirror and an elliptical mirror is used as optical system 86, a configuration using a system using an elliptical mirror as described in International Application No. PCT / JP2014 / 084619 or International Application No. PCT / JP2014 / 084630 may be used. The disclosures of International Application No. PCT / JP2014 / 084619 filed on December 26, 2014 (International Publication No. WO2016 / 103484) and International Application No. PCT / JP2014 / 084630 filed on December 26, 2014 (International Publication No. WO2016 / 103489) are each incorporated herein by reference in their entirety.

[0018] When the ophthalmologic apparatus 110 is placed on a horizontal plane, the horizontal direction is defined as the "X direction," the vertical direction relative to the horizontal plane is defined as the "Y direction," and the direction connecting the center of the pupil of the anterior segment of the subject's eye 12 and the center of the eyeball is defined as the "Z direction." Therefore, the X direction, Y direction, and Z direction are perpendicular to each other.

[0019] A color fundus image is obtained by simultaneously photographing the fundus of the subject's eye 12 with G light and R light. More specifically, the control unit 20 controls the light sources 42 and 44 to emit light simultaneously, and the G light and R light are scanned across the fundus of the subject's eye 12 by the wide-angle optical system 80. The G light reflected from the fundus of the subject's eye 12 is detected by the G light detection element 72, and image data of a second fundus image (G-color fundus image) is generated by the CPU 22 of the ophthalmic apparatus 110. Similarly, the R light reflected from the fundus of the subject's eye 12 is detected by the R light detection element 74, and image data of a first fundus image (R-color fundus image) is generated by the CPU 22 of the ophthalmic apparatus 110. Furthermore, when IR light is irradiated, the IR light reflected from the fundus of the subject's eye 12 is detected by the IR light detection element 76, and image data of the IR fundus image is generated by the CPU 22 of the ophthalmic apparatus 110.

[0020] The CPU 22 of the ophthalmologic apparatus 110 mixes the first fundus image (red fundus image) and the second fundus image (green fundus image) at a predetermined ratio to display a color fundus image on the display 32. Note that instead of the color fundus image, the first fundus image (red fundus image), the second fundus image (green fundus image), or an IR fundus image may be displayed.

[0021] Image data of the first fundus image (red-colored fundus image), image data of the second fundus image (green-colored fundus image), and image data of the IR fundus image are sent from the ophthalmologic apparatus 110 to the management server 140 via the communication IF 166. The various fundus images are used to generate a choroidal blood vessel image.

[0022] The axial length measurement device 120 in FIG. 1 has two modes, a first mode and a second mode, for measuring the axial length of the subject's eye 12, which is the length in the axial direction (Z direction). In the first mode, light from a light source (not shown) is guided to the subject's eye 12, and then interference light between reflected light from the fundus and reflected light from the cornea is received, and the axial length is measured based on an interference signal indicating the received interference light. In the second mode, ultrasound (not shown) is used to measure the axial length. The axial length measurement device 120 transmits the axial length measured in the first mode or the second mode to the management server 140. The axial length may be measured in both the first mode and the second mode. In this case, the average of the axial lengths measured in both modes is transmitted to the management server 140 as the axial length. The axial length is stored as patient information in the management server 140 as part of the subject's data, and is also used for fundus image analysis.

[0023] Next, the configuration of the management server 140 will be described with reference to Fig. 3. As shown in Fig. 3, the management server 140 includes a control unit 160 and a display / operation unit 170. The control unit 160 includes a computer including a CPU 162, a memory 164 which is a storage device, and a communication interface (I / F) 166. An image processing program is stored in the memory 164. The display / operation unit 170 is a graphic user interface which displays images and accepts various instructions, and includes a display 172 and an input / instruction device 174 such as a touch panel.

[0024] The configuration of the image viewer 150 is the same as that of the management server 140, and therefore a description thereof will be omitted.

[0025] Next, various functions realized by the CPU 162 of the management server 140 executing the image processing program will be described with reference to Fig. 4. The image processing program has an image processing function, a display control function, and a processing function. When the CPU 162 executes the image processing program having these functions, the CPU 162 functions as an image processing unit 182, a display control unit 184, and a processing unit 186, as shown in Fig. 4.

[0026] Next, the image processing by the management server 140 will be described in detail with reference to Fig. 5. The image processing shown in the flowchart of Fig. 5 is realized by the CPU 162 of the management server 140 executing an image processing program.

[0027] The image processing program is executed when the management server 140 generates a choroidal blood vessel image based on image data of a fundus image captured by the ophthalmologic apparatus 110. The choroidal vessel image is generated as follows. First, the information contained in the first fundus image (red fundus image) and the second fundus image (green fundus image) will be described. The eye is structured such that the vitreous body is covered by multiple layers with different structures. These multiple layers, from innermost on the vitreous body side to outermost, include the retina, choroid, and sclera. R light passes through the retina and reaches the choroid. Therefore, the first fundus image (R-color fundus image) contains information on the blood vessels present in the retina (retinal blood vessels) and the blood vessels present in the choroid (choroidal blood vessels). In contrast, G light only reaches the retina. Therefore, the second fundus image (G-color fundus image) contains information on the blood vessels present in the retina (retinal blood vessels). Therefore, a choroidal blood vessel image can be obtained by extracting the retinal blood vessels from the second fundus image (G-color fundus image) and removing the retinal blood vessels from the first fundus image (R-color fundus image).

[0028] Next, a method for generating a choroidal blood vessel image will be described. The image processing unit 182 of the management server 140 extracts retinal blood vessels from the second fundus image (green fundus image) by applying black hat filtering to the second fundus image. Next, the image processing unit 182 removes retinal blood vessels from the first fundus image (red fundus image) by inpainting using the retinal blood vessels extracted from the second fundus image (green fundus image). That is, the image processing unit 182 performs a process of filling in the retinal blood vessel structure of the first fundus image (red fundus image) with the same value as the surrounding pixels using position information of the retinal blood vessels extracted from the second fundus image (green fundus image). The image processing unit 182 then performs contrast limited adaptive histogram equalization (CLAHE) on the image data of the first fundus image (red fundus image) from which the retinal blood vessels have been removed, thereby enhancing the choroidal blood vessels in the first fundus image (red fundus image). This produces the choroidal vessel image shown in Figure 7. The generated choroidal vessel image is stored in memory 164. Furthermore, the choroidal blood vessel image is generated from the first fundus image (red fundus image) and the second fundus image (green fundus image), but the image processing unit 182 may then generate the choroidal blood vessel image using the first fundus image (red fundus image) or an IR fundus image captured with IR light. Regarding a method for generating a choroidal fundus image, the disclosure of Japanese Patent Application No. 2018-052246, filed on March 20, 2018, is incorporated herein by reference in its entirety.

[0029] When the image processing program starts, in step 202 of FIG. 5, the processing unit 186 reads out the choroidal blood vessel image (see FIG. 7) and the G-colored fundus image from the memory 164 as fundus images.

[0030] In step 204, the image processing unit 182 estimates the position of the macula from the G-colored fundus image. Specifically, since the macula is a dark region in the G-colored fundus image, the image processing unit 182 detects the region of a predetermined number of pixels with the smallest pixel values ​​in the read-out G-colored fundus image as the position of the macula.

[0031] In step 206, the image processing unit 182 detects the position of the optic disc from the G-colored fundus image. Specifically, the image processing unit 182 detects the optic disc in the G-colored fundus image by pattern matching a predetermined image of the optic disc with the read-out G-colored fundus image. Furthermore, since the optic disc is the brightest area in the G-colored fundus image, the area of ​​a predetermined number of pixels with the largest pixel value in the read-out G-colored fundus image may be detected as the position of the optic disc. The choroidal blood vessel image is created by processing the R-color fundus image and the G-color fundus image as described above. Therefore, when the coordinate system of the G-color fundus image is superimposed on the coordinate system of the choroidal blood vessel image, each position in the coordinate system of the G-color fundus image is the same as each position in the coordinate system of the choroidal blood vessel image. Therefore, each position on the choroidal blood vessel image corresponding to each position of the macula and optic disc detected from the G-color fundus image is the respective position of the macula and optic disc. Therefore, in the process of step 204, the position of the macula may be detected from the choroidal blood vessel image instead of the green fundus image. Similarly, in the process of step 206, the position of the optic disc may be detected from the choroidal fundus image instead of the green fundus image.

[0032] In step 208, the image processor 182 detects the positions of vortex veins (hereinafter referred to as "VV") in the choroidal vessel image. Here, vortex veins VV are outflow routes for blood flowing into the choroid, and there are four to six of them near the posterior pole of the equator of the eyeball. The image processing unit 182 determines the blood vessel running direction for each pixel in the choroidal blood vessel image. Specifically, the image processing unit 182 repeats the following process for all pixels. That is, the image processing unit 182 sets an area (cell) consisting of multiple pixels surrounding a pixel. Then, the image processing unit 182 calculates the brightness gradient direction (indicated as an angle between 0 degrees and less than 180 degrees, where 0 degrees is defined as the direction of a straight line (horizontal line)) for each pixel in the cell based on the brightness values ​​of the pixels surrounding the pixel to be calculated. This gradient direction calculation is performed for all pixels in the cell.

[0033] Next, to create a histogram with nine bins (each 20° wide) with gradient directions of 0°, 20°, 40°, 60°, 80°, 100°, 120°, 140°, and 160°, the number of pixels in the cell with the gradient direction corresponding to each bin is counted. The width of one bin in the histogram corresponds to 20°, and the 0° bin contains the number of pixels (count value) in the cell with a gradient direction between 0° and 10° and between 170° and 180°. The 20° bin contains the number of pixels (count value) in the cell with a gradient direction between 10° and 30°. Similarly, count values ​​are set for the 40°, 60°, 80°, 100°, 120°, 140°, and 160° bins. Because the histogram has nine bins, the vascular direction of a pixel is defined as one of nine directions. The resolution of the blood vessel direction can be improved by narrowing the bin width and increasing the number of bins. The count value in each bin (the vertical axis of the histogram) is normalized, and a histogram for the analysis point is created.

[0034] Next, the image processing unit 182 identifies the blood vessel running direction of the analysis point from the histogram. Specifically, it identifies the bin with the smallest count value (assumed to be 60 degrees), and identifies 60 degrees, the gradient direction of the identified bin, as the blood vessel running direction of the pixel. The gradient direction with the smallest count is determined to be the blood vessel running direction for the following reason: the brightness gradient is small in the blood vessel running direction, while the brightness gradient is large in other directions (for example, there is a large difference in brightness between blood vessels and non-blood vessels). Therefore, when a histogram of the brightness gradient of each pixel is created, the count value of the bin for the blood vessel running direction is small. In a similar manner, a histogram is created for each pixel in the choroidal blood vessel image, and the blood vessel running direction of each pixel is calculated. The calculated blood vessel running direction of each pixel is stored in memory 164.

[0035] Then, the image processing unit 182 sets initial positions of virtual particles, M in the vertical direction and N in the horizontal direction, totaling L, at equal intervals on the choroidal blood vessel image. For example, M=10, N=50, and a total of L=500 initial positions are set.

[0036] Furthermore, the image processing unit 182 acquires the blood vessel running direction of the initial position (any of the L positions), moves the virtual particle a predetermined distance along the acquired blood vessel running direction, acquires the blood vessel running direction again at the moved position, and moves the virtual particle a predetermined distance along the acquired blood vessel running direction. This process of moving the virtual particle a predetermined distance along the blood vessel running direction is repeated a preset number of times. The above process is performed for all L positions. After the set number of movements has been performed for all L virtual particles, the point where a certain number or more of virtual particles are gathered is detected as the VV position. Then, the number of detected VVs and VV position information (coordinates indicating the VV positions in the choroidal vessel image) are stored in the memory 164.

[0037] In step 210, the image processing unit 182 increments the variable n by 1 and selects one VV identified by the variable n. The initial value of n is zero.

[0038] In step 212, the image processing unit 182 calculates a first feature related to distance (VV distance) and a second feature related to angle (VV angle) for each VV. The first feature is the distance between the VV and a characteristic structure of the fundus, such as the distance between the macula and the VV, or the distance between the optic disc and the VV. The second feature is the angle specified by three points between the VV and the first and second characteristic structures of the fundus, such as the angle between the VV, the macula, and the optic disc, or the angle between the VV, the macula, and the optic disc.

[0039] In step 214, the image processing unit 182 determines whether processing has been completed for all VVs by determining whether the variable n is equal to the total number N of detected VVs. If it is not determined that processing has been completed for all VVs, the image processing returns to step 210, the variable n is incremented by 1, and the above processing (steps 210 to 214) is repeated.

[0040] If it is determined that processing has been completed for all VVs, the image processing of the flowchart in FIG. 5 ends.

[0041] Next, we will explain the calculation process of two types of VV feature amounts, the first feature amount (VV distance) and the second feature amount (VV angle), in step 212. The calculation process in step 212 includes the VV distance calculation process shown in Fig. 6A and the VV angle calculation process shown in Fig. 6B.

[0042] First, the VV distance calculation process will be described with reference to FIG. 6A. In step 221, the image processing unit 182 acquires the coordinates of the optic disc ONH, macula M, and VV positions in the choroidal fundus image. Next, in step 223, the image processing unit 182 projects the coordinates of the optic disc ONH, macula M, and VV positions onto a virtual spherical surface as shown in Fig. 10. The virtual spherical surface shown in Fig. 10 is a spherical surface with the center of the eyeball as C and the radius as R (the axial length of the eye is 2R). The position of the VV is projected onto this spherical surface as V, the position of the optic disc ONH as O, and the position of the macula as M. Considering this virtual sphere as an eyeball model, in step 225, the image processing unit 182 calculates the great circle distance between two points on the spherical surface as the VV distance. That is, a great circle is defined as a cross section of the sphere that passes through the center O of the sphere, and the great circle distance is defined as the length of the arc of a great circle that connects two points on the spherical surface that are the target of distance measurement (VV position: V and optic disc position: O, or VV position: V and macula: M). If the latitude and longitude of VV position: V on the virtual sphere are (latitude θ1, longitude φ1) and the latitude and longitude of optic disc position: O are (latitude θ2, longitude φ2), in step 223, the image processing unit 182 calculates the VV distance between the VV position and the optic disc position, i.e., the great circle distance OV, using the formulas of spherical trigonometry.

[0043]

number

[0044] Next, the calculation process of the VV angle will be described. As shown in Figure 8, the VV angle includes the angle θ formed by the position of the macula M, the position of the optic disc ONH, and the position of the VV, and the angle formed by the position of the optic disc ONH, the position of the macula M, and the position of the VV.

[0045] There are two methods for calculating the angle θ formed by the position of the macula M, the position of the optic disc ONH, and the position of VV: a method of calculation using conformal projection and a method of calculation using spherical trigonometry.

[0046] First, with reference to FIG. 6B, a method for calculating the angle θ from a conformal projection will be described.

[0047] In the method of calculating from the normal dot product, it is not possible to distinguish between positive and negative values ​​of the calculated angle, and it is not possible to distinguish between VV ((x3, y3) in FIG. 8) in the upper hemisphere and VV' ((x3, -y3)) in the lower hemisphere. In addition, in the method using the arctangent function, it is possible to distinguish between positive and negative values, but the calculation direction of θ is always constant (for example, counterclockwise), so when anatomical features (nasal side / ear side) are used as a reference, the values ​​of the upper and lower hemispheres for the left and right eyes are reversed. Therefore, in this embodiment, the left and right eye adjustment code f sign is used to adjust the sign of the calculated angle.

[0048] In step 222 of FIG. 6B, the image processing unit 182 calculates the eye-left / right adjustment code f sign As shown in FIG. 8, the position of the macula M is (x1, y1), the position of the optic disc ONH is (x2, y2), and the position of the VV is (x3, y3).

[0049] Left and right eye coordination code f sign teeth, f sign = +1 (when x1 > x2) f sign = -1 (when x1 < x2) is set as follows. Anatomically, from the positions of the macula and the optic disc, it can be determined that x1 > x2 for the left eye and x1 < x2 for the right eye. Also, if x1 = x2, then f sign = +1

[0050] In step 224, the image processing unit 182 calculates cosθ and sinθ using formulas 2 and 3 based on the definitions of the inner product and outer product of vectors. Here, the angle θ formed by the positions of the macula M - the optic disc ONH - the position of VV is the angle formed by the vector OM (the vector connecting the optic disc position O and the macula position M) and the vector OV (the vector connecting the optic disc position O and the vortex vein position V).

[0051]

Number

[0052]

Number

[0053] In step 226, the image processing unit 182 calculates θ using the inverse tangent function in the four quadrants as follows.

[0054]

Number

[0055] As shown in FIG. 9, θ obtained by the inverse tangent function in the four quadrants takes into account not only the value of y / x but also the sign of x in each of the four quadrants.

[0056] In step 228, the image processing unit 182 determines the sign of the calculated θ as f signIn step 230, the image processing unit 182 stores the value of θ thus obtained as the VV angle in the memory 164. In a similar manner, the angle formed by the position of the optic disc ONH, the position of the macula M, and the VV position can also be obtained.

[0057]

number

[0058] Next, a method for calculating the angle θ using spherical trigonometry will be described.

[0059] As shown in Figure 10, the position M of the macula, the position O of the optic disc ONH, and the position V of the VV are located on the surface of a sphere with the center C of the eyeball and a radius R when the axial length is 2R. If the apex angle O of the triangle OMV, whose vertices are the position M of the macula, the position O of the optic disc ONH, and the position V of the VV, is α, then

[0060]

number

[0061]

number

[0062] and α (where α is in the range [0,π]) can be calculated.

[0063] To change the value of α to θ (within the open interval [-π,π]), use g sign Considering (the positional relationship between the macula M and the optic disc ONH, the positional relationship between the VV position V and the macula M) = {1, -1}, θ = α·g sign The value of θ thus obtained is stored in the memory 164 as the VV angle. The image processing program includes either a program for calculating the VV angles from the conformal projection shown in FIG. 6B, or a program for calculating from the spherical trigonometry described above.

[0064] Next, the data of the display screen in the choroidal vessel analysis mode will be described. The management server 140 has the following data of the display screen in the choroidal vessel analysis mode.

[0065] First, as described above, image data of fundus images (first fundus image (red-colored fundus image) and second fundus image (green-colored fundus image)) is transmitted from the ophthalmologic apparatus 110 to the management server 140, and the management server 140 has the image data of the fundus images (first fundus image (red-colored fundus image) and second fundus image (green-colored fundus image)). The management server 140 has image data of a choroidal blood vessel image (see FIG. 7), the position of VV, the position of the optic disc ONH, the position of the macula M, and feature quantities of VV. As described above, the feature quantities of VV include the distance between the position of the optic disc ONH and the position of VV, the distance between the position of the macula and the position of VV, the angle θ formed by the position of the macula M - the position of the optic disc ONH - the position of VV, and the angle formed by the position of the optic disc ONH - the position of the macula M - the position of VV.

[0066] Furthermore, when the fundus of the patient is photographed, the patient's personal information is input to the ophthalmologic device 110. The personal information includes the patient's ID, name, age, eyesight, etc. Furthermore, when the fundus of the patient is photographed, information indicating whether the eye whose fundus is to be photographed is the right eye or the left eye is also input. Furthermore, when the fundus of the patient is photographed, the date and time of photographing is also input. The ophthalmologic device 110 transmits data on the personal information, information on the right eye / left eye, and the date and time of photographing to the management server 140. The management server 140 has data on the personal information, information on the right eye / left eye, and the date and time of photographing. The management server 140 has data on the axial length.

[0067] As described above, the management server 140 has the data of the display screen of the choroidal vessel analysis mode. Incidentally, a doctor in a room equipped with the image viewer 150 may want to know the state of choroidal blood vessels when diagnosing a patient. In this case, the doctor transmits an instruction to the management server 140 via the image viewer 150 to transmit display screen data for the choroidal blood vessel analysis mode. Upon receiving this instruction, the management server 140 transmits the display screen data for the choroidal blood vessel analysis mode to the image viewer 150. The image viewer 150, having received the display screen data for the choroidal blood vessel analysis mode, displays the display screen 300 for the choroidal blood vessel analysis mode shown in FIG. 11 on the display based on the display screen data for the choroidal blood vessel analysis mode.

[0068] The display screen of the image viewer 150, which will be described later, displays icons and buttons for instructing the generation of an image, which will be described later. When the ophthalmologist clicks on an icon or the like, an instruction signal corresponding to the clicked icon or the like is transmitted from the image viewer 150 to the image management server 140. Upon receiving the instruction signal from the image viewer 150, the management server 140 generates an image corresponding to the instruction signal and transmits image data of the generated image to the image viewer 150. Upon receiving image data from the image management server 140, the image viewer 150 displays the image on the display based on the received image data. The display screen generation process in the management server 140 is performed by a display screen generation program running on the CPU 162. The management server 140 is an example of the "image processing device" of the technology of the present disclosure.

[0069] Here, we will explain the display screen 300 of the choroidal vessel analysis mode shown in Fig. 11. As shown in Fig. 11, the display screen 300 of the choroidal vessel analysis mode has a personal information display field 302 that displays the patient's personal information, an image display field 320, and a choroidal analysis tool display field 330.

[0070] The personal information display field 302 has a patient ID display field 304, a patient name display field 306, an age display field 308, an axial length display field 310, a visual acuity display field 312, and a patient selection icon 314. Each piece of information is displayed in the patient ID display field 304, the patient name display field 306, the age display field 308, the axial length display field 310, and the visual acuity display field 312. When the patient selection icon 314 is clicked, a list of patients is displayed on the display 172 of the image viewer 150, and the user (such as an ophthalmologist) is allowed to select a patient to analyze.

[0071] The image display field 320 has a photographing date display field 322N1, a right eye information display field 324R, a left eye information display field 324L, an RG image display field 326, a choroidal blood vessel image display field 328, and an information display field 342. The RG image is an image obtained by combining the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image) at a predetermined ratio (for example, 1:1) of the magnitude of each pixel value.

[0072] The choroid analysis tool display field 330 includes a plurality of choroid analysis tools that instruct the image viewer 150 to perform processing, such as a vortex vein location analysis icon 332, a symmetry icon 334, a vessel diameter icon 336, a vortex vein and macula / optic disc icon 338, and a choroid analysis report icon 340. The vortex vein location analysis icon 332 instructs the user to identify the vortex vein location. The symmetry icon 334 instructs the user to analyze the symmetry of the vortex vein. The vessel diameter icon 336 instructs the user to execute a tool that analyzes the diameter of the choroidal vessels. The vortex vein and macula / optic disc icon 338 instructs the user to analyze the position between the vortex vein, the macula, and the optic disc. The choroid analysis report icon 340 instructs the user to display a choroid analysis report.

[0073] In the example shown in FIG. 11, RG images and choroid images can be displayed for the fundus of the right eye (324R is lit) of a patient identified by patient ID: 123456, photographed on March 10, 2018, December 10, 2017, and September 10, 2017. When the photographing date display field 322N1 is clicked, the RG image and choroid image photographed on March 10, 2018 are displayed. Furthermore, a follow-up observation graph icon 346 and a scatter plot display icon 348 are displayed in the image display field 320.

[0074] When the vortex vein-macula / optic disc icon 338 is clicked on the display screen 300 shown in FIG. 11, the positions between the vortex vein, macula, and optic disc, etc., are displayed as shown in FIG. 12. In the example shown in FIG. 12, two VVs have been detected, and a message stating "Two vortex veins have been detected" is displayed in the information display field 342. The positions of the optic disc, macula, and VV are superimposed on the RG image in the RG image display field 326 and the choroidal vessel image in the choroidal vessel image display field 328. The distances between the position of the optic disc ONH and the positions of VV1 and VV2 (VV1 distance = 11.1 mm, VV2 distance = 15.4 mm) are superimposed on the RG image in the RG image display field 326 and the choroidal vessel image in the choroidal vessel image display field 328. The angle formed by the position of the macula M, the position of the optic disc ONH, and the positions of VV1 and VV2 (VV1 angle = 60 degrees, VV2 angle = -62 degrees) is superimposed on the RG image in the RG image display field 326 and the choroidal vessel image in the choroidal vessel image display field 328.

[0075] When the follow-up graph icon 346 is clicked on the display screen 300 in Fig. 12, a follow-up graph 350 shown in Fig. 13 is displayed. In the example shown in Fig. 13, the message "Vortex vein 1 has been graphed" is displayed in the information display field 342. The follow-up graph 350 displays follow-up graphs of the VV1 distance and VV1 angle for vortex vein 1 (VV1) on each imaging date (March 10, 2018, December 10, 2017, and September 10, 2017).

[0076] When the scatter diagram display icon 348 is clicked on the display screen 300 in Fig. 12, a scatter diagram 352A shown in Fig. 14 is displayed. Scatter diagram 352A displays VV positions according to a positional relationship based on the vertical axis and the temporal-nasal axis. The scatter diagram is not limited to scatter diagram 352A shown in Fig. 14, and a scatter diagram 352B shown in Fig. 15 may be displayed, in which the VV positions are displayed according to a positional relationship based on an angle relative to a horizontal line extending from the center to the right side of the fundus, with the center of the fundus image as the center. Note that the display content of the choroidal vessel image display field 328 in Figs. 14 and 15 remains unchanged.

[0077] As described above, in this embodiment, feature values ​​(quantitative VV values) related to the positional relationship between the vortex vein position and the macula and optic disc are calculated, and the VV feature values ​​(VV angle and VV distance) are superimposed on a fundus image (RG image or choroidal vessel image). In this embodiment, VV feature amounts (VV angle and VV distance) can be displayed. By displaying the relationship between the VV position and the positions of the macula and optic disc, which are characteristic structures of the fundus, as numerical values ​​of distance and angle, fundus diagnosis by ophthalmologists can be supported. is doing.

[0078] Next, various modifications of the technique of the present disclosure will be described. <First Modification> In the above embodiment, the distance and angle between the position of the optic disc ONH and the position of the VV are calculated and superimposed on the fundus image, but the technology of the present disclosure is not limited to this, and either the distance or the angle between the position of the optic disc ONH and the position of the VV may be superimposed on the fundus image. The same applies to the position of the macula M and the position of the VV.

[0079] <Second Modification> In the above embodiment, the distance between the position of the optic disc ONH and the position of the VV is a distance on a spherical surface, but the technology of the present disclosure is not limited to this, and the distance may be a linear distance between the position of the optic disc ONH and the position of the VV. Note that in the first modified example, the distance between the position of the macula M and the position of the VV may also be a linear distance.

[0080] <Third Modification> In the above embodiment, the management server 140 executes the image processing program shown in Fig. 5 in advance, but the technology of the present disclosure is not limited to this. For example, the following may be performed. When the vortex vein / macula / optic disc icon 338 shown in Fig. 11 is clicked, the image viewer 150 sends a command to the management server 140. In response to this, the management server 140 executes the image processing program shown in Fig. 5, and the image viewer 150 displays the display screen shown in Fig. 12.

[0081] <Fourth Modification> In the above embodiment, an example has been described in which a fundus image with an internal light irradiation angle of approximately 200 degrees is acquired by the ophthalmic apparatus 110. The technology of the present disclosure is not limited to this, and the technology of the present disclosure may also be applied to a fundus image captured by an ophthalmic apparatus with an internal irradiation angle of 100 degrees or less, or to a montage image in which multiple fundus images are combined. <Fifth Modification> In the above embodiment, fundus images are captured using an ophthalmic apparatus 110 equipped with an SLO imaging unit. However, the technology of the present disclosure may also be applied to fundus images captured using a fundus camera capable of capturing images of choroidal blood vessels, or to images obtained by OCT angiography.

[0082] <Sixth Modification> In the above embodiment, the management server 140 executes the image processing program. However, the technology of the present disclosure is not limited to this. For example, the ophthalmic apparatus 110 or the image viewer 150 may execute the image processing program. When the ophthalmic apparatus 110 executes the image processing program, the image processing program is stored in the memory 24. When the image viewer 150 executes the image processing program, the image processing program is stored in the memory 164 of the image viewer 150.

[0083] <Seventh Modification> In the above embodiment, the ophthalmic system 100 including the ophthalmic apparatus 110, the axial length measurement apparatus 120, the management server 140, and the image viewer 150 has been described as an example. However, the technology of the present disclosure is not limited thereto. For example, as a first example, the axial length measurement apparatus 120 may be omitted, and the ophthalmic apparatus 110 may further have the functions of the axial length measurement apparatus 120. Furthermore, as a second example, the ophthalmic apparatus 110 may further have the functions of at least one of the management server 140 and the image viewer 150. For example, if the ophthalmic apparatus 110 has the functions of the management server 140, the management server 140 can be omitted. In this case, the image processing program is executed by the ophthalmic apparatus 110 or the image viewer 150. Furthermore, if the ophthalmic apparatus 110 has the functions of the image viewer 150, the image viewer 150 can be omitted. As a third example, the management server 140 may be omitted, and the image viewer 150 may perform the functions of the management server 140.

[0084] <Eighth Modification> In the above embodiment, a red-colored fundus image captured with red light is used as the first fundus image, but an IR fundus image captured with infrared light may also be used. In other words, since red light or infrared light is used, light that reaches the choroid of multiple layers including the retina and choroid, which cover the vitreous body of the eye and are located from the innermost to the outermost on the vitreous body side and have different structures, is used.

[0085] <Ninth Variation> In the above embodiment, the fundus image is obtained by simultaneously photographing the fundus of the subject's eye 12 using G light and R light. The technology of the present disclosure is not limited to this. For example, the fundus of the subject's eye 12 may be photographed using G light and R light at different times. In this case, the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image) are aligned in advance.

[0086] <Other variations> The data processing described in the above embodiment is merely an example, and it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed, without departing from the spirit of the invention. Furthermore, in the above embodiment, an example was given in which data processing is realized by a software configuration using a computer, but the technology of the present disclosure is not limited to this. For example, instead of a software configuration using a computer, data processing may be performed only by a hardware configuration such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). Part of the data processing may be performed by a software configuration, and the remaining part may be performed by a hardware configuration.

Claims

1. an estimation step of analyzing the choroidal vessel image and estimating the position of the vortex vein; a calculation step of calculating a feature amount indicating a positional relationship between the position of the vortex vein and the position of a specific site on the fundus; An image processing method comprising:

2. the feature amount is a distance between the position of the vortex vein and the position of the specific part; The image processing method according to claim 1 .

3. The position of the specific site is the position of the optic disc or the position of the macula.

3. The image processing method according to claim 1.

4. the position of the specific site includes a position of a first specific site and a position of a second specific site; the calculating step calculates an angle between a first line segment connecting the position of the vortex vein and the position of the first specific part and a second line segment connecting the position of the second specific part and the position of the first specific part; The image processing method according to any one of claims 1 to 3.

5. The location of the first specific site is the optic disc, and the location of the second specific site is the macula. The image processing method according to claim 4.

6. the calculating step projects the coordinates of the vortex vein and the specific site onto a surface of a virtual sphere, and calculates a feature amount indicating a positional relationship between the position of the vortex vein and the specific site on the virtual sphere. The image processing method according to any one of claims 1 to 5.

7. the calculating step calculates the feature amount based on the center of the virtual sphere. The image processing method according to claim 6.

8. A program for causing a computer to execute the image processing method according to any one of claims 1 to 7.

9. a storage device that stores a program for causing the processing device to execute the image processing method; a processing device that executes the image processing method by executing a program stored in the storage device; An image processing device comprising: The image processing method is the image processing method according to any one of claims 1 to 7. Image processing device.

Images