Image processing method, program, image processing device, and ophthalmic system

Abstract

To acquire the asymmetry of a chorioid.SOLUTION: In a chorioid blood vessel image, there is acquired a blood vessel running direction of each analysis point of a plurality of groups of analysis points in line symmetry with respect to a line connecting a macula lutea and a discus nervi optici. The asymmetry of each analysis point of the plurality of groups is analyzed from the blood vessel running direction of each analysis point. The group of the analysis points of the asymmetry is surrounded by a frame to be emphatically displayed.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The technology disclosed herein relates to an image processing method, a program, an image processing device, and an ophthalmologic system. [Background technology]

[0002] JP 2015-202236 A discloses a technique for extracting a blood vessel region and measuring the blood vessel diameter. There has been a demand for analyzing a fundus image to measure the blood vessel diameter. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-202236 A Summary of the Invention

[0004] An image processing method of a first aspect of the technology disclosed herein includes the steps of setting a first analysis point and a second analysis point in a fundus image that are symmetrical with respect to a reference line, determining a first blood vessel running direction at the first analysis point and a second blood vessel running direction at the second analysis point, and analyzing asymmetry between the first blood vessel running direction and the second blood vessel running direction.

[0005] An image processing method of a second aspect of the technology disclosed herein includes the steps of setting a plurality of first analysis points within a first region and setting a plurality of second analysis points within a second region in a fundus image, determining a first blood vessel running direction for each of the plurality of first analysis points and a second blood vessel running direction for each of the plurality of second analysis points, and defining a plurality of combinations of first analysis points and second analysis points that are line-symmetric between the plurality of first analysis points and the plurality of second analysis points, and determining a symmetry index indicating the symmetry between the first blood vessel running direction and the second blood vessel running direction for each of the defined combinations.

[0006] A program according to a third aspect of the technique of the present disclosure causes a computer to execute the image processing method according to the first or second aspect.

[0007] An image processing device of a fourth aspect of the technology disclosed herein is an image processing device comprising a storage device that stores a program for causing a processing device to execute an image processing method, and a processing device that executes the image processing method by executing the program stored in the storage device, wherein the image processing method is an image processing method of the first aspect or the second aspect.

[0008] An ophthalmologic system according to a fifth aspect of the disclosed technique includes the image processing device according to the fourth aspect, and an ophthalmologic apparatus that captures the fundus image. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram of an ophthalmology system 100. [Diagram 2] 1 is a schematic diagram showing the overall configuration of an ophthalmic apparatus 110. FIG. [Diagram 3] 2 is a block diagram of the electrical configuration of the management server 140. FIG. [Figure 4] 2 is a functional block diagram of a CPU 162 of the management server 140. FIG. [Diagram 5] 13 is a flowchart of an image processing program. [Figure 6] 6 is a flowchart of a processing program for analyzing the blood vessel running direction in step 210 of FIG. 5. [Figure 7] 6 is a flowchart of a processing program for analyzing symmetry of blood vessel running directions in step 212 of FIG. 5. [Figure 8A] FIG. 13 is a diagram showing an image of choroidal blood vessels. [Figure 8B] FIG. 13 is a diagram showing a plurality of analysis points set on a choroidal blood vessel image. [Figure 9] This figure shows a histogram of the gradient directions of analysis points 242 and 246 arranged in line symmetry with respect to the line LIN in a choroidal vessel image rotated so that the line LIN connecting the macula M and the optic disc is horizontal. [Figure 10] FIG. 13 is a diagram showing the positional relationship between a straight line LIN, each analysis point, and each histogram. [Figure 11] FIG. 3 is a diagram showing a display screen 300 in a choroidal blood vessel analysis mode. [Figure 12] This is a display screen that is displayed when symmetry icon 334 is clicked on the display screen of FIG. [Figure 13] This is a display screen that is displayed when asymmetric histogram display icon 346 is clicked on the display screen of FIG. [Figure 14] This is a display screen that is displayed when the asymmetry color display icon 348 is clicked on the display screen of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. For convenience of explanation, a scanning laser ophthalmoscope will be referred to as "SLO" below.

[0011] The configuration of an ophthalmologic system 100 will be described with reference to Fig. 1. As shown in Fig. 1, the ophthalmologic system 100 includes an ophthalmologic apparatus 110, an axial length measuring device 120, a management server apparatus (hereinafter referred to as "management server") 140, and an image display device (hereinafter referred to as "image viewer") 150. The ophthalmologic apparatus 110 acquires a fundus image. The axial length measuring device 120 measures the axial length of a patient. The management server 140 stores a plurality of fundus images and axial lengths obtained by photographing the funduses of a plurality of patients by the ophthalmologic apparatus 110 in association with the patient's ID.

[0012] The ophthalmologic apparatus 110, the axial length measuring device 120, the management server 140, and the image viewer 150 are connected to each other via a network 130.

[0013] In addition, other ophthalmic devices (examination devices for OCT (Optical Coherence Tomography) measurement, visual field measurement, intraocular pressure measurement, etc.) and diagnostic support devices that perform image analysis using artificial intelligence may be connected to the ophthalmic device 110, the axial length measuring device 120, the management server 140, and the image viewer 150 via the network 130.

[0014] Next, the configuration of the ophthalmic apparatus 110 will be described with reference to Fig. 2. As shown in Fig. 2, the ophthalmic apparatus 110 includes a control unit 20, a display / operation unit 30, and an SLO unit 40, and captures an image of the posterior segment (fundus) of the subject's eye 12. The apparatus may further include an OCT unit (not shown) that acquires OCT data of the fundus.

[0015] The control unit 20 includes a CPU 22, a memory 24, and a communication interface (I / F) 26. The display / operation unit 30 is a graphic user interface that displays captured images and receives various instructions including instructions for shooting, and includes a display 32 and an input / instruction device 34 such as a touch panel.

[0016] The SLO unit 40 includes a light source 42 of G light (green light: wavelength 530 nm), a light source 44 of R light (red light: wavelength 650 nm), and a light source 46 of IR light (infrared light (near infrared light): wavelength 800 nm). The light sources 42, 44, and 46 emit their respective lights in response to commands from the control unit 20. The SLO unit 40 includes optical systems 50, 52, 54, and 56 that reflect or transmit the light from the light sources 42, 44, and 46 and guide it to one optical path. The optical systems 50 and 56 are mirrors, and the optical systems 52 and 54 are beam splitters. The G light is reflected by the optical systems 50 and 54, the R light is transmitted through the optical systems 52 and 54, and the IR light is reflected by the optical systems 52 and 56 and each is guided to one optical path.

[0017] The SLO unit 40 includes a wide-angle optical system 80 that two-dimensionally scans the light from the light sources 42, 44, and 46 over the posterior segment (fundus) of the subject's eye 12. The SLO unit 40 includes a beam splitter 58 that reflects G light from the posterior segment (fundus) of the subject's eye 12 and transmits light other than G light. The SLO unit 40 includes a beam splitter 60 that reflects R light from the light that has transmitted through the beam splitter 58 and transmits light other than R light. The SLO unit 40 includes a beam splitter 62 that reflects IR light from the light that has transmitted through the beam splitter 60. The SLO unit 40 includes a G light detection element 72 that detects G light reflected by the beam splitter 58, an R light detection element 74 that detects R light reflected by the beam splitter 60, and an IR light detection element 76 that detects IR light reflected by the beam splitter 62.

[0018] The wide-angle optical system 80 includes an X-direction scanning device 82 composed of a polygon mirror that scans the light from the light sources 42, 44, and 46 in the X-direction, a Y-direction scanning device 84 composed of a galvanometer mirror that scans in the Y-direction, and an optical system 86 that includes a slit mirror and an elliptical mirror (not shown) and converts the scanned light into a wide angle. The optical system 86 makes the field of view (FOV) of the fundus an ultra-wide field around the fundus, and can photograph a wide range of the fundus area. Specifically, the fundus area can be photographed at an external light irradiation angle of about 120 degrees from the outside of the subject's eye 12 (an internal light irradiation angle of about 200 degrees that can be practically photographed by irradiating the fundus of the subject's eye 12 with the scanning light, with the center O of the eyeball of the subject's eye 12 as the reference position). The optical system 86 may be configured using a group of lenses instead of the slit mirror and the elliptical mirror. Each of the X-direction scanning device 82 and the Y-direction scanning device 84 may use a two-dimensional scanner configured using a MEMS mirror.

[0019] When a system including a slit mirror and an elliptical mirror is used as the optical system 86, a configuration using a system using an elliptical mirror described in International Application PCT / JP2014 / 084619 or International Application PCT / JP2014 / 084630 may be used. The disclosures of International Application PCT / JP2014 / 084619 (International Publication WO2016 / 103484) filed on December 26, 2014 and International Application PCT / JP2014 / 084630 (International Publication WO2016 / 103489) filed on December 26, 2014 are each incorporated herein by reference in their entirety.

[0020] When the ophthalmic apparatus 110 is placed on a horizontal plane, the horizontal direction is defined as the "X direction", the vertical direction to the horizontal plane is defined as the "Y direction", and the direction connecting the center of the pupil of the anterior part of the subject's eye 12 and the center of the eyeball is defined as the "Z direction". Therefore, the X direction, Y direction, and Z direction are perpendicular to each other.

[0021] The color fundus image is obtained by photographing the fundus of the subject's eye 12 simultaneously with G light and R light. More specifically, the control unit 20 controls the light sources 42 and 44 to emit light simultaneously, and the G light and R light are scanned over the fundus of the subject's eye 12 by the wide-angle optical system 80. Then, the G light reflected from the fundus of the subject's eye 12 is detected by the G light detection element 72, and image data of the second fundus image (G color fundus image) is generated by the CPU 22 of the ophthalmic device 110. Similarly, the R light reflected from the fundus of the subject's eye 12 is detected by the R light detection element 74, and image data of the first fundus image (R color fundus image) is generated by the CPU 22 of the ophthalmic device 110. In addition, when IR light is irradiated, the IR light reflected from the fundus of the subject's eye 12 is detected by the IR light detection element 76, and image data of the IR fundus image is generated by the CPU 22 of the ophthalmic device 110.

[0022] The eye is structured such that the vitreous body is covered by multiple layers with different structures. The multiple layers include, from the innermost on the vitreous body side to the outermost, the retina, choroid, and sclera. R light passes through the retina and reaches the choroid. Therefore, the first fundus image (R-color fundus image) contains information on the blood vessels present in the retina (retinal blood vessels) and the blood vessels present in the choroid (choroidal blood vessels). In contrast, G light only reaches the retina. Therefore, the second fundus image (G-color fundus image) contains information on the blood vessels present in the retina (retinal blood vessels).

[0023] The CPU 22 of the ophthalmologic apparatus 110 mixes the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image) at a predetermined ratio, and displays the result as a color fundus image on the display 32. Note that instead of the color fundus image, the first fundus image (R-color fundus image), the second fundus image (G-color fundus image), or the IR fundus image may be displayed.

[0024] Image data of the first fundus image (R-color fundus image), image data of the second fundus image (G-color fundus image), and image data of the IR fundus image are sent from the ophthalmic device 110 to the management server 140 via the communication IF26 and stored in the memory 164 described later.

[0025] In this way, the fundus of the test eye 12 is photographed simultaneously using G light and R light, so that each position in the first fundus image (R-color fundus image) and the corresponding position in the second fundus image (G-color fundus image) are the same positions on the fundus.

[0026] The axial length measuring instrument 120 in FIG. 1 has two modes, a first mode and a second mode, for measuring the axial length, which is the length of the eye 12 in the axial direction (Z direction). In the first mode, light from a light source (not shown) is guided to the eye 12, and then the interference light between the reflected light from the fundus and the reflected light from the cornea is received, and the axial length is measured based on an interference signal indicating the received interference light. In the second mode, the axial length is measured using ultrasound (not shown). The axial length measuring instrument 120 transmits the axial length measured in the first mode or the second mode to the management server 140. The axial length may be measured in the first mode and the second mode, and in this case, the average of the axial lengths measured in both modes is transmitted to the management server 140 as the axial length.

[0027] The axial length is stored in the memory 164 as one piece of patient data in the management server 140 as patient information, and is also used in fundus image analysis.

[0028] Next, the configuration of the management server 140 will be described with reference to Fig. 3. As shown in Fig. 3, the management server 140 includes a control unit 160 and a display / operation unit 170. The control unit 160 includes a computer including a CPU 162, a memory 164 which is a storage device, and a communication interface (I / F) 166. An image processing program is stored in the memory 164. The display / operation unit 170 is a graphic user interface which displays images and receives various instructions, and includes a display 172 and an input / instruction device 174 such as a touch panel. The management server 140 is an example of an "image processing device" of the technology of the present disclosure.

[0029] The configuration of the image viewer 150 is similar to that of the management server 140, and therefore a description thereof will be omitted.

[0030] Next, various functions realized by the CPU 162 of the management server 140 executing the image processing program will be described with reference to Fig. 4. The image processing program has an image processing function, a display control function, and a processing function. By the CPU 162 executing the image processing program having these functions, the CPU 162 functions as an image processing unit 182, a display control unit 184, and a processing unit 186, as shown in Fig. 4.

[0031] Next, the image processing by the management server 140 will be described in detail with reference to Fig. 5. The image processing shown in the flowchart of Fig. 5 is realized by the CPU 162 of the management server 140 executing an image processing program.

[0032] The image processing program is executed when the management server 140 generates a choroidal blood vessel image based on image data of a fundus image captured by the ophthalmologic apparatus 110.

[0033] The choroidal blood vessel image is generated as follows. The image processing unit 182 of the management server 140 extracts retinal blood vessels from the second fundus image (G-color fundus image) by applying black hat filter processing to the second fundus image (G-color fundus image). Next, the image processing unit 182 removes retinal blood vessels from the first fundus image (R-color fundus image) by inpainting processing using the retinal blood vessels extracted from the second fundus image (G-color fundus image). That is, the image processing unit 182 performs processing to paint the retinal blood vessel structure of the first fundus image (R-color fundus image) to the same value as the surrounding pixels using the position information of the retinal blood vessels extracted from the second fundus image (G-color fundus image). Then, the image processing unit 182 enhances the choroidal blood vessels in the first fundus image (R-color fundus image) by applying adaptive histogram equalization processing (Contrast Limited Adaptive Histogram Equalization) to the image data of the first fundus image (R-color fundus image) from which the retinal blood vessels have been removed. As a result, the choroidal vessel image shown in Fig. 8A is obtained. The generated choroidal vessel image is stored in memory 164. The choroidal vessel image is an example of a "fundus image" of the technology of the present disclosure. In addition, the choroidal blood vessel image is generated from the first fundus image (red fundus image) and the second fundus image (green fundus image), and the image processing unit 182 may then generate the choroidal blood vessel image using the first fundus image (red fundus image) or an IR fundus image captured with IR light. The disclosure of Japanese Patent Application No. 2018-052246, filed on March 20, 2018, regarding a method for generating a choroidal fundus image, is incorporated herein by reference in its entirety.

[0034] When the image processing program starts, in step 202 in Fig. 5, the processing unit 186 reads out the choroidal blood vessel image (see Fig. 8A) and the green fundus image from the memory 164. The green fundus image clearly captures the macula and optic disc, and is easier to distinguish between the macula and optic disc by image processing than the choroidal blood vessel image. Therefore, the green fundus image is used for detecting the positions of the macula and optic disc, which will be described below.

[0035] In step 204, the image processing unit 182 detects the optic disc ONH (see also FIG. 9) from the G-colored fundus image. Since the G-colored (green) laser light is reflected by the retinal layer, it is preferable to use a G-colored fundus image photographed with the G-colored laser light in order to extract retinal structures. Since the optic disc ONH is the brightest area in the G-colored fundus image, the image processing unit 182 detects the area of ​​a predetermined number of pixels with the largest pixel value in the read-out G-colored fundus image as the optic disc (ONH). The center position of the area including the brightest pixel is calculated as the coordinates where the optic disc (ONH) is located, and is stored in the memory 164.

[0036] In step 206, the image processor 182 detects the macula M (see also FIG. 9 ) from the green fundus image. Specifically, since the macula is a dark region in the choroidal blood vessel image, the image processor 182 detects an area of ​​a predetermined number of pixels with the smallest pixel value in the above-read choroidal blood vessel image as the macula M. The center position of the area including the darkest pixel is calculated as the coordinates where the macula M is located, and is stored in the memory 164.

[0037] In step 208, the image processing unit 182 reads out the coordinates of the macula M and the coordinates of the optic disc ONH calculated from the G-color fundus image, as shown in Fig. 8B and Fig. 9. The image processing unit 182 sets each of the read out coordinates on the choroidal blood vessel image, and sets a straight line LIN connecting the macula M and the optic disc ONH on the choroidal blood vessel image. Here, since the choroidal blood vessel image is generated from the G-color fundus image and the R-color fundus image, the coordinates of the macula M and the optic disc ONH detected in the G-color fundus image also match the positions of the macula M and the optic disc ONH in the choroidal blood vessel image. Then, the image processing unit 182 rotates the choroidal blood vessel image so that this straight line LIN becomes horizontal.

[0038] In step 210, the image processing unit 182 analyzes the vascular running direction of the choroidal blood vessels, in step 212, the image processing unit 182 analyzes the symmetry of the vascular running direction of the choroidal blood vessels, and in step 214, the image processing unit 182 stores the analysis result in the memory 164. The processing of steps 210 and 212 will be described in detail below.

[0039] Next, the analysis process of the blood vessel running direction in step 210 will be described with reference to Figures 6, 8B, and 9. In step 222 in Figure 6, the image processing unit 182 sets analysis points as follows.

[0040] 8B, a first region 274 and a second region 272 are set by a straight line LIN in the choroidal vessel image. Specifically, the first region is located above the straight line LIN, and the second region is located below the straight line LIN.

[0041] The image processing unit 182 arranges the analysis points 240KU in the first region 274 so that they are located in a grid pattern at equal intervals, with M (natural number) rows in the vertical direction and N (natural number) columns in the horizontal direction. In FIG. 8B, the number of analysis points in the first region 264 is M(3)×N(7) (=L:21). Note that since the choroidal blood vessel image is displayed according to a conformal projection, the analysis points are located in a grid pattern, but if the choroidal blood vessel image is displayed in another projection, the image processing unit 182 arranges the analysis points in a pattern that matches the other projection. The image processing unit 182 places the analysis point 240KD in the second region 272 at a position that is line-symmetrical to the analysis point 240KU placed in the first region 274 with respect to the line LIN.

[0042] In addition, the analysis points 240KU, 240KD need only be located in positions that are linearly symmetrical with respect to the line LIN in the first region 274 and the second region 272, so they are not limited to being located in an equally spaced grid pattern, and may not be equally spaced or in a grid pattern. The sizes of the first region 274 and the second region 272 may be changed according to the axial length. The number of L, M, and N are not limited to the above example, and can be set to various values. Increasing the number increases the resolution.

[0043] In step 224, the image processing unit 182 calculates the direction of choroidal blood vessels at each analysis point. Specifically, the image processing unit 182 repeats the following process for each of all analysis points. That is, as shown in FIG. 9, the image processing unit 182 sets an area (cell) 244 consisting of a plurality of surrounding pixels centered on a central pixel corresponding to the analysis point 242. Region 244 is shown upside down in Figures 8B and 9 for ease of comparison with region 248, which contains the upper set of analysis points 246.

[0044] Then, the image processing unit 182 calculates the luminance gradient direction (indicated by an angle between 0 degrees and less than 180 degrees. Note that 0 degrees is defined as the direction of the straight line LIN (horizon)) of each pixel in the cell 244 based on the luminance values ​​of the pixels surrounding the calculation target pixel. This calculation of the gradient direction is performed for all pixels in the cell 244.

[0045] Next, the image processing unit 182 counts the number of pixels in the cell 244 of the gradient direction corresponding to each bin to create a histogram 242H having nine bins (each bin has a width of 20 degrees) with gradient directions of 0 degrees, 20 degrees, 40 degrees, 60 degrees, 80 degrees, 100 degrees, 120 degrees, 140 degrees, and 160 degrees based on the angle reference line. The angle reference line is a straight line LIN. The width of one bin of the histogram corresponds to 20 degrees, and the number of pixels (count value) in the cell 244 having a gradient direction of 0 degrees or more and less than 10 degrees and 170 degrees or more and less than 180 degrees is set in the 0 degree bin. The number of pixels (count value) in the cell 244 having a gradient direction of 10 degrees or more and less than 30 degrees is set in the 20 degree bin. Similarly, the count values ​​of the 40 degree, 60 degree, 80 degree, 100 degree, 120 degree, 140 degree, and 160 degree bins are also set. Since the number of bins in the histogram 242 is 9, the blood vessel running direction of the analysis point 242 is defined as one of 9 different directions. Note that the resolution of the blood vessel running direction can be improved by narrowing the bin width and increasing the number of bins.

[0046] The count values ​​in each bin (the vertical axis of histogram 242H) are normalized to generate histogram 242H for analysis point 242 shown in FIG.

[0047] Next, the image processing unit 182 identifies the blood vessel running direction of the analysis point from the histogram 242H. Specifically, the bin with the smallest count value, 60 degrees in the example shown in FIG. 9, is identified, and the gradient direction of the identified bin, 60 degrees, is identified as the blood vessel running direction of the analysis point 242. The gradient direction with the smallest count is determined to be the blood vessel running direction for the following reason. The luminance gradient is small in the blood vessel running direction, while the luminance gradient is large in other directions (for example, there is a large difference in luminance between blood vessels and non-blood vessels). Therefore, when a histogram of the luminance gradient of each pixel is created, the count value of the bin for the blood vessel running direction is small.

[0048] Similarly, a cell 248 is set for analysis point 246, and a histogram 246H is created. The bin with the smallest count value of 160 degrees is identified among the bins in histogram 246H. Therefore, the blood vessel running direction of analysis point 246 is identified as 160 degrees. The histogram 242H and the histogram 246H are examples of the "first histogram" and the "second histogram" of the technique of the present disclosure.

[0049] By performing the above process for all analysis points in all the first and second regions, the blood vessel running direction at each analysis point set in the choroidal blood vessel image is specified. That is, as shown in FIG. 10, a histogram for each analysis point is obtained. In FIG. 10, the histograms in the second region below the line LIN are displayed in a different order. In FIG. 10, the histogram corresponding to analysis point U1 is histogram U1H, and the histogram corresponding to the corresponding analysis point D1 is histogram D1H. The order of the histograms in the first and second regions is the same (the histograms in the second region are arranged in the same order as in the first region).

[0050] In step 226, the image processing unit 182 stores the following data: the position of the macula M, the position of the optic disc ONH, the rotation angle obtained by rotating the choroidal blood vessel image so that the line LIN is horizontal, the positions (XY coordinates) of the (L) analysis points, combination information (combination of the numbers of the first and second regional analysis points) of analysis points that are linearly symmetric with respect to the line LIN, the blood vessel running direction at each analysis point, and the histogram of each analysis point are stored in the memory 164.

[0051] Next, the analysis process of symmetry of blood vessel running direction in step 212 in Fig. 5 will be described with reference to Fig. 7. In step 232 in Fig. 7, the image processor 182 reads out each analysis point on the top and bottom (first and second regions) and the blood vessel running direction at that point. Specifically, the image processor 182 reads out each analysis point and the blood vessel running direction at that point for each pair of analysis points that are line symmetric with respect to the line LIN.

[0052] In step 234, the image processor 182 calculates a value indicating asymmetry for each pair of analysis points that are linearly symmetric with respect to the line LIN. The value indicating asymmetry is the difference in the direction of blood vessels running, and this difference is found from the histograms of each analysis point in each pair. The difference Δh in the frequency of each bin in the histograms of the pair is found, and Δh is squared. Then, the Δh of each bin is calculated. 2 The sum of ΣΔh 2 It is calculated by ΣΔh 2 If is large, the shapes of the histograms will be significantly different, and the asymmetry will be large; if is small, the shapes of the histograms will be similar, and the asymmetry will be small. The histograms of each analysis point in each set are examples of the "first blood vessel running direction" and the "second blood vessel running direction" of the technology of the present disclosure. The value indicating the asymmetry is not limited to the sum of squared errors of the histograms of each analysis point of each set. A representative angle may be determined from the histograms of each analysis point of each set, and the absolute difference between the representative angles may be calculated.

[0053] In step 236, the image processor 182 detects pairs of asymmetric analysis points. Specifically, when the value indicating the asymmetry of each pair is equal to or greater than a threshold, the image processor 182 detects the pair as an asymmetric analysis point. The threshold is a constant value set in advance, but may be the overall average value of the values ​​indicating the asymmetry of each pair. FIG. 10 is a diagram showing the analysis result of step 236. The analysis point U1 of the first region (upper region 274) and the analysis point D1 of the second region (lower region 272) are in a line-symmetric relationship and form a pair, and similarly, the analysis point U11 and the analysis point D11 are in a line-symmetric relationship and form a pair. As a result of the analysis in step 234, these pairs are determined to have a value indicating asymmetry equal to or greater than a threshold, and are identified as pairs having asymmetric analysis points. The arrow UA1 indicates the blood vessel running direction of the analysis point U1, and points in the direction of 160 degrees. Similarly, the arrow DA1 indicates the blood vessel running direction of the analysis point D1, and points in the direction of 60 degrees. The arrow UA11 indicates the blood vessel running direction of the analysis point U11, and points in the direction of 160 degrees. Similarly, the arrow DA11 indicates the blood vessel running direction of the analysis point D11, and points in the direction of 40 degrees.

[0054] In step 238, the image processing unit 182 stores the following data in the memory 164. That is, the image processing unit 182 stores in the memory 164, for each pair, a value indicating asymmetry, a flag indicating whether the value indicating asymmetry is equal to or greater than a threshold value (whether or not it is asymmetric), and the angle of the blood vessel running direction of the analysis point for each pair.

[0055] Next, the display screen of the choroidal vessel analysis mode will be described. The memory 164 of the management server 140 stores data for creating the display screen of the choroidal vessel analysis mode described below, or content data to be displayed on the display screen.

[0056] Specifically, the data is as follows. The ophthalmologic apparatus 110 transmits image data of fundus images (first fundus image (red-colored fundus image) and second fundus image (green-colored fundus image)) to the management server 140, and the management server 140 has image data of the fundus images (first fundus image (red-colored fundus image) and second fundus image (green-colored fundus image)). The management server 140 has image data of a choroidal blood vessel image (see FIG. 8A). The management server 140 has the position of the macula M, the position of the optic disc ONH, the rotation angle obtained by rotating the choroidal blood vessel image so that the straight line LIN becomes horizontal, each position of the analysis points (L pieces), a set of analysis points that are line-symmetric with respect to the straight line LIN, and an angle indicating a histogram and a running direction, which are feature quantities of each analysis point. The management server 140 has a value indicating asymmetry for the set of analysis points and a flag indicating whether the value indicating asymmetry is equal to or greater than a threshold (whether it is asymmetric).

[0057] When the fundus of the patient is photographed, the patient's personal information is input to the ophthalmologic device 110. The personal information includes the patient's ID, name, age, eyesight, etc. When the fundus of the patient is photographed, information indicating whether the eye whose fundus is to be photographed is the right eye or the left eye is also input. When the fundus of the patient is photographed, the photographing date and time is also input. The ophthalmologic device 110 transmits the personal information, the right eye / left eye information, and the photographing date and time data to the management server 140. The management server 140 has the personal information, the right eye / left eye information, and the photographing date and time data. The management server 140 has the axial length data.

[0058] As described above, the management server 140 has data for creating the display screen of the choroidal vessel analysis mode.

[0059] When an ophthalmologist diagnoses a patient, the ophthalmologist performs the diagnosis while viewing the display screen of the choroidal blood vessel analysis mode displayed on the image viewer 150. In this case, the ophthalmologist transmits a display request of the choroidal blood vessel analysis mode screen to the management server 140 through a menu screen (not shown) via the image viewer 150. Upon receiving the request, the display control unit 184 of the management server 140 creates a display screen of the choroidal blood vessel analysis mode using the content data of the specified patient ID, and the processing unit 186 transmits image data of the display screen to the image viewer 150. It should be noted that processing unit 186 is an example of the "output unit" of the technology of the present disclosure. The image viewer 150, which has received the data of the display screen in the choroidal vessel analysis mode, displays the display screen 300 in the choroidal vessel analysis mode shown in FIG. 11 on the display 172 based on the data of the display screen in the choroidal vessel analysis mode.

[0060] Here, we will explain the display screen 300 in the choroidal vessel analysis mode shown in Fig. 11. As shown in Fig. 11, the display screen 300 in the choroidal vessel analysis mode has a personal information display field 302 that displays the personal information of the patient, an image display field 320, and a choroidal analysis tool display field 330.

[0061] The personal information display field 302 has a patient ID display field 304, a patient name display field 306, an age display field 308, an axial length display field 310, a visual acuity display field 312, and a patient selection icon 314. Each piece of information is displayed in the patient ID display field 304, the patient name display field 306, the age display field 308, the axial length display field 310, and the visual acuity display field 312. When the patient selection icon 314 is clicked, a list of patients is displayed on the display 172 of the image viewer 150, and the user (such as an ophthalmologist) is allowed to select a patient to be analyzed.

[0062] The image display field 320 has photographing date display fields 322N1 to 322N3, a right eye information display field 324R, a left eye information display field 324L, an RG image display field 326, a choroidal blood vessel image display field 328, and an information display field 342. The RG image is an image obtained by combining the first fundus image (R color fundus image) and the second fundus image (G color fundus image) at a predetermined ratio (for example, 1:1) of the size of each pixel value.

[0063] The choroid analysis tool display field 330 displays icons for selecting a plurality of choroid analyses, including a vortex vein location icon 332, a symmetry icon 334, a vessel diameter icon 336, a vortex vein-macula / optical disk icon 338, and a choroid analysis report icon 340. The vortex vein location icon 332 indicates that the vortex vein location is to be displayed. The symmetry icon 334 indicates that the symmetry of the analysis points is to be displayed. The vessel diameter icon 336 indicates that the analysis results related to the diameter of the choroidal vessels are to be displayed. The vortex vein-macula / optic disc icon 338 indicates that the analysis results of the analysis of the positions between the vortex vein, the macula, and the optic disc are to be displayed. The choroid analysis report icon 340 indicates that the choroid analysis report is to be displayed.

[0064] Icons and buttons for instructing to generate an image, which will be described later, are displayed on a display screen of the image viewer 150, which will be described later. When a user of the viewer 150 (such as an ophthalmologist) clicks on an icon or the like, an instruction signal corresponding to the clicked icon or the like is transmitted from the image viewer 150 to the management server 140. Upon receiving the instruction signal from the image viewer 150, the management server 140 generates an image corresponding to the instruction signal and transmits image data of the generated image to the image viewer 150. Upon receiving the image data from the management server 140, the image viewer 150 displays an image on the display 172 based on the received image data. The display screen generation process in the management server 140 is performed by a display screen generation program running on the CPU 162.

[0065] Figure 11 shows a screen in which the shooting date display field 322N1 is clicked, and the RG image and choroidal vessel image are displayed when the fundus of the right eye of a patient identified by patient ID: 123456 (icon 324R is lit) is photographed on January 1, 2016.

[0066] When the symmetry icon 334 in the choroid analysis tool display field 330 in Fig. 11 is clicked, the display screen is changed to one displaying the analysis points shown in Fig. 12. As shown in Fig. 12, the image viewer 150 displays each of the above-mentioned pairs of analysis points as a point on the choroidal blood vessel image displayed in the choroidal blood vessel image display field 328. Note that the image viewer 150 is not limited to displaying each of the above-mentioned pairs of analysis points as a point, and may display arrows UA1, UA11, DA1, and DA11 (see Fig. 10) indicating the blood vessel running direction that visualizes the characteristics of each analysis point, or may display a mark such as an ellipse instead of or together with the arrow. An asymmetric histogram display icon 346 and an asymmetric color display icon 348 are provided in the image display field 320 of the display screen of FIG.

[0067] When the asymmetry histogram display icon 346 in the image display field 320 of the display screen in Fig. 12 is clicked, a screen showing the asymmetry is displayed. Specifically, as shown in Fig. 13, the image viewer 150 displays a value indicating asymmetry (ΣΔh 2A set of analysis points U1, D1, U11, and D11 for which the difference between the analysis points U1, D1, U11, and D11 is equal to or greater than a predetermined value is highlighted, for example, by adding a frame. The frames are the same color for the same set, but different colors from other sets. For example, the frames surrounding the analysis points U1 and D1 are both colored a first color (e.g., red), and the frames surrounding the analysis points U11 and D11 are both colored a second color (e.g., orange). Furthermore, the image viewer 150 displays arrows UA1, DA1, UA11, and DA11 along the angles of 60 degrees, 160 degrees, 50 degrees, and 150 degrees of the blood vessel running direction of each of the analysis points U1, D1, U11, and D11 of the set on the choroidal blood vessel image. The arrows UA1, UA11, and the arrows UDA1 and DA11 are examples of the "first index" ("first arrow") and "second index" (second arrow) of the technology of the present disclosure.

[0068] Moreover, the image viewer 150 displays a histogram of each analysis point in the histogram display field 350 instead of the RG image display field 326, and displays a value indicating asymmetry (ΣΔh 2 ) is a predetermined value or more, the histogram pair is highlighted. The information display field 342 displays the analysis point numbers of the asymmetric pair.

[0069] 13, analysis points U1 and D1, and analysis points U11 and D11 are defined as asymmetric pairs. Therefore, in the choroidal blood vessel image display field 328, the image viewer 150 displays frames of the same color for the same pairs and frames of a different color from other pairs, such as frames surrounding analysis points U1 and D1 both being a first color (e.g., red) and frames surrounding analysis points U11 and D11 both being a second color (e.g., orange).

[0070] In the histogram display field 350, the image viewer 150 displays frames of the same color for the same set and frames of different colors for other sets, such as a first color (e.g., red) for the frames surrounding the histogram U1H of analysis point U1 and the histogram D1H of analysis point D1, and a second color (e.g., orange) for the frames surrounding the histogram U11H of analysis point U11 and the histogram D11H of analysis point D11. The color of the frame surrounding the analysis point in the choroidal blood vessel image display field 328 and the color of the frame surrounding the histogram in the histogram display field 350 are the same color if the analysis point numbers are the same, improving visibility.

[0071] Image viewer 150 displays in information display area 342 that analysis points U1 and D1, and analysis points U11 and D11 are asymmetric pairs, specifically, the text "U1 and D1 are asymmetric" and "U11 and D11 are asymmetric."

[0072] When the asymmetry color display icon 348 in the image display field 320 of the display screen of Fig. 12 is clicked, the display screen shown in Fig. 14 may be displayed instead of the display screen of Fig. 13. In each of the display screens of Fig. 13 and Fig. 14, a histogram display field 350 is displayed in Fig. 13, and in Fig. 14, instead of the histogram display field 350, a value indicating the asymmetry (ΣΔh 2 ) is displayed in a color map display field 360. Only the color map display field 360 will be described below.

[0073] First, the memory 164 of the image viewer 150 stores a value indicating asymmetry (ΣΔh 2 ) are stored in advance in correspondence with colors. For example, the value (ΣΔh 2 The larger the size of the analysis point, the darker the color. In the color map display field 360, rectangular regions are defined according to the number and positions of each analysis point.

[0074] The image viewer 150 calculates the asymmetry value (ΣΔh 2 ) and the asymmetry value (ΣΔh 2) and a color determined in advance according to the size of the asymmetry value (ΣΔh 2 ) to display the color corresponding to its size.

[0075] Furthermore, the image viewer 150 displays frames of the same color but different from the other pairs in the rectangular regions corresponding to pairs of asymmetric analysis points in the histogram display field 360. For example, a frame of a first color (e.g., red) is displayed in the rectangular regions RU1 and RD1 corresponding to the analysis points U1 and D1, and a frame of a second color (e.g., orange) is displayed in the rectangular regions RU11 and RD11 corresponding to the analysis points U11 and D11.

[0076] As described above, in this embodiment, the choroidal blood vessel image is analyzed, the asymmetry of the set of analysis points that are linearly symmetrical with respect to the straight line LIN connecting the macula M and the optic disc is analyzed, and the set of asymmetric analysis points is highlighted. Therefore, the asymmetry of the running direction of the choroidal blood vessels can be grasped. Furthermore, by visualizing the asymmetry of the running direction of the choroidal blood vessels, it is possible to assist ophthalmologists in diagnosing the fundus. In addition, the SLO unit uses a wide-angle optical system to obtain ultra-wide-angle UWF-SLO images with an angle range of more than 200 degrees from the center of the eye. Using UWF-SLO images, it is possible to analyze the symmetry of a wide range, including the peripheral part of the fundus.

[0077] Next, various modified examples of the technique of the present disclosure will be described. <First Modification> In the above embodiment, the choroidal blood vessel image is divided into a first region and a second region by a straight line LIN connecting the macula and the optic disc, and the analysis points are arranged in the first region and the second region at positions that are linearly symmetrical with respect to the straight line LIN. The technology of the present disclosure is not limited to this. For example, the choroidal blood vessel image is divided into an ear region and a nose region by a line (orthogonal line) that is perpendicular to the line LIN with the center between the macula and the optic disc as a reference. Then, the analysis points may be arranged at positions that are linearly symmetrical with respect to the orthogonal line. Furthermore, the choroidal blood vessel image may be divided by a line (crossing line) that intersects the straight line LIN at a predetermined angle, for example, 45 degrees or 135 degrees with the center between the macula and the optic disc as a reference, and the analysis points may be arranged at positions that are linearly symmetrical with respect to the crossing line.

[0078] <Second Modification> In the above embodiment, the choroidal blood vessel running direction of each analysis point is generated. The technology of the present disclosure is not limited to this. For example, the three-dimensional position of each pixel of the choroidal blood vessel image may be specified, and the choroidal blood vessel running direction may be calculated as a direction in three-dimensional space. The three-dimensional position and the direction in three-dimensional space are calculated using OCT volume data obtained using an OCT (Optical Coherence Tomography) unit provided in the ophthalmic device 110 (not shown).

[0079] <Third modified example> In the above embodiment, the management server 140 executes the image processing program shown in Fig. 5 in advance, but the technology of the present disclosure is not limited to this. When the symmetry icon 334 shown in Fig. 11 is clicked, the image viewer 150 transmits an image processing command to the management server 140. In response to this, the management server 140 may execute the image processing program of Fig. 5.

[0080] <Fourth modified example> In the above embodiment, an example has been described in which a fundus image with an internal light irradiation angle of about 200 degrees is acquired by the ophthalmic apparatus 110. The technology of the present disclosure is not limited to this, and the technology of the present disclosure may be applied to a fundus image captured by an ophthalmic apparatus with an internal irradiation angle of 100 degrees or less, or a montage image in which multiple fundus images are combined.

[0081] <Fifth modified example> In the above embodiment, a fundus image is captured by an ophthalmic apparatus 110 equipped with an SLO imaging unit, but the technology of the present disclosure may also be applied to a fundus image captured by a fundus camera capable of capturing images of choroidal blood vessels, or to an image obtained by OCT angiography.

[0082] <Sixth Modification> In the above embodiment, asymmetry is analyzed from the running direction of choroidal blood vessels, but the technology of the present disclosure may be applied so as to analyze asymmetry from the running direction of retinal blood vessels.

[0083] <Seventh Variation> In the above embodiment, the management server 140 executes the image processing program. However, the technology of the present disclosure is not limited to this. For example, the ophthalmic apparatus 110 or the image viewer 150 may execute the image processing program.

[0084] <Eighth Modification> In the above embodiment, the ophthalmic system 100 including the ophthalmic device 110, the axial length measuring device 120, the management server 140, and the image viewer 150 has been described as an example, but the technology of the present disclosure is not limited thereto. For example, as a first example, the axial length measuring device 120 may be omitted, and the ophthalmic device 110 may further have the function of the axial length measuring device 120. Also, as a second example, the ophthalmic device 110 may further have at least one of the functions of the management server 140 and the image viewer 150. For example, when the ophthalmic device 110 has the function of the management server 140, the management server 140 can be omitted. In this case, the image processing program is executed by the ophthalmic device 110 or the image viewer 150. Also, when the ophthalmic device 110 has the function of the image viewer 150, the image viewer 150 can be omitted. As a third example, the management server 140 may be omitted, and the image viewer 150 may execute the function of the management server 140.

[0085] <Other Modifications> The data processing described in the above embodiment is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed without departing from the spirit of the invention. In the above embodiment, the data processing is realized by a software configuration using a computer, but the technology of the present disclosure is not limited to this. For example, instead of a software configuration using a computer, the data processing may be performed only by a hardware configuration such as a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC). A part of the data processing may be performed by a software configuration, and the remaining processing may be performed by a hardware configuration.

Claims

[Claim 1] setting a first analysis point and a second analysis point symmetrical with respect to a reference line on a fundus image; determining a first blood vessel running direction at the first analysis point and a second blood vessel running direction at the second analysis point; Analyzing asymmetry between the first blood vessel direction and the second blood vessel direction; An image processing method comprising:

Images