Ophthalmological image observation program

Abstract

To provide an ophthalmologic image observation program which can excellently perform positioning between images captured by a plurality of imaging methods.SOLUTION: An ophthalmologic image observation program is executed by a processor of an ophthalmologic computer to acquire a first inspection image being a front image of a subject eye, a first observation image corresponding to the first inspection image, a second inspection image of the subject eye and a second observation image corresponding to the second inspection image by first and second inspection image acquisition steps, perform first positioning processing between the first inspection image and the first observation image, perform second positioning processing between the first observation image and the second observation image, and associate the positions between images of the first inspection image and the second inspection image on the basis of the displacement in inspection data as the displacement in the first positioning processing and the second positioning processing, and the displacement between the inspection data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an ophthalmic image viewing program. [Background technology]

[0002] 2. Description of the Related Art In the field of ophthalmology, in order to diagnose and observe an eye to be examined from multiple angles, the eye to be examined is photographed using a plurality of different photographing methods.

[0003] There is a positional deviation between a plurality of images captured using different methods.

[0004] Patent Document 1 discloses a method for associating the positions of multiple types of fluorescent angiograms taken using different imaging methods, by utilizing the fact that the positional shift between the fluorescent angiograms taken using each imaging method and the infrared fundus images acquired simultaneously represents the positional shift between the fluorescent angiograms taken using each imaging method.

[0005] In addition, a method has been disclosed in which, when aligning a fluorescent contrast image and a color fundus image, the positional misalignment between an infrared fundus image acquired simultaneously with the fluorescent contrast image and a fundus image showing one of the color components that make up the color fundus image is regarded as a positional misalignment between the fluorescent contrast image and the color fundus image, and the positions between the images are associated. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-059400 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in Patent Document 1, when the positional shift between infrared images can be used as the intended positional shift between images, it is dependent on the shooting method, and therefore it can be difficult to accurately associate the positions between images.

[0008] In contrast, the present disclosure is based on the problems of the conventional technology, and its technical objective is to provide an ophthalmologic image observation program that can effectively align images captured using multiple imaging methods. [Means for solving the problem]

[0009] Book Disclosure No. 1 The ophthalmologic image observation program according to the embodiment is executed by a processor of an ophthalmologic computer to perform a first test image acquisition step of acquiring a first test image, which is a front image of the subject's eye, and a first observation image, which is a front observation image corresponding to the first test image; a second test image acquisition step of acquiring a second test image of the subject's eye, which is captured by a method different from that of the first test image, and a second observation image, which is a front observation image corresponding to the second test image; a first alignment step of performing a first alignment process between the first test image and the first observation image; and a second alignment step of performing a second alignment process between the first observation image and the second observation image. The method has an ophthalmological computer execute an alignment step and an association step of associating positions between the first test image and the second test image based on a displacement within the test data, which is a displacement obtained based on the first alignment process, and a displacement between the test data, which is a displacement obtained based on the second alignment process, and further, in the first alignment process, the displacement between the first test image and the first observation image is obtained by local matching, and in the second alignment process, the displacement between the first observation image and the second observation image is obtained by global matching and local matching. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to satisfactorily align images captured using a plurality of imaging methods. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an outline of a registration technique according to an embodiment. [Figure 2] 1 is a diagram showing the appearance of an ophthalmologic image observation system according to an embodiment; [Figure 3] 1 is a diagram illustrating an optical system of an ophthalmologic photographing apparatus according to an embodiment. [Figure 4] 10 is a table showing the correspondence between each light receiving element of the photographing optical system and the photographing method. [Figure 5] 10 is a flowchart showing the flow of a photographing operation according to an embodiment. [Figure 6] FIG. 10 is a diagram showing an example of a display screen. [Figure 7] 10 is a flowchart showing the flow of a display operation according to an embodiment. [Figure 8] 1 is a flowchart showing a flow of alignment; [Figure 9] FIG. 10 is a diagram showing an example of a display screen according to a modification example. DETAILED DESCRIPTION OF THE INVENTION

[0012] "overview" An embodiment of the present disclosure will be described below. An ophthalmologic image observation system according to the embodiment will be described. The ophthalmologic image observation system is used to capture an image of an eye to be examined and display an examination image that is the result of the capture.

[0013] In an embodiment, the ophthalmic image observation system includes an ophthalmic computer and an ophthalmic imaging device. The ophthalmic imaging device and the ophthalmic computer may be separate entities or may be integrated. If they are integrated, the imaging operation may be controlled by the ophthalmic computer. If they are separate entities, the ophthalmic computer and the ophthalmic imaging device are connected via a wire or wirelessly and can communicate with each other. This allows test images to be sent and received from the ophthalmic imaging device to the ophthalmic computer.

[0014] <Ophthalmic imaging device> The ophthalmologic imaging apparatus has at least an imaging optical system (not shown). In the imaging optical system, the subject's eye is irradiated with light, and then the light returning from the subject's eye is received by a light receiving element. In this embodiment, a front image of the subject's eye is generated based on a signal from the light receiving element. Here, in this embodiment, the first test image and the front observation image may be generated based on a signal from the same light receiving element.

[0015] The imaging optical system may be a scanning optical system. In this case, the imaging optical system may include at least a scanning means (optical scanner) that scans light from a light source on the tissue of the subject's eye and a light-receiving element that receives light returning from the tissue. The imaging optical system may capture a front image of the fundus or a front image of the anterior segment as the first test image and the front observation image.

[0016] The first test image is a front image of the subject's eye. The first test image is captured based on a capture trigger. The first test image may be, for example, a reflection image based on multiple colors of visible light (hereinafter referred to as a color image), or a reflection image based on a single color of visible light. The first test image may also be a fluorescence image. The fluorescence image may be a contrast fluorescence image (e.g., an FA image or an IA image), or an autofluorescence image (FAF image).

[0017] Furthermore, the front image of the subject's eye acquired as the first test image may be a retro image based on retro photography. The retro image is captured by selectively receiving scattered light scattered before and after the light focusing position on the observation surface of the subject's eye using an eccentric fundus diaphragm. The retro image depicts tissues before and after the light focusing position. For details of the retro photography technique in SLO, see, for example, Japanese Patent Application Laid-Open No. 2009-95632.

[0018] The front observation image is a front image of the subject's eye captured by infrared light. For example, imaging conditions in the imaging optical system, such as focus and brightness, may be adjusted based on the front observation image acquired as needed. The imaging optical system irradiates the subject's eye with infrared light and captures the front observation image based on the returned infrared light.

[0019] The ophthalmologic imaging apparatus may further capture a second test image of the subject's eye. The second test image may be captured using a different method from the first test image. The second test image may be captured using a photographic optical system. In this case, the second test image may be generated based on a light-receiving signal from the same light-receiving element as the first test image. Alternatively, the second test image may be generated based on a light-receiving signal from a light-receiving element different from that used for the first test image. Alternatively, the second test image may be captured using a second photographic optical system different from the photographic optical system. The second photographic optical system may be, for example, an OCT optical system that captures OCT data of the subject's eye. In this case, the second test image may be an OCT image, such as a tomographic image based on a B-scan, a three-dimensional image based on a volume scan, or an OCT front image. The OCT image may be a functional OCT image, or a more specific example, a motion contrast image (MC image). The motion contrast image is generated based on motion contrast OCT data. For example, a pseudo angiographic image (OCT angiographic image) can be obtained as the motion contrast image.

[0020] <Ophthalmological image observation program> The ophthalmological image observation program according to the embodiment is stored in a non-volatile memory that can be read by a processor of an ophthalmological computer. When the ophthalmological image observation program is executed by the processor, the following steps are executed by the ophthalmological computer.

[0021] <First inspection image acquisition step> In the first test image acquisition step, the first test image and the first observation image are acquired based on signals from the same light-receiving element. The first observation image is a front observation image corresponding to the first test image. That is, the position on the subject's eye at which the first test image was captured is (approximately) identified by the first observation image. However, since the first test image and the first observation image are based on signals from the same light-receiving element, the capture timings of the first test image and the first observation image may differ. Therefore, there is some positional misalignment between the first test image and the first observation image.

[0022] <Second inspection image acquisition step> In the second test image acquiring step, a second test image and a second observational image are acquired. The second test image and the second observational image may be captured simultaneously (at the same timing). The second observational image is a frontal observational image corresponding to the second test image. That is, the position on the subject's eye where the second test image was captured is (approximately) identified by the second observational image.

[0023] <First alignment step> In this embodiment, the first inspection image and the first observational image are based on signals from the same light-receiving element, so the first inspection image and the first observational image may not be captured simultaneously. In this case, even if the first inspection image and the first observational image are captured at close timing, a slight positional misalignment may occur. In contrast, in this embodiment, a first alignment process is performed between the first inspection image and the first observational image in the first alignment step. As a result of the first alignment process, the displacement between the first inspection image and the first observational image may be acquired as a displacement within the inspection data.

[0024] In the first alignment process, the displacement between the first inspection image and the first observation image may be obtained by local matching. In this case, it is desirable to reduce the possibility of a large positional deviation occurring by capturing the first inspection image and the first observation image consecutively. In this embodiment, the local matching determines the displacement for each local region between the images, and at least one of the images is transformed based on the displacement for each local region. Further details will be described in the examples.

[0025] In particular, when the imaging optical system is a scanning optical system, various distortion correction processes may be additionally performed because different distortions (also referred to as local positional misalignment) may occur between the first test image and the first observation image due to the influence of fixation eye movement during imaging. In this case, for example, the distortion correction process disclosed in Japanese Patent Laid-Open No. 2020-162926 by the present applicant may be used.

[0026] Whether or not it is necessary to calculate the displacement within the inspection data related to the second inspection image depends on the method for capturing the second inspection image. For example, if the wavelengths of the illumination light used for capturing the second inspection image and the second observation image are different, resulting in different magnifications due to chromatic aberration of magnification, the displacement between the second inspection image and the second observation image may be calculated. Furthermore, even if the second inspection image and the second observation image were not captured simultaneously, such as when the second inspection image and the second observation image were captured based on light-receiving signals from the same light-receiving element, the displacement between the second inspection image and the second observation image may also be calculated.

[0027] <Second alignment step> In this embodiment, the first and second inspection images are assumed to be acquired at relatively different times. In response to this, a second alignment process is performed between the first and second observational images in the second alignment step. As a result of the second alignment process, the displacement between the first and second observational images may be acquired as a displacement between the inspection data (considered as a displacement between the first and second inspection images). In the second alignment process, global matching is performed, followed by local matching. In local matching, multiple local regions are defined for each image to be processed (for example, divided into multiple patches), and corresponding points between the images are found for each local region. The deviation between corresponding points for each local region represents the displacement for each local region. As a matching method for finding corresponding points for local regions, for example, any of various correlation-based methods and template matching can be used. Further details will be described in the examples.

[0028] <Association step> As shown in FIG. 1 , in this embodiment, in the associating step, the positions of the first and second inspection images are associated based on at least the displacement within the inspection data (the displacement between the first inspection image and the first observational image) and the displacement between the inspection data (the displacement between the first observational image and the second observational image). For example, the displacement between the images is corrected. As a result, the first and second inspection images are accurately associated with each other. Note that the associating step does not need to be a process independent of the first and second alignment steps, and may be performed together with the first and second alignment steps.

[0029] In the registration method of this embodiment, when aligning a first test image and a second test image captured by different methods, the first test image and the second test image are not directly matched, but the first test image and the first observational image (and, in some cases, the second test image and the second observational image) and the first observational image and the second observational image are matched, which makes it easier to achieve more robust registration between images captured by different methods. Furthermore, the registration method of this embodiment makes it easier to reduce the number of modality combinations for which matching processing needs to be prepared.

[0030] As shown in FIG. 1, in this embodiment, three or more test images captured by different imaging methods may be aligned. For example, in the case of aligning three test images, at least a third test image captured by a third imaging method different from both the first and second imaging methods and a third observation image associated with the third test image may be acquired (third test image acquisition step). In this case, in the second alignment step, for example, the displacements of the observation images relative to the template may be calculated as the displacements between the test data, and the three test images may be aligned based on the displacements within the test data and the displacements between the test data. The template may be any of the first to third observation images, or another observation image may be used as the template. Furthermore, alignment can be performed in a similar manner when there are four or more test images.

[0031] <Switching display> A display control step may further be executed in which the first test image and the second test image, with their positions correlated, are switched and displayed in the same display area. At this time, the displacement between the images may be corrected. Since the areas on the subject's eye in each area of ​​the first test image and the second test image match each other, the examiner can easily grasp the correspondence between the areas in the first test image and the second test image. Furthermore, the difference between the first test image and the second test image can be easily grasped as a change in visual information, making it easy to compare them.

[0032] <Parallel display> Furthermore, a display control step may be further executed in which the first and second test images are displayed side by side with their positions associated with each other.

[0033] "Example" Next, an embodiment will be described. An ophthalmic apparatus 1 according to the embodiment is one example of an ophthalmic image observation system in which an ophthalmic imaging apparatus and a computer are integrated. The ophthalmic apparatus 1 acquires various images of the subject's eye by photographing them itself. However, this is not necessarily limited to this, and images of the subject's eye photographed by another apparatus may be acquired via a network or the like.

[0034] As an example, the ophthalmologic apparatus 1 according to the embodiment is a scanning fundus imaging apparatus (e.g., a scanning light ophthalmoscope (SLO)). The ophthalmologic apparatus 1 may be an apparatus integrated with other ophthalmologic apparatuses such as an optical coherence tomography (OCT) apparatus, a perimeter, etc.

[0035] The ophthalmologic apparatus 1 captures a frontal fundus image using a plurality of imaging methods. At least one of the wavelength band of light used, the substance used for contrast imaging (in the case of contrast imaging), and the imaging principle may be different between the imaging methods.

[0036] For example, in this embodiment, the ophthalmologic apparatus 1 will be described as capturing IR images, color images, and fluorescent images as frontal fundus images. IR images are reflection images of infrared light. In this embodiment, IR images are used at least as fundus observation images. IR images are also used to align fundus images captured using at least one of different capture methods and capture times. Color images are captured by color photography. Color images are reflection images of multiple colors of visible light. Fluorescent images are acquired by fluorescent photography. Fluorescent images may be captured using either indocyanine green angiography (IA) or fluorescein angiography (FA). Two types of fluorescent angiographic images may be captured simultaneously by simultaneously injecting two types of contrast agents intravenously. Hereinafter, images based on indocyanine green fluorescence will be referred to as IA images, and images based on fluorescein fluorescence will be referred to as FA images.

[0037] <Device configuration> First, a schematic configuration of the ophthalmic apparatus 1 will be described with reference to Fig. 2. The ophthalmic apparatus 1 has at least an imaging unit 100 and a control unit 70. Furthermore, the ophthalmic apparatus 1 is connected to a monitor 80 and an input interface 75.

[0038] The photographing unit 100 has at least photographing optical systems 10 and 20 (described later) in order to photograph the subject's eye E (see FIG. 3). The photographing unit 100 may also have a driving unit or the like for adjusting the position (alignment) between the subject's eye and the device.

[0039] The control unit 70 is an ophthalmologic computer that executes the photographing operation via the photographing unit 100, the display control of the photographed images, etc. The control unit 70 includes an arithmetic control unit 70 and a memory 71.

[0040] The arithmetic control unit 70 is a processor in the ophthalmologic apparatus 1. The arithmetic control unit 70 may be configured with a CPU, RAM, ROM, etc. The arithmetic control unit 70 performs operations for capturing a front fundus image, display control of the front fundus image, and other processes based on a program prepared in advance (at least an ophthalmologic image observation program according to the embodiment).

[0041] The arithmetic and control unit 70 is connected to a memory 71, an operation unit 75, a monitor 80, an imaging unit 100, and the like via a data bus or the like.

[0042] In this embodiment, the memory 71 is a non-volatile storage device. For example, a hard disk or a flash memory can be used as the memory 71. In this embodiment, various programs are pre-stored in the memory 71. The memory 71 may be rewritable. In this case, various frontal fundus images acquired from the ophthalmologic apparatus 1 may be stored in the memory 71, as will be described in the following examples.

[0043] The operation unit 75 is an input interface for the computer 1. The arithmetic and control unit 70 receives a signal corresponding to an operation input to the operation unit 75.

[0044] For convenience, the various operations in the following description are based on the assumption that a mouse is used as the operation unit 75. However, the operation unit 75 may be various other devices. For example, at least one of a touchpad and a keyboard may be used as the operation unit 75.

[0045] In this embodiment, the monitor 80 is used as a display unit (display device) for images captured by the ophthalmologic apparatus 1. The monitor 80 may be, for example, a general-purpose monitor or a monitor built into the apparatus.

[0046] <Photographing optical system> The photographing optical systems 10 and 20 according to the embodiment will be described with reference to Fig. 3. The photographing optical systems 10 and 20 are used to capture a front image of the fundus of the subject's eye E. As shown in Fig. 3, the photographing optical systems 10 and 20 include at least an illumination optical system 10 and a light-receiving optical system 20.

[0047] The irradiation optical system 10 includes at least a light source 11 and an objective lens system 17. As shown in FIG. 3, the irradiation optical system 10 may further include a light emitting unit 11, a perforated mirror 13, and a scanning unit 16.

[0048] In this embodiment, the light emitting unit 11 emits laser light. In this embodiment, light from the light emitting unit 11 is used as illumination light to be irradiated onto the fundus Er from the irradiation optical system 10. The light emitting unit 11 may include, for example, a laser diode (LD) and a superluminescent diode (SLD). Although a detailed description of the structure will be omitted, the light emitting unit 11 emits light in at least one wavelength range. In this embodiment, light of multiple colors is emitted from the light emitting unit 11 simultaneously or selectively. For example, in this embodiment, light of four colors is emitted from the light emitting unit 11: three colors in the visible range, namely blue, green, and red, and one color in the infrared range. The light of each color can be emitted simultaneously or alternately.

[0049] Light from the light emitting unit 11 passes through an opening formed in the perforated mirror 13 and travels toward the scanning unit 16. The light reflected by the scanning unit 16 passes through an objective lens system 17 and is then irradiated onto the fundus Er of the subject's eye E.

[0050] The scanning unit 16 (also referred to as an "optical scanner") is a unit for scanning the fundus with light emitted from the light emitting unit 11. In the following description, unless otherwise specified, the scanning unit 16 includes two optical scanners with different scanning directions. That is, the scanning unit 16 includes an optical scanner 16a for main scanning (for scanning in the X direction, for example) and an optical scanner 16b for sub-scanning (for scanning in the Y direction, for example). In the following description, the optical scanner 16a for main scanning is a resonant scanner, and the optical scanner 16b for sub-scanning is a galvanometer mirror. However, other optical scanners may be used for each of the optical scanners 16a and 16b. For example, other reflective mirrors (such as a galvanometer mirror, a polygon mirror, a resonant scanner, or a MEMS), as well as an acousto-optical element (AOM) that changes the traveling (deflection) direction of light, may be used for each of the optical scanners 16a and 16b.

[0051] The objective lens system 17 is an objective optical system in the ophthalmic apparatus 1. The objective lens system 17 is used to guide light scanned by the scanning unit 16 to the fundus Er. To this end, the objective lens system 17 forms a pivot point P around which the light passing through the scanning unit 16 is pivoted. The pivot point P is located on the optical axis L1 of the irradiation optical system 10 and is formed at a position optically conjugate to the scanning unit 16 with respect to the objective lens system 17. Note that, in this disclosure, "conjugate" does not necessarily mean a perfect conjugate relationship but also includes "approximate conjugate." In other words, the term "conjugate" in this disclosure also includes a position that is shifted from a perfect conjugate position within an allowable range in relation to the intended use of the fundus image (e.g., observation, analysis, etc.). Note that the objective optical system in the ophthalmic imaging apparatus 1 is not limited to a lens system but may be a mirror system, a combination of a lens system and a mirror system, or other optical system.

[0052] The light that has passed through the objective lens system 17 is rotated around the pivot point P in accordance with the operation of the scanning unit 16. As a result of the alignment, the pivot point P is positioned at the anterior segment of the subject's eye E, and the light is scanned two-dimensionally on the fundus Er. As a result, the light from the light emitting unit 11 is reflected and scattered by the fundus Er, or excites fluorescent substances present in the fundus, causing fluorescence from the fundus. This light (i.e., reflected and scattered light, fluorescence, etc.) is emitted from the pupil as returned light.

[0053] Next, the light receiving optical system 20 will be described. The light receiving optical system 20 has a plurality of light receiving elements. For example, as shown in FIG. 3, the light receiving optical system 20 may have a plurality of light receiving elements 25, 27, and 29. In this case, light from the fundus Er is received by the light receiving elements 25, 27, and 29.

[0054] 3, the light-receiving optical system 20 in this embodiment may share the components arranged from the objective lens system 17 to the perforated mirror 13 with the illumination optical system 10. In this case, light from the fundus is guided back along the optical path of the illumination optical system 10 to the perforated mirror 13. The perforated mirror 13 guides the light from the fundus Er to an independent optical path of the light-receiving optical system 20 while removing at least a portion of noise light caused by reflections on the cornea of ​​the subject's eye and on the optical system inside the device (for example, the lens surface of the objective lens system, etc.).

[0055] The optical path splitting member that splits the light into the irradiation optical system 10 and the light receiving optical system 20 is not limited to the perforated mirror 13, and other beam splitters may be used.

[0056] The light receiving optical system 20 of this embodiment has a lens 21, a pinhole plate 23, and a light separating section (light separating unit) 30 in the reflected light path of the perforated mirror 13. In this embodiment, the light separating section (light separating unit) 30 is used as a spectroscopic section.

[0057] The pinhole plate 23 is disposed on a fundus conjugate plane and functions as a confocal diaphragm in the SLO 1. That is, when the diopter is properly corrected by the diopter adjustment unit 40, light from the fundus Er that has passed through the lens 21 is focused at the opening of the pinhole plate 23. The pinhole plate 23 removes light from positions other than the focal point (or focal plane) of the fundus Er, and the remaining light (light from the focal point) is mainly guided to the light receiving elements 25, 27, and 29.

[0058] The light separating unit 30 separates light from the fundus Er. In this embodiment, the light separating unit 30 separates the light from the fundus Er in a wavelength-selective manner. The light separating unit 30 may also serve as a light branching unit that branches the optical path of the light receiving optical system 20. For example, as shown in FIG. 3, the light separating unit 30 may include two dichroic mirrors (dichroic filters) 31 and 32 that have different optical separation characteristics (wavelength separation characteristics). The optical path of the light receiving optical system 20 is branched into three by the two dichroic mirrors 31 and 32. One of the light receiving elements 25, 27, and 29 is disposed at the end of each branched optical path.

[0059] For example, the light separating unit 30 separates the wavelengths of light from the fundus Er and causes the three light receiving elements 25, 27, and 29 to receive light of different wavelength ranges. For example, light of three colors, blue, green, and red, is received by the light receiving elements 25, 27, and 29, one color at a time. In this case, a color image can be obtained from the light receiving results of the light receiving elements 25, 27, and 29.

[0060] Furthermore, the light separating unit 30 causes the infrared light used in infrared photography to be received by at least one of the light receiving elements 25, 27, and 29. In this embodiment, for example, the fluorescence used in fluorescence photography and the infrared light used in infrared photography are received by different light receiving elements.

[0061] Here, with reference to FIG. 4, the spectral characteristics of the light separating unit 30 in this embodiment will be described. Light in the wavelength range reflected by the dichroic mirror 31 is guided to the optical path on the light receiving element 25 side. The dichroic mirror 31 reflects at least light in the red wavelength range and light in the infrared range (first infrared range), and transmits light in other wavelength ranges, which are received by the light receiving element 25. As shown in FIG. 4, as a result, light in the red wavelength range and light in the infrared range (first infrared range) are received by the light receiving element 25. The red wavelength range is used, for example, for color photography. The first infrared range is used, for example, for IA photography. That is, in this embodiment, the first infrared range is set so as to include an infrared component, which is the fluorescent wavelength of indocyanine green.

[0062] Light in a wavelength range that passes through dichroic mirror 31 and is reflected by dichroic mirror 32 is guided to the optical path on the light receiving element 27 side. In this embodiment, dichroic mirror 32 reflects at least light in the green wavelength range. As a result, as shown in FIG. 4, light in the green wavelength range is received by light receiving element 27. The green wavelength range is used for color photography. The green wavelength range may also be used for FA photography. That is, in this embodiment, the green wavelength range may be set so as to include the green component, which is the fluorescent wavelength of fluorescein.

[0063] Light in wavelength ranges that pass through the two dichroic mirrors 31 and 32 is guided to the optical path on the light receiving element 29 side. In this embodiment, at least light in the blue wavelength range and light in the infrared range are transmitted. The infrared light that passes through each dichroic mirror 31 and 32 has a wavelength range on the shorter wavelength side compared to the infrared light reflected by the dichroic mirror 31. The light that passes through each dichroic mirror 31 and 32 is received by the light receiving element 29. As shown in FIG. 4, as a result, the light receiving element 29 receives light in the blue wavelength range and light in a second infrared range that is shorter in wavelength than the first infrared range. The blue wavelength range is used, for example, for color photography. The second infrared range is used, for example, for capturing IR images.

[0064] <Operation description> Next, the operation of the ophthalmologic apparatus 1 will be described with reference to FIGS.

[0065] <Shooting operation> First, the operation of the ophthalmologic apparatus 1 for capturing a frontal fundus image will be described with reference to the flowchart of FIG.

[0066] In the ophthalmologic apparatus 1, first, the positional relationship between the subject's eye and the apparatus is adjusted (S1). Then, acquisition of a fundus observation image is started (S2). That is, the ophthalmologic apparatus 1 continuously emits infrared light from the light emitting unit 11, and generates an IR image as a fundus observation image as needed based on a signal from the light receiving element 29. Various adjustments such as focus adjustment are performed based on the IR image acquired as needed (S3).

[0067] Next, an imaging method is selected (S4). The imaging method may be selected according to a predetermined imaging sequence. Alternatively, an imaging method selection operation may be accepted each time. In this embodiment, the imaging operation differs depending on the imaging method. For ease of explanation, the imaging methods will be broadly divided into two types: color imaging and fluorescent imaging (IA imaging or FA imaging).

[0068] After the imaging method is selected, an imaging trigger is input to perform imaging.

[0069] <Color photography> In the ophthalmologic apparatus 1 of this embodiment, the light receiving element 29 is used both when capturing color images and when capturing IR images, so IR images and color images cannot be captured simultaneously. In contrast, in this embodiment, when a capture trigger is input while color capture is selected, the light emitting unit 11 stops emitting infrared light, and the acquisition of the infrared observation image is interrupted. Next, the light emitting unit 11 simultaneously emits three colors of blue, green, and red, and a color image is captured based on the light receiving signals simultaneously output from the light receiving elements 25, 27, and 29 (S5). After the capture, the acquisition of the infrared observation image is resumed. In this way, color images and IR images are captured continuously.

[0070] After the photographing is completed, the color image and one IR image acquired consecutively to the color image are stored in the memory 71 in association with each other (S6).

[0071] <Fluorescence photography> In the ophthalmologic apparatus 1 of this embodiment, different light-receiving elements are used when capturing fluorescence images (IA images and FA images) and when capturing IR images, so fluorescence images and IR images can be captured simultaneously. In this embodiment, when a capture trigger is input with fluorescence photography selected, infrared light from the light emitting unit 11 and excitation light corresponding to the selected type of fluorescence photography are simultaneously emitted. As a result, fluorescence images and IR images are captured simultaneously (S7).

[0072] For example, in the early stage of contrast enhancement, multiple frames of fluorescent images may be captured based on an imaging trigger. At this time, an IR image may be captured simultaneously with each frame of fluorescent image. In the early stage of contrast enhancement, a still image of the fluorescent image may be captured in one shot for each imaging trigger based on an imaging trigger.

[0073] After the image capturing is completed, the control unit 70 stores the simultaneously captured fluorescent image and IR image in the storage unit 71 in association with each other (S8).

[0074] As described above, color fundus images, fluorescent images, and IR images associated with each are captured by the imaging device 100 based on an ophthalmologic imaging trigger and stored in the memory 71 (in other words, acquired by the imaging device 100, which is the ophthalmologic computer in this embodiment). In this case, various frontal fundus images are associated with identification information indicating the imaging method and the date and time of imaging, and then stored in the memory 71. Furthermore, fluorescent angiographic images and IR images acquired simultaneously with the fluorescent angiographic images may be further associated with information indicating the measurement time of the contrast timer at the time of imaging.

[0075] <Display operation> In this embodiment, a plurality of front fundus images photographed by different photographing methods are comparatively displayed in the manner shown in Fig. 6. That is, according to Fig. 6, a plurality of front fundus images photographed by different photographing methods are alternatively displayed in one display area.

[0076] Here, the operation of displaying a frontal fundus image will be described with reference to the flowchart of FIG.

[0077] First, from among a plurality of frontal fundus images acquired as a result of photography, a plurality of images to be used for comparison display are registered (S11). At this time, it is preferable that the photographing range on the subject's eye is approximately the same between each registered image.

[0078] The registered image may be selected from a plurality of frontal fundus images acquired as a result of photography, based on a registration operation by the examiner. For convenience, in this embodiment, a case will be described in which a color fundus image and a fluorescent angiographic image are selected as targets for comparison display.

[0079] <Detection of displacement information between registered images> In order to perform the comparative display, displacement information between the registered images is acquired by the control unit 70 (S12). The detailed acquisition operation is shown in FIG. 8. In this embodiment, displacement information between the test data and displacement information within the test data are acquired. In this embodiment, the displacement information between the registered images is obtained by combining the displacement information between the test data and the displacement information within the test data.

[0080] <Method for obtaining deviations between inspection data> The displacement information between test data represents the displacement (positional deviation) between each registered image and the associated IR image. Since the timing of capturing each registered image may be significantly different, the capturing positions may be shifted due to factors such as eye rotation or fixation deviation.

[0081] <Global Matching> In contrast, in this embodiment, global matching is first performed between each registered image and its corresponding IR image (S21). In this embodiment, the displacement of the entire image between the images is determined by global matching. In this embodiment, the amount of translation is determined as the displacement of the entire image. However, this is not necessarily limited to this, and for example, at least one of rotation, distortion, and magnification may also be determined. As a matching method for determining the displacement of the entire image, for example, any of various methods using correlations using the entire image (e.g., orientation correlation, phase-only correlation, normalized cross-correlation, etc.), template matching, etc. can be used. Furthermore, the matching process does not necessarily have to be performed using the entire image; the displacement in a specific region, such as the center of the image, may be considered to be the displacement of the entire image and may be determined.

[0082] <Local matching> In the inter-registration of inspection data in this embodiment, local matching is performed after global matching (S22). In local matching, a plurality of local regions are set for each image to be processed (for example, divided into a plurality of patches), and corresponding points between the images are found for each local region. In this embodiment, when setting the local regions, the displacement of the entire image found by global matching is taken into consideration. For example, the translation of the entire image may be corrected, and then local regions may be set at corresponding positions on each image. The deviation of corresponding points for each local region represents the displacement for each local region. As a matching method for finding corresponding points for local regions, for example, any of the methods using various correlations similar to those described above, template matching, etc. can be used.

[0083] <Affine transformation> Next, based on the information of the corresponding points found by local matching, an affine transformation (or projective transformation) of the entire image is found (S23). Furthermore, in this embodiment, the validity of the affine transformation is evaluated (S24). Specifically, one of the two images to be processed is retransformed using the calculated affine transformation, and it is determined whether the deviation between the corresponding points after the affine transformation in one image and the corresponding points in the other image is within an acceptable range. If it is within the acceptable range, the transformation is considered successful. If it is outside the acceptable range, the transformation is considered to have failed. Images for which the transformation has failed may be excluded from the display target in the comparison display described below. Furthermore, during the comparison display, identification information indicating that the transformation has failed may be displayed together with the image for which the transformation has failed. Furthermore, since deviations outside the acceptable range are considered to represent image distortion due to eye movement during scanning, various types of distortion correction may be additionally performed.

[0084] <Method for obtaining displacement in inspection data> The displacement information in the inspection data represents the displacement (positional deviation) between a selected registered image and the IR image associated with that registered image.

[0085] As described above, the color fundus image and the IR image associated with the color fundus image are based on signals from the same light-receiving element, but are captured at different times. Therefore, there is a positional shift between the two images. However, since the two images were captured consecutively, there is unlikely to be a significant difference in the capture positions. Therefore, the above-described global matching is not necessarily required to obtain displacement information within the test data.

[0086] However, local matching is performed to correct positional deviations due to slight differences in the timing of image capture (S22). In this embodiment, when the registered image is a color fundus image, the above-mentioned local matching is performed between the color fundus image and an IR image acquired in association with the color fundus image, and displacement information within the test data is acquired based on the local matching. In this embodiment, an affine transformation of the entire image is acquired as displacement information within the test data (S23). The validity of the affine transformation may also be evaluated (S24).

[0087] For example, the displacement at a plurality of corresponding points (or corresponding regions) between the first inspection image and the first observational image may be determined for each corresponding point (or each corresponding region).

[0088] Furthermore, in this embodiment, as described above, the fluorescence image and the IR image acquired in association with the fluorescence image are captured simultaneously, so no positional misalignment occurs between the two images (i.e., the displacement is zero). Therefore, in this case, the process of acquiring displacement information based on matching between the fluorescence image and the IR image acquired in association with the fluorescence image is omitted.

[0089] <Correction Process (Association Process in the Example)> Next, the displacement between the registered images is corrected based on the displacement information between the test data and the displacement information within the test data (S13), thereby matching the relationship between each position on each registered image and each position on the fundus.

[0090] <Comparison display of registered images> The registered image is displayed for comparison (S14) in the manner shown in Fig. 6. For convenience, the screen at this time is referred to as a comparison screen.

[0091] First, one of the registered images (for convenience, referred to as the first image) is alternatively displayed in a predetermined display area. Then, based on a switching instruction, a newly selected registered image (for convenience, referred to as the second image) is switched from the first image and displayed in the comparison display area 301 (for example, FIG. 6(a) ⇔ FIG. 6(b)). At this time, a part or the whole of the second image is displayed in the comparison display area 301, which is adjusted so that the area on the subject's eye displayed in the predetermined comparison display area matches the first image. The adjustment is performed based on the displacement information acquired by the processing of S13 (for example, the displacement information for each of the first image and the second image).

[0092] In this embodiment, when the image displayed in the predetermined display area is switched from a first image to a second image, the second image is adjusted so that the area on the subject's eye displayed in the predetermined display area matches the first image immediately before the switch. Therefore, the examiner can easily grasp the correspondence between the areas in the first and second images. Furthermore, the differences between the first and second images can be easily grasped as changes in visual information. As a result, the switchable display of this embodiment allows the examiner to easily compare multiple registered images.

[0093] "Example of transformation" The present disclosure has been described above based on the embodiments and examples, but the present disclosure is not necessarily limited to these and various modifications are possible.

[0094] For example, in the above embodiment, a case has been described in which multiple frontal fundus images captured using different photographing methods are switched and displayed in the same display area, but they may also be displayed side by side in separate display areas, as shown in Figure 9.

[0095] In this case, the multiple frontal fundus images displayed in parallel may be displayed with the displacement corrected based on the displacement information between the test data and the displacement information within the test data. Also, as shown in Fig. 9, an index may be displayed at a corresponding position on each image. In Fig. 9, cursors 302 and 402 are displayed as indexes. The relative positions of the cursors in each display area are matched. By correcting the displacement between the images, each cursor can indicate the same fundus region. [Explanation of symbols]

[0096] 1 Ophthalmology equipment 70 Control Unit

Claims

1. By being executed by the processor of the ophthalmic computer, a first test image acquiring step of acquiring a first test image which is a front image of the subject's eye and a first observation image which is a front observation image corresponding to the first test image; a second test image acquiring step of acquiring a second test image of the subject's eye, the second test image being captured by a method different from that of the first test image, and a second observation image, which is a front observation image corresponding to the second test image; a first alignment step of performing a first alignment process between the first inspection image and the first observation image; a second alignment step of performing a second alignment process between the first observational image and the second observational image; and causing an ophthalmic computer to execute an associating step of associating positions between the first test image and the second test image based on a displacement within the test data, which is a displacement obtained based on the first alignment process, and a displacement between the test data, which is a displacement obtained based on the second alignment process; and the first alignment process includes acquiring a displacement between the first inspection image and the first observation image by local matching; an ophthalmologic image observation program, wherein in the second alignment process, a displacement between the first observational image and the second observational image is obtained by global matching and local matching.

2. the first alignment step further includes performing a first alignment process between the second inspection image and the second observation image; 2. An ophthalmological image observation program as described in claim 1, wherein the association step associates the positions between the first test image and the second test image based on a displacement in the first test data between the first test image and the first observation image, which is a displacement obtained based on the first alignment process, a displacement in the second test data between the second test image and the second observation image, and a displacement between the test data, which is a displacement obtained based on the second alignment process.

3. 3. The ophthalmologic image observation program according to claim 1, wherein the first test image and the first observation image are acquired based on a signal from the same light receiving element.

4. An ophthalmological image observation program as described in any one of claims 1 to 3, further comprising a display control step of switching between and displaying the first test image and the second test image in the same display area based on a switching instruction, the first test image and the second test image being associated in position based on the displacement within the test data and the displacement between the test data.

5. An ophthalmological image observation program as described in any one of claims 1 to 3, further comprising a display control step of displaying the first test image and the second test image in parallel, with the positions of the images correlated based on the displacement within the test data and the displacement between the test data.

Images