Ophthalmic imaging device and ophthalmic image processing device

Abstract

To improve performance of an ophthalmologic imaging device for scanning a three-dimensional region of an eye to be examined with slit light.SOLUTION: A first image collection part 1010 of an ophthalmologic imaging device 1000 scans a three-dimensional region of an eye to be examined with slit light, and collects a series of Scheimpflug images. A second image collection part 1020 repeatedly images the eye to be examined in parallel to the scanning of the three-dimensional region by the first image collection part, and collects a series of time-series images. A first image analysis part 1030 analyzes the series of time series images collected by the second image collection part 1020, and obtains time-series displacement of the slit light in the scanning by the first image collection part 1010. An image interpolation part 1040 executes interpolation for the series of Scheimpflug images collected by the first image collection part 1010 on the basis of the time-series displacement of the slit light obtained by the first image analysis part 1030.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an ophthalmic photographing apparatus and an ophthalmic image processing apparatus. [Background technology]

[0002] Diagnostic imaging plays an important role in the field of ophthalmology. Various types of ophthalmic imaging devices are used for diagnostic imaging. Ophthalmic imaging devices include slit lamp microscopes, fundus cameras, scanning laser ophthalmoscopes (SLO), and optical coherence tomography (OCT). In addition, various examination and measurement devices such as refractometers, keratometers, tonometers, specular microscopes, wavefront analyzers, and microperimeters are also equipped with the function of photographing the anterior segment and fundus.

[0003] Among these various ophthalmic imaging devices, one of the most widely and frequently used devices is the slit lamp microscope. The slit lamp microscope is an ophthalmic device for illuminating the subject's eye with slit light and observing and photographing the illuminated cross section from the side with a microscope (see, for example, Patent Documents 1 and 2). In addition, a slit lamp microscope is also known that can scan a three-dimensional area of ​​the subject's eye at a relatively high speed by using an optical system configured to satisfy the Scheimpflug condition (see, for example, Patent Document 3). In addition to the slit lamp microscope, there is also a rolling shutter camera as an imaging method for scanning an object with slit light. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-159073 A [Patent Document 2] JP 2016-179004 A [Patent Document 3] JP 2019-213733 A Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present invention is to improve the performance of an ophthalmic imaging apparatus that scans a three-dimensional area of ​​a subject's eye with a slit light. [Means for solving the problem]

[0006] The ophthalmic imaging device of the first exemplary embodiment includes a first image acquisition unit, a second image acquisition unit, a first image analysis unit, and an image interpolation unit. The first image acquisition unit is configured to scan a three-dimensional region of the subject's eye with a slit light to acquire a series of Scheimpflug images. The second image acquisition unit is configured to repeatedly capture images of the subject's eye in parallel with the scanning of the three-dimensional region by the first image acquisition unit to acquire a series of time-series images. The first image analysis unit is configured to analyze the series of time-series images to determine a time-series displacement of the slit light in the scan. The image interpolation unit is configured to perform interpolation of the series of Scheimpflug images based on the time-series displacement of the slit light.

[0007] A second exemplary aspect is an ophthalmic imaging device of the first aspect, wherein a first image analysis unit analyzes a series of time-series images to detect a series of images of the slit light, and determines a time-series displacement of the slit light based on the series of images of the slit light.

[0008] A third exemplary aspect is the ophthalmologic imaging device of the first or second aspect, in which the scanning of the three-dimensional region by the first image acquisition unit and the repeated imaging of the subject's eye by the second image acquisition unit are performed in synchronization with each other, and the image interpolation unit performs interpolation of the series of Scheimpflug images based on a correspondence relationship between the series of Scheimpflug images and the series of time-series images based on the synchronization between the scanning and the repeated imaging.

[0009] A fourth exemplary aspect is an ophthalmologic imaging device according to any one of the first to third aspects, in which the first image acquisition unit includes a first imaging system and a second imaging system that respectively image the subject's eye from different directions during scanning. The first imaging system acquires a first Scheimpflug image group, and the second imaging system acquires a second Scheimpflug image group. The series of Scheimpflug images includes the first Scheimpflug image group and the second Scheimpflug image group.

[0010] A fifth exemplary aspect is the ophthalmologic imaging apparatus of the fourth aspect, in which the imaging by the first imaging system and the imaging by the second imaging system are performed synchronously with each other.

[0011] A sixth exemplary aspect is the ophthalmologic imaging device of the fifth aspect, further including an image selection unit. The image selection unit is configured to select a new series of Scheimpflug images corresponding to the scan from the first Scheimpflug image group and the second Scheimpflug image group based on a correspondence relationship between the first Scheimpflug image group and the second Scheimpflug image group based on synchronization between the imaging by the first imaging system and the imaging by the second imaging system. The image interpolation unit performs interpolation of the new series of Scheimpflug images.

[0012] A seventh exemplary aspect is the ophthalmologic imaging device of the sixth aspect, in which the first image acquisition unit further includes an illumination system that projects a slit light onto a three-dimensional region. The optical axis of the first imaging system and the optical axis of the second imaging system are arranged to be inclined in opposite directions to each other with respect to the optical axis of the illumination system. The image selection unit selects a new series of Scheimpflug images by selecting a plurality of Scheimpflug images that do not include a corneal reflection artifact from the first Scheimpflug image group and the second Scheimpflug image group based on a correspondence relationship between the first Scheimpflug image group and the second Scheimpflug image group.

[0013] An eighth exemplary aspect is an ophthalmic imaging device of any of the first to seventh aspects, wherein the image interpolation unit interpolates a series of Scheimpflug images using an optical flow that represents, as a vector, the displacement of the test eye depicted in the series of Scheimpflug images.

[0014] A ninth exemplary aspect is the ophthalmologic imaging device according to any one of the first to eighth aspects, in which the image interpolation unit uses a pre-constructed inference model to interpolate a series of Scheimpflug images. The inference model is constructed in advance by applying machine learning using training data including at least Scheimpflug images of the eye to a neural network. The inference model is configured to receive input of two or more Scheimpflug images each representing two or more different positions of the same eye, and to output an interpolated image obtained by spatially interpolating the two or more Scheimpflug images.

[0015] A tenth exemplary aspect is the ophthalmologic imaging device according to any one of the first to ninth aspects, in which the image interpolation unit includes a first interpolation number determination unit. The first interpolation number determination unit is configured to determine the number of interpolations based on at least one of the operation information of the first image collection unit and the time-series displacement of the slit light. The image interpolation unit interpolates a series of Scheimpflug images using the interpolated number of images.

[0016] An eleventh exemplary aspect is the ophthalmologic imaging device according to any one of the first to tenth aspects, in which the first image analysis unit further analyzes the series of time-series images collected by the second image collection unit to obtain a time-series displacement of the subject's eye. The image interpolation unit includes a second interpolation number determination unit. The second interpolation number determination unit is configured to determine the number of interpolations based on at least one of the operation information of the first image collection unit and the time-series displacement of the subject's eye. The image interpolation unit interpolates a series of Scheimpflug images using the interpolated number of images.

[0017] The ophthalmic imaging device of a twelfth exemplary embodiment includes a first image acquisition unit, a second image acquisition unit, a second image analysis unit, and an identifier assignment unit. The first image acquisition unit is configured to scan a three-dimensional region of the subject's eye with a slit light to acquire a series of Scheimpflug images. The second image acquisition unit repeatedly images the subject's eye in parallel with the scanning of the three-dimensional region by the first image acquisition unit to acquire a series of time-series images. The second image analysis unit is configured to analyze the series of time-series images to detect a series of images of the slit light. The identifier assignment unit is configured to assign, to each of the series of Scheimpflug images, an identifier indicating the ocular tissue to which the slit light was projected when the Scheimpflug image was acquired, based on the series of images of the slit light detected from the series of time-series images.

[0018] A thirteenth exemplary aspect is the ophthalmologic imaging device of the twelfth aspect, in which the scanning of the three-dimensional region by the first image acquisition unit and the repeated imaging of the subject's eye by the second image acquisition unit are performed synchronously with each other, and the identifier assignment unit assigns the identifier based on a correspondence relationship between the series of Scheimpflug images and the series of time-series images based on the synchronization between the scanning and the repeated imaging.

[0019] A fourteenth exemplary aspect is an ophthalmic photography device of the thirteenth aspect, in which the identifier assignment unit assigns an identifier to each of the series of time-series images based on a series of images of slit light, and assigns an identifier to each of the series of Scheimpflug images based on the series of identifiers assigned to each of the series of time-series images and the correspondence relationship.

[0020] A fifteenth exemplary aspect is the ophthalmologic imaging device according to any one of the twelfth to fourteenth aspects, in which the second image analysis unit further analyzes the series of Scheimpflug images to detect a series of images of the slit light, and the identifier assignment unit further assigns an identifier based on the series of images of the slit light detected from the series of Scheimpflug images.

[0021] A sixteenth exemplary aspect is any one of the twelfth to fifteenth ophthalmologic imaging devices, in which the second image analysis unit further analyzes the series of Scheimpflug images to detect an image of a site of interest in the subject's eye. The identifier assignment unit further assigns an identifier indicating the site of interest to any one of the series of time-series images collected by the second image collection unit based on a detection result of the image of the site of interest from the series of Scheimpflug images.

[0022] A seventeenth exemplary aspect is the ophthalmologic imaging apparatus according to any one of the twelfth to sixteenth aspects, wherein the identifier assigning unit assigns an identifier to a partial region of the Scheimpflug image corresponding to an eye tissue.

[0023] An eighteenth exemplary aspect is an ophthalmologic imaging device according to any one of the twelfth to seventeenth aspects, in which the first image acquisition unit includes a first imaging system and a second imaging system that respectively image the subject's eye from different directions during scanning. The first imaging system acquires a first Scheimpflug image group, and the second imaging system acquires a second Scheimpflug image group. The series of Scheimpflug images includes the first Scheimpflug image group and the second Scheimpflug image group.

[0024] A nineteenth exemplary aspect is the ophthalmologic imaging apparatus of the eighteenth aspect, in which the imaging by the first imaging system and the imaging by the second imaging system are performed synchronously with each other.

[0025] A twentieth exemplary aspect is the ophthalmologic imaging device of the nineteenth aspect, further including an image selection unit. The image selection unit is configured to select a new series of Scheimpflug images corresponding to the scan from the first Scheimpflug image group and the second Scheimpflug image group based on a correspondence relationship between the first Scheimpflug image group and the second Scheimpflug image group based on synchronization between the imaging by the first imaging system and the imaging by the second imaging system. The identifier assignment unit assigns an identifier to each of the new series of Scheimpflug images.

[0026] A twenty-first exemplary aspect is the ophthalmic imaging device of the twentieth aspect, in which the first image acquisition unit further includes an illumination system. The illumination system is configured to project a slit light onto a three-dimensional region. The optical axis of the first imaging system and the optical axis of the second imaging system are arranged to be inclined in opposite directions to each other with respect to the optical axis of the illumination system. The image selection unit selects a new series of Scheimpflug images by selecting a plurality of Scheimpflug images that do not include a corneal reflection artifact from the first Scheimpflug image group and the second Scheimpflug image group based on a correspondence relationship between the first Scheimpflug image group and the second Scheimpflug image group.

[0027] A 22nd exemplary aspect is an ophthalmic imaging device of any of the 1st to 21st aspects, in which the first image collection unit scans a three-dimensional area by translating the slit light in a direction perpendicular to the longitudinal direction of the slit light.

[0028] A twenty-third exemplary aspect is the ophthalmologic imaging apparatus of the twenty-second aspect, in which the longitudinal direction of the slit light substantially coincides with the body axis direction of the subject.

[0029] A twenty-fourth exemplary aspect is the ophthalmologic imaging apparatus of the twenty-third aspect, wherein the dimension of the slit light in the longitudinal direction is equal to or greater than the corneal diameter in the body axis direction, and the distance of translation of the slit light is equal to or greater than the corneal diameter in the direction perpendicular to the body axis direction.

[0030] The ophthalmologic image processing device of the twenty-fifth exemplary embodiment includes a reception unit, a first image analysis unit, and an image interpolation unit. The reception unit receives a series of images acquired by scanning a three-dimensional region of the subject's eye with a scanning light, and a series of time-series images collected by repeatedly photographing the subject's eye in parallel with the scanning of the three-dimensional region. The first image analysis unit is configured to analyze the series of time-series images to obtain a time-series displacement of the scanning light. The image interpolation unit is configured to perform an interpolation of the series of images based on the time-series displacement of the scanning light.

[0031] The ophthalmic image processing device of the 26th exemplary embodiment includes a receiving unit, a second image analysis unit, and an identifier assignment unit. The receiving unit receives a series of images acquired by scanning a three-dimensional region of the subject's eye with the scanning light, and a series of time-series images collected by repeatedly photographing the subject's eye in parallel with the scanning of the three-dimensional region. The second image analysis unit is configured to analyze the series of time-series images to detect a series of images of the scanning light. The identifier assignment unit is configured to assign an identifier indicating the eye tissue to which the scanning light was projected when the image was acquired to each of the series of images acquired by scanning the three-dimensional region with the scanning light, based on the series of images of the scanning light detected from the series of time-series images. Effect of the Invention

[0032] According to the exemplary embodiment, it is possible to improve the performance of an ophthalmologic imaging apparatus that scans a three-dimensional area of ​​a subject's eye with a slit light. [Brief description of the drawings]

[0033] [Figure 1] 1 is a schematic diagram illustrating a configuration of an ophthalmic imaging apparatus according to an exemplary embodiment. [Diagram 2] 1 is a schematic diagram for explaining a process executed by an ophthalmologic imaging apparatus according to an exemplary embodiment. [Diagram 3] 1 is a flow diagram illustrating a process executed by an ophthalmologic imaging apparatus according to an exemplary embodiment. [Figure 4] 1 is a schematic diagram illustrating a configuration of an ophthalmic imaging apparatus according to an exemplary embodiment. [Diagram 5] 1 is a schematic diagram illustrating a configuration of an ophthalmic imaging apparatus according to an exemplary embodiment. [Figure 6A] 1 is a schematic diagram illustrating a configuration of an ophthalmic imaging apparatus according to an exemplary embodiment. [Figure 6B] 1 is a schematic diagram illustrating a configuration of an ophthalmic imaging apparatus according to an exemplary embodiment. [Figure 7] 1 is a schematic diagram illustrating a configuration of an ophthalmic imaging apparatus according to an exemplary embodiment. [Figure 8]1 is a schematic diagram illustrating a configuration of an ophthalmic imaging apparatus according to an exemplary embodiment. [Figure 9] 1 is a flow diagram illustrating a usage pattern of an ophthalmologic imaging apparatus according to an exemplary embodiment. [Figure 10] 1 is a schematic diagram illustrating a configuration of an ophthalmic imaging apparatus according to an exemplary embodiment. [Figure 11] 1 is a flow diagram illustrating a usage pattern of an ophthalmologic imaging apparatus according to an exemplary embodiment. [Figure 12] 1 is a schematic diagram illustrating a configuration of an ophthalmic imaging apparatus according to an exemplary embodiment. [Figure 13] 1 is a schematic diagram illustrating a configuration of an ophthalmic imaging apparatus according to an exemplary embodiment. [Figure 14A] 1 is a schematic diagram for explaining the operation of an ophthalmologic imaging apparatus according to an exemplary embodiment. [Figure 14B] 1 is a schematic diagram for explaining the operation of an ophthalmologic imaging apparatus according to an exemplary embodiment. [Figure 15] 1 is a schematic diagram for explaining the operation of an ophthalmologic imaging apparatus according to an exemplary embodiment. [Figure 16] 1 is a schematic diagram illustrating a configuration of an ophthalmic imaging apparatus according to an exemplary embodiment. [Figure 17] 1 is a schematic diagram illustrating a configuration of an ophthalmic imaging apparatus according to an exemplary embodiment. [Figure 18] FIG. 1 is a data flow diagram showing a usage pattern of an ophthalmologic imaging apparatus according to an exemplary embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] Some exemplary aspects of the embodiment will be described in detail with reference to the drawings. Any known technology can be combined with the embodiment. For example, any known technology in the technical field, such as any matter disclosed in the documents cited in this specification, can be combined with the embodiment. For example, all contents disclosed in Patent Document 3 (JP Patent Publication No. 2019-213733) by Topcon Corporation, which is included in the applicant of this application, are incorporated by reference into this disclosure. The same applies to other documents (patent applications, papers, etc.) related to the technology by any of the applicants of this application. In addition, any two or more of the various aspects disclosed in this specification can be at least partially combined.

[0035] At least a portion of the functionality of the elements described in this disclosure is implemented using circuitry or processing circuitry. The circuitry or processing circuitry may be a general purpose processor, a special purpose processor, an integrated circuit, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), a field programmable gate array (FPGA), or a combination of a programmable logic device (PLC) and a programmable logic device (PLC)) configured and / or programmed to perform at least a portion of the disclosed functionality. The term "circuitry", "unit", "means" or the like includes any of the above-mentioned circuitry, including a transistor and / or other circuitry, a processor, a processor array, a conventional circuitry, and any combination thereof. A processor is considered to be a processing circuitry or circuitry, including transistors and / or other circuitry. In this disclosure, a circuitry, a unit, a means, or the like is hardware that performs at least some of the disclosed functions or is hardware that is programmed to perform at least some of the disclosed functions. The hardware may be hardware disclosed herein or may be known hardware that is programmed and / or configured to perform at least some of the described functions. In the case where the hardware is a processor that can be considered to be a type of circuitry, a circuitry, a unit, a means, or the like is a combination of hardware and software, and the software is used to configure the hardware and / or the processor.

[0036] <Overview of the embodiment> The embodiments relate to a technique for processing a series of images collected by scanning a three-dimensional region of a test eye with a slit light. Some exemplary aspects of the embodiments relate to a technique for spatially interpolating a series of images collected by scanning a three-dimensional region of a test eye with a slit light (spatial image interpolation). Some other exemplary aspects relate to a technique for assigning an identifier indicating an ocular tissue to a series of images collected by scanning a three-dimensional region of a test eye with a slit light (ocular tissue identifier assignment). Some other exemplary aspects relate to both spatial image interpolation and ocular tissue identifier assignment. Scanning using a slit light may be referred to as slit scanning.

[0037] In the embodiment, an exemplary aspect in which the slit scan is applied to the anterior part of the subject's eye will be described, but the site to which the slit scan is applied is not limited to the anterior part (e.g., tissues such as the cornea, iris, anterior chamber, angle, ciliary body, Zonula gracilis, lens, nerve, blood vessel, etc.; lesion; treatment scar; artificial objects such as an intraocular lens and a minimally invasive glaucoma surgery (MIGS) device). In some exemplary aspects, the slit scan can be applied to the posterior part of the eye (e.g., tissues such as the vitreous body, retina, choroid, sclera, optic disc, lamina cribrosa, macula, nerve, blood vessel, etc.; lesion; treatment scar; artificial objects such as an artificial retina). In some exemplary aspects, the slit scan can be applied to tissues near the eyeball such as the eyelids and meibomian glands. Furthermore, in some aspects, the slit scan can be applied to a three-dimensional area including any two or all of at least a part of the anterior part of the eyeball, at least a part of the posterior part of the eyeball, and at least a part of the tissues near the eyeball.

[0038] <Spatial image interpolation> The background, motivation, and overview of the embodiment related to spatial image interpolation will be described. In a typical conventional anterior ocular slit scan, a scan is applied to a three-dimensional region including the cornea (between the temporal sclera and the nasal sclera). In this slit scan, the number of frames collected by the Scheimpflug camera is about several hundred, and the vertical dimension of each frame (Scheimpflug image) is about 1000 to 3000 pixels.

[0039] When constructing a three-dimensional image (volume image) from such collected images (collected frames), it is necessary to reduce the interval between adjacent frames (reducing the array density of collected frames) in order to provide a group of images arranged at sufficiently small intervals to the three-dimensional imaging processor. However, this results in an aspect ratio of the obtained three-dimensional image (the ratio of the dimension in the X direction to the dimension in the Y direction, which will be described later) that is significantly different from that of the actual anterior segment, making it difficult to perform actions such as observation and diagnosis, and to perform processing such as analysis.

[0040] The inventors of the present disclosure have focused on this problem and developed a spatial image interpolation technique according to an embodiment to solve this problem. In the embodiment, in order to perform 3D imaging without narrowing the interval between collected frames, a computer-generated frame is inserted between adjacent frames by applying spatial image interpolation to the collected frames. This makes it possible to obtain a series of images (series of frames) arranged at sufficiently narrow intervals, and to construct a 3D image without narrowing the interval between collected frames, thereby making it possible to obtain a 3D image with an aspect ratio according to the actual shape of the anterior segment.

[0041] The same is true for rendered images obtained from such 3D images. For example, when rendering a front image (enface image, image defined in an XY coordinate system) from a 3D image with an aspect ratio according to the actual shape of the anterior segment, the obtained front image has an aspect ratio according to the actual shape of the anterior segment. The same is true when constructing other types of rendered images, such as an angle image (slow-axis scan tomographic image).

[0042] By providing an image having an aspect ratio according to the actual shape of the anterior segment, actions such as observation and diagnosis and processes such as analysis become easier than ever before, and the quality of these actions and processes (e.g., accuracy, precision, reliability, reproducibility, etc.) is improved. Therefore, it becomes possible to improve the performance of slit scanning of a three-dimensional area of ​​the subject's eye.

[0043] In the embodiment, the number of interpolated frames between adjacent frames may or may not be constant. For example, when a first frame and a second frame are adjacent to each other and a third frame and a fourth frame are adjacent to each other, the number of frames interpolated between the first frame and the second frame and the number of frames interpolated between the third frame and the fourth frame may be equal to or different from each other.

[0044] In some aspects, the number of interpolated frames between adjacent frames can be determined. In determining the number of interpolated frames, any information that affects the adjacent frame interval (distance between adjacent frames), such as the distance the slit light moves in the slit scan, the eye movement of the subject's eye, and the shooting interval (frame rate, frame collection rate) of the Scheimpflug camera, can be referenced. This makes it possible to correct the deviation of the adjacent frame interval caused by the eye movement and the variation of the adjacent frame interval caused by the non-uniform speed of the scan.

[0045] The method of spatial image interpolation may be selected arbitrarily. In some aspects, a method of stretching an original image (e.g., an original three-dimensional image constructed only from acquired frames, and its rendered image) may be applied using the resize function of OpenCV (Open Source Computer Vision Library) or similar software. However, this method is inappropriate for expressing tissues such as the cornea (limbus) that have clear boundaries and whose position changes significantly between frames. When the inventors of the present disclosure actually tried it, they confirmed that the corneal boundary becomes discontinuous in the image after the interpolation.

[0046] Optical flow is a method for frame interpolation of moving images that obtains a vector representation of the movement of an object by analyzing the movement of feature points between frames. Optical flow methods include phase correlation, block-based methods, differential methods (e.g., Lucas-Kanade method, Horn-Schunck method, Buxton-Buxton method, Black-Jepson method, general variational methods), and discrete optimization methods.

[0047] A newer optic flow method uses a convolutional neural network (CNN), as disclosed in, for example, the following literature: Huaizu Jiang et al., "Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 9000-9008 (arXiv:1712.00080).

[0048] The inventors of the present disclosure have compared a frame interpolation method using simple linear interpolation with a frame interpolation method using CNN (comparison of frontal images), and confirmed that the latter method expresses the corneal boundary more smoothly. In some embodiments, any method of optical flow may be used to perform spatial image interpolation. In some embodiments, any neural network to which any training (machine learning) has been applied may be used to perform spatial image interpolation. In some embodiments, any method of optical flow using CNN to which any training (machine learning) has been applied may be used to perform spatial image interpolation. For example, in some embodiments, the frame interpolation method described in the above-mentioned document by Huaizu Jiang et al. may be applied to spatial image interpolation. It should be noted that the frame interpolation method described in the above-mentioned document by Huaizu Jiang et al. is different from the above-mentioned document in that it interpolates a group of frames arranged in time (a group of frames constituting a moving image), whereas the exemplary embodiment to which it is applied is at least one that interpolates a series of images arranged in space.

[0049] The above-described matters are only a part of the spatial image interpolation according to the embodiment. Various matters of the spatial image interpolation according to the embodiment will be described and technically specified in the exemplary aspects described below. Any image processing technique such as image analysis and image interpolation can be combined with the spatial image interpolation according to the embodiment. In addition, any image acquisition technique, any shooting technique, any image processing technique (e.g., pre-processing such as image correction) may be applied to the image provided to the spatial image interpolation according to the embodiment.

[0050] <About assigning eye tissue identifiers> The background, motivation, and overview of the embodiment of the ocular tissue identifier assignment will be described. A series of images collected from a three-dimensional region of the anterior eye by slit scanning and processed images thereof (three-dimensional images, rendered images, etc.) are provided for actions such as observation and diagnosis, and for processing such as analysis. In these applications, it is important to be able to recognize the type of ocular tissue depicted in each image. However, since each image collected by slit scanning represents a cross-section of the anterior eye, it may be difficult to determine (judge, recognize) which cross-section of the ocular tissue is from the image. Similarly, it may be difficult to determine a characteristic part of the subject's eye, such as the corneal apex, from each image, or to determine a characteristic position of the imaging protocol, such as the turn-around position in a reciprocating scan, from each image.

[0051] The inventors of the present disclosure have focused on this problem and developed an ocular tissue identifier assignment technology according to an embodiment to solve this problem. In the embodiment, a series of time-series images are collected in parallel with the collection of a series of images by slit scanning, and the ocular tissue depicted in each image obtained by slit scanning is determined by analyzing the corresponding time-series images. This makes it possible to assign an identifier (ocular tissue identifier) ​​indicating the ocular tissue onto which the slit light was projected when the image was acquired to each image acquired by slit scanning, and makes it possible to (e.g., easily and / or correctly) recognize the type of ocular tissue depicted in an image used in an action such as observation or diagnosis or a process such as analysis.

[0052] Any number of ocular tissue identifiers may be assigned to an image, for example, 1, 2 or more. If no ocular tissue included in the predefined set of ocular tissues is detected in an image, no information may be assigned to the image, or information indicating that the particular ocular tissue is not depicted may be assigned to the image.

[0053] In some embodiments, an image acquired by slit scanning can be analyzed to assign an ocular tissue identifier to the image. In some embodiments, an image acquired by slit scanning can be analyzed to assign an ocular tissue identifier to a time series image corresponding to the image. These embodiments are considered to be useful when dealing with ocular tissues (sites) that can only be detected from images acquired by slit scanning, or when ocular tissues are more preferably detected (e.g., with high accuracy or precision) from images acquired by slit scanning. For example, these embodiments are useful when assigning an identifier to the corneal apex, an identifier to tissues inside the eyeball, etc.

[0054] The above-described matters are only a part of the eye tissue identifier assignment according to the embodiment. Various matters of the eye tissue identifier assignment according to the embodiment will be described and technically specified in the exemplary aspects described below. The eye tissue identifier assignment according to the embodiment can be combined with any image processing technique such as image analysis or segmentation. In addition, any image collection technique, any photographing technique, any image processing technique (e.g., pre-processing such as image correction) may be applied to the image provided for the eye tissue identifier assignment according to the embodiment.

[0055] <First aspect> The first aspect provides an exemplary embodiment of an ophthalmic imaging apparatus capable of performing spatial image interpolation, and a specific example (embodiment) of the ophthalmic imaging apparatus according to this aspect will be described later.

[0056] 1 shows the configuration of an ophthalmic imaging apparatus according to this embodiment. The ophthalmic imaging apparatus 1000 includes a first image acquisition unit 1010, a second image acquisition unit 1020, a first image analysis unit 1030, and an image interpolation unit 1040.

[0057] A plurality of Scheimpflug images (including a series of Scheimpflug images collected by the first image collecting unit 1010 and a group of Scheimpflug images interpolated by the image interpolating unit 1040) obtained through spatial image interpolation by the image interpolating unit 1040 are used in various post-processing such as construction of a three-dimensional image, construction of a rendering image, image analysis, image diagnosis, etc. Elements (processor, user interface, etc.) for performing post-processing may be provided in the ophthalmic imaging apparatus 1000 or may be provided in a device different from the ophthalmic imaging apparatus 1000.

[0058] The first image acquisition unit 1010 is configured to collect a series of Scheimpflug images by scanning a three-dimensional region of the subject's eye with a slit light. The first image acquisition unit 1010 collects a series of Scheimpflug images by repeatedly capturing images while moving a projection position of the slit light relative to the three-dimensional region of the subject's eye. The first image acquisition unit 1010 has an optical system configured to satisfy the Scheimpflug condition and functions as a Scheimpflug camera.

[0059] In some exemplary embodiments, the first image acquisition unit 1010 may be configured to scan a three-dimensional region of the subject's eye by translating the slit light in a direction perpendicular to the longitudinal direction of the slit light. Such a scanning manner is different from that of a conventional anterior segment imaging device that scans the anterior segment by rotating the slit light.

[0060] Here, the longitudinal direction of the slit light is the longitudinal direction of the beam cross section of the slit light at the projection position on the subject's eye, in other words, the longitudinal direction of the image of the slit light formed on the subject's eye, and may be approximately the same as the direction along the subject's body axis (body axis direction). Also, the dimension of the slit light in the longitudinal direction may be equal to or larger than the corneal diameter in the body axis direction of the subject, and the distance of translation of the slit light may be equal to or larger than the corneal diameter in the direction perpendicular to the body axis direction of the subject.

[0061] Although the series of Scheimpflug images collected by the first image collection unit 1010 is a group of images (group of frames) collected consecutively in time, they are a group of images collected from multiple different positions in a three-dimensional region of the test eye, and therefore are a group of images distributed spatially, unlike general moving images.

[0062] The optical system of the first image collecting unit 1010 includes an illumination system that projects slit light onto a three-dimensional area of ​​the subject's eye, and a photographing system that photographs the three-dimensional area of ​​the subject's eye onto which the slit light from the illumination system is projected. The photographing system includes an image sensor. The illumination system and the photographing system satisfy the Scheimpflug condition. That is, a plane passing through the optical axis of the illumination system (a plane including the object surface), a principal plane of the photographing system, and an image capturing surface of the image sensor intersect on the same straight line. This allows the photographing system to be focused on all positions within the object surface (all positions in the direction along the optical axis of the illumination system).

[0063] The imaging system of the first image acquisition unit 1010 may include at least two imaging systems. The imaging system of the first image acquisition unit 1010 according to some exemplary embodiments includes a first imaging system and a second imaging system that respectively capture images of a three-dimensional region of the subject's eye from different directions in the slit scan. For example, when the first image acquisition unit 1010 is configured such that the longitudinal direction of the beam cross section of the slit light at the incident position on the subject's eye is the vertical direction (Y direction) and the moving direction of the slit light is the horizontal direction (left-right direction, X direction), the first imaging system and the second imaging system may be arranged such that one captures the subject's eye from a left oblique direction and the other captures the subject's eye from a right oblique direction. A series of Scheimpflug images collected by the first imaging system is called a first Scheimpflug image group, and a series of Scheimpflug images collected by the second imaging system is called a second Scheimpflug image group. The series of Scheimpflug images collected by such a first image collection unit 1010 includes a first Scheimpflug image group and a second Scheimpflug image group.

[0064] The second image acquisition unit 1020 is configured to acquire a series of time-series images by repeatedly capturing images of the subject's eye in parallel with the scanning of the three-dimensional region of the subject's eye by the first image acquisition unit 1010. That is, the series of Scheimpflug images and the series of time-series images are acquired during the same period. The series of time-series images are a plurality of images arranged in time. Each time-series image is one image. For example, the second image acquisition unit 1020 captures a moving image of the three-dimensional region of the subject's eye, and the series of time-series images are moving images (a plurality of frames arranged in time), and each time-series image is one frame.

[0065] In some exemplary embodiments, the slit scan of the three-dimensional region of the subject's eye by the first image acquisition unit 1010 and the repeated photographing of the subject's eye by the second image acquisition unit 1020 may be performed in synchronization with each other. This allows a correspondence between a series of Scheimpflug images acquired by the first image acquisition unit 1010 and a series of time-series images acquired by the second image acquisition unit 1020 to be obtained. This correspondence is a correspondence between a series of Scheimpflug images arranged spatially and a series of time-series images arranged temporally, and pairs the Scheimpflug images and the time-series images acquired substantially simultaneously. The synchronization between the first image acquisition unit 1010 and the second image acquisition unit 1020 may be realized by any known synchronization process, for example, by sending a synchronization signal to the first image acquisition unit 1010 and the second image acquisition unit 1020 from a control processor operating based on a synchronization control program.

[0066] In some other exemplary aspects, post-processing can be performed to determine a correspondence between a series of Scheimpflug images collected by the first image collecting unit 1010 and a series of time-series images collected by the second image collecting unit 1020. For example, a correspondence between a series of Scheimpflug images and a series of time-series images can be determined by comparing a predetermined processing timing of the Scheimpflug images sequentially acquired by the first image collecting unit 1010 with a predetermined processing timing of the time-series images sequentially acquired by the second image collecting unit 1020 with each other.

[0067] The predetermined processing timing for determining the correspondence between the series of Scheimpflug images and the series of time-series images may be, for example, a combination of the timing at which the Scheimpflug images are input from the first image collecting unit 1010 to a control processor (not shown) (i.e., the timing at which the control processor receives each Scheimpflug image) and the timing at which the time-series images are input from the second image collecting unit 1020 to the control processor (i.e., the timing at which the control processor receives each time-series image). As another example, the predetermined processing timing may be a combination of the timing at which the control processor transfers each Scheimpflug image to a storage device (not shown) and the timing at which the control processor transfers each time-series image to the storage device. As yet another example, the predetermined processing timing may be a combination of the timing at which each Scheimpflug image acquired by the first image collecting unit 1010 is stored in a buffer memory (or the timing at which each Scheimpflug image is read out from the buffer memory) and the timing at which each time-series image acquired by the second image collecting unit 1020 is stored in a buffer memory (or the timing at which each Scheimpflug image is read out from the buffer memory).

[0068] The second image acquisition unit 1020 according to some exemplary embodiments may be configured to capture an image of the subject's eye from the front (front direction or oblique direction). In some exemplary embodiments, when the illumination system of the first image acquisition unit 1010 is configured to project illumination light from the front direction onto the subject's eye, the optical system and the image sensor of the second image acquisition unit 1020 may be provided in an optical path branched from the optical path of the illumination system. For example, the optical system of the second image acquisition unit 1020 in this example may be disposed coaxially with the illumination system of the first image acquisition unit 1010. The second image acquisition unit 1020 in this example can acquire front images of the subject's eye as time-series images.

[0069] In some exemplary embodiments in which the second image acquisition unit 1020 is configured to capture an image of the subject's eye from an oblique direction, the imaging system of the first image acquisition unit 1010 may be configured to project illumination light from an oblique direction onto the subject's eye, and the optical system and the image sensor of the second image acquisition unit 1020 may be provided on an optical path branched from the optical path of the imaging system of the first image acquisition unit 1010. For example, the optical system of the second image acquisition unit 1020 in this example may be disposed coaxially with the imaging system of the first image acquisition unit 1010. In some other exemplary embodiments, the orientation of the optical axis of the second image acquisition unit 1020 may be different from both the orientation of the optical axis of the illumination system of the first image acquisition unit 1010 and the orientation of the optical axis of the imaging system.

[0070] The second image acquisition unit 1020 according to some exemplary embodiments may be configured to capture images of the subject's eye from two or more different directions. A specific example of the configuration and method is disclosed in Japanese Patent Application Laid-Open No. 2013-248376 by Topcon Corporation, which is the assignee of the present application. The second image acquisition unit 1020 of this example includes two anterior eye cameras arranged in different directions relative to the subject's eye. Each anterior eye camera is, for example, a video camera. The two anterior eye cameras capture moving images of the anterior eye of the subject's eye from different directions.

[0071] The ophthalmic imaging apparatus 1000 (for example, the first image analysis unit 1030) can select (pair) two frames (captured images) captured substantially simultaneously from two moving images captured by two anterior eye cameras, and analyze the paired two frames to determine the three-dimensional position of the subject's eye. The three-dimensional position of the subject's eye may be the three-dimensional position of a specific part of the anterior eye, and the specific part of the anterior eye may be the pupil (center, center of gravity, outline, etc. of the pupil).

[0072] The first image analysis unit 1030 is configured to obtain a time-series displacement of the slit light in scanning a three-dimensional region of the subject's eye by the first image acquisition unit 1010 by analyzing a series of time-series images collected by the second image acquisition unit 1020. The time-series displacement of the slit light is a time-series change in the projection position of the slit light on the subject's eye, in other words, a moving state of the projection position of the slit light on the subject's eye. The first image analysis unit 1030 includes a first image analysis processor that operates based on a first image analysis program.

[0073] In some exemplary embodiments, the first image analysis unit 1030 may be configured to perform a process of analyzing a series of time-series images collected by the second image collection unit 1020 to detect a series of images of the slit light, and a process of determining a time-series displacement of the slit light based on the detected series of images of the slit light.

[0074] The process of detecting a series of images of the slit light includes a process of analyzing each time-series image to detect the image of the slit light. Detecting the image of the slit light in the time-series images corresponds to identifying the position of the slit light in the time-series images. By performing this slit light image detection process (slit light position identification process) on (a part or all of) the series of time-series images, the time-series displacement of the slit light during the period when the series of time-series images was collected can be obtained.

[0075] A specific example of the processing executed by the first image analysis unit 1030 will be described with reference to FIG. 2. In this example, the longitudinal direction of the cross section of the slit light at the projection position on the subject's eye is along the Y direction, and the slit light is moved in the X direction during slit scanning. Symbol F(n) indicates one time-series image (frame) acquired by the second image acquisition unit 1020. Here, n is an integer equal to or greater than 1, and F(n) indicates the n-th time-series image acquired. A slit light image L(n) is depicted in the time-series image F(n).

[0076] When the time-series image F(n) is input to the first image analysis unit 1030, the first image analysis unit 1030 integrates (adds) the luminance values ​​of the pixels of the time-series image F(n) in the Y direction. That is, the first image analysis unit 1030 divides the pixel group P(X,Y) constituting the time-series image F(n) into partial pixel groups P(X,Y) corresponding to each X coordinate (partial pixel groups aligned in the Y direction) X (Y) X In (Y), X is fixed, and Y spans the range (domain) of the time-series image F(n) in the Y direction. As a result, the partial pixel group P corresponding to each X coordinate of the time-series image F(n) is X (Y) is defined.

[0077] Next, the first image analysis unit 1030 calculates the corresponding partial pixel group P X The brightness values ​​of multiple pixels included in (Y) are accumulated. By associating the X coordinate value with the accumulated brightness value (accumulated brightness value), a graph (accumulated brightness value distribution) B(n) is obtained that shows the distribution of the accumulated brightness value in the domain in the X direction of the time-series image F(n).

[0078] In the time-series image F(n), the slit light image L(n) is depicted brighter than other parts. The first image analysis unit 1030 detects the peak of the integrated luminance value distribution B(n), i.e., the maximum value in the integrated luminance value distribution B(n), and identifies the X-coordinate X(n) corresponding to the detected peak. The identified X-coordinate X(n) is recorded as the position (X-coordinate) of the slit light image L(n) in the time-series image F(n).

[0079] By executing such a series of processes for each time-series image F(n), the change in the position of the slit light moved in the X direction during the slit scan, that is, the time-series displacement of the slit light, can be obtained. The process in this example is only a calculation process related to the brightness value, and therefore has advantages such as high processing speed, short processing time, and small processing resources. Another example of a process for the same purpose is a method of detecting a slit light image by applying segmentation to the time-series images. Note that any process for identifying a partial region in an image and / or any process for identifying a position in an image may be applied.

[0080] Some application examples of such slit light position detection will be described below. In some exemplary embodiments, when the imaging protocol is a reciprocating scan, the position of the slit light at the X coordinate farthest from the X coordinate of the scan start position can be specified as the return position of the reciprocating scan.

[0081] The difference between the X coordinate of the scan start position and the X coordinate of the turn-back position corresponds to the dimension of the scan range in the X direction (the moving distance of the slit light, the scan length). Note that in the case of imaging protocols other than reciprocating scanning, the difference between the X coordinate of the scan start position and the X coordinate of the slit light farthest from the scan start position corresponds to the scan length.

[0082] Furthermore, in some exemplary embodiments, a series of Scheimpflug images collected by reciprocating scanning can be classified into a series of Scheimpflug images (first group) collected by forward scanning (e.g., slit scanning from the left end to the right end of the scanning range, i.e., slit scanning from the scanning start position to the turning position) and a series of Scheimpflug images (second group) collected by backward scanning (e.g., slit scanning from the right end to the left end of the scanning range, i.e., slit scanning from the turning position to the scanning end position). Note that the Scheimpflug image corresponding to the turning position may be included in both the first group and the second group.

[0083] Additionally, some exemplary embodiments may construct a three-dimensional image based on the first group (and a group of images obtained by interpolating the first group), and may construct a three-dimensional image based on the second group (and a group of images obtained by interpolating the second group). Some other exemplary embodiments may select a series of Scheimpflug images ranging from the right end to the left end of the scan range from the first group and the second group, and construct a three-dimensional image based on the selected series of Scheimpflug images (and a group of images obtained by interpolating the series of Scheimpflug images).

[0084] As will be described in detail in the explanation of the image interpolation unit 1040, in spatial image interpolation for constructing a three-dimensional image or a rendering image, the time series displacement of the slit light can be used to determine the number of images to be interpolated (the number of interpolated frames mentioned above).

[0085] As described above, the first image analysis unit 1030 is configured to analyze the series of time-series images collected by the second image collection unit 1020 to obtain the time-series displacement of the slit light in the slit scan. In addition, the first image analysis unit 1030 may be configured to analyze the series of time-series images collected by the second image collection unit 1020 to obtain the time-series displacement of the subject's eye during the period when the slit scan is being performed. The time-series displacement of the subject's eye is a time-series change in the position of the subject's eye relative to the ophthalmic imaging apparatus 1000 (particularly, the first image collection unit 1010). By using the time-series displacement of the subject's eye, it becomes possible to detect the relative position (time-series change in the relative position) between the subject's eye and the slit light during the period when the slit scan is being performed with higher accuracy and higher precision.

[0086] The first image analysis unit 1030 can identify the position of the subject's eye at the time point corresponding to each time-series image by analyzing each time-series image acquired by the second image acquisition unit 1020. Furthermore, the first image analysis unit 1030 can determine the time-series displacement of the subject's eye during the period during which the time-series images are acquired, i.e., the period during which the Scheimpflug images are acquired, by arranging in time series the positions of the subject's eye obtained from the time-series images in this manner.

[0087] An example of the process executed by the first image analysis unit 1030 to obtain the time-series displacement of the subject's eye will be described with reference to Fig. 3. In this example, the position of the subject's eye is defined based on a characteristic point of the pupil (e.g., the pupil center, the pupil centroid, a predetermined position on the pupil outline, etc.). The time-series images in this example are color images (RGB images). Note that the time-series images may be grayscale images, in which case, for example, step S3 in Fig. 3 is omitted.

[0088] When the first image collection unit 1010 starts collecting Scheimpflug images and the second image collection unit 1020 starts collecting time-series images (S1), the time-series images sequentially acquired by the second image collection unit 1020 are input sequentially and in real time to the first image analysis unit 1030 (S2).

[0089] The first image analysis unit 1030 extracts an image of a predetermined color channel from the input time-series image (RGB image) (S3). According to the study by the inventors of the present disclosure, the image with the clearest pupil boundary (pupil outline) was the image of the red channel (image of the red component, R image). In this example, the first image analysis unit 1030 extracts the image of the red channel from the images of the three color channels of the input time-series image.

[0090] Next, the first image analysis unit 1030 crops a predetermined area from the R image extracted in step S3 (S4). In this example, this predetermined area (crop area) is an area in which the image of the pupil fits. The crop area may be an area of ​​preset dimensions. For example, the crop area is an area of ​​512 pixels x 512 pixels. The preset dimensions of the crop area may be set based on, for example, design data of the optical system of the second image acquisition unit 1020, the working distance of the ophthalmic imaging apparatus 1000 (the second image acquisition unit 1020), standard pupil dimensions (standard pupil diameter), and the like. In addition, the crop area dimensions for the subject's eye may be determined based on the time-series images (or images of the subject's eye acquired in advance by the second image acquisition unit 1020).

[0091] Next, the first image analysis unit 1030 applies contrast enhancement processing to the cropped region obtained in step S4 (S5). This contrast enhancement processing may be, for example, histogram equalization, and more specifically, may be adaptive histogram equalization, contrast-limited adaptive histogram equalization (CLAHE), multi-peak histogram equalization (MPHE), multi-objective beta-optimized bi-histogram equalization (MBOBHE), etc. Also, a contrast enhancement method other than histogram equalization may be used.

[0092] Next, the first image analysis unit 1030 detects a pupil image (pupil region) from the cropped region whose contrast was enhanced in step S5 (S6). Any method may be used to detect the pupil region. For example, image segmentation using a CNN trained by machine learning to input an eye image and output a pupil region detection result, graph-based image segmentation using Dijkstra's algorithm, etc. may be used.

[0093] Next, the first image analysis unit 1030 determines the position of the pupil based on the pupil region detected in step S6 (S7). The position of the pupil determined in this step is the position (coordinates) of a predetermined feature point of the pupil, and is treated as the position of the subject's eye at the time corresponding to the time-series images. The information obtained as the position of the pupil (position of the subject's eye) in this step includes at least the X coordinate (position in the moving direction of the scanning light in the slit scan) and may further include the Y coordinate (position in the longitudinal direction of the cross section of the slit light). As described above, when the second image acquisition unit 1020 includes two anterior eye cameras, the first image analysis unit 1030 can determine three-dimensional position information (X coordinate, Y coordinate, Z coordinate) as the position of the pupil (position of the subject's eye). The position information of the subject's eye determined in this step is associated with, for example, the order information of the time-series images or information (time information) indicating the time corresponding to the time-series images, and is stored in a storage device (not shown).

[0094] A series of processes from steps S3 to S7 are applied to the time-series images sequentially acquired by the second image acquisition unit 1020 until the second image acquisition unit 1020 finishes collecting time-series images (and / or until the first image acquisition unit 1010 finishes collecting Scheimpflug images, or until the user instructs to end the series of processes) (S8: No). In some exemplary aspects, the series of processes from steps S3 to S7 are applied to all the time-series images collected by the second image acquisition unit 1020, but in some other exemplary aspects, the series of processes from steps S3 to S7 are applied only to a true subset of all the time-series images collected by the second image acquisition unit 1020 (in other words, only to a group of time-series images selected or extracted from all the time-series images).

[0095] When the second image collection unit 1020 has completed collection of time-series images (and / or when the first image collection unit 1010 has completed collection of Scheimpflug images, or when the user instructs the end of the series of processes), processing proceeds to step S9 (S8: Yes).

[0096] At the stage when the process proceeds to step S9, a series of position information of the subject's eye corresponding to the series of time-series images (at least a part thereof) collected by the second image collecting unit 1020 is stored. Each of the series of position information of the subject's eye is associated with sequence information or time information of the corresponding time-series image. The first image analyzing unit 1030 can generate data indicating the time-series displacement of the subject's eye by arranging the series of position information of the subject's eye on the time axis based on the series of sequence information or the series of time information (S9). The time-series displacement data of the subject's eye may be the correspondence relationship itself between the series of sequence information or the series of time information already acquired at the time of proceeding to step S9 and the series of position information of the subject's eye, or may be any data obtained from this correspondence relationship. The time-series displacement data of the subject's eye generated in step S9 is stored in a storage device (end), not shown.

[0097] As will be described in detail in the explanation of the image interpolation unit 1040, in spatial image interpolation for constructing a three-dimensional image or a rendering image, the time-series displacement of the test eye can be used to determine the number of images to be interpolated (the number of interpolated frames mentioned above).

[0098] The image interpolation unit 1040 is configured to perform interpolation of a series of Scheimpflug images collected by the first image collection unit 1010 based on the time-series displacement of the slit light acquired by the first image analysis unit 1030. This allows an image of a three-dimensional region of the subject's eye to which slit scanning is applied (a three-dimensional image) to be constructed based on a plurality of images including a series of collected Scheimpflug images and a group of images that interpolate these images, without narrowing the intervals between the series of collected Scheimpflug images. The aspect ratio of the three-dimensional image constructed in this manner corresponds to the actual shape of the anterior segment. The same is true for a rendering image obtained from this three-dimensional image.

[0099] As described above, in some exemplary embodiments, the slit scan by the first image acquisition unit 1010 and the repeated imaging of the subject's eye by the second image acquisition unit 1020 may be performed synchronously with each other.

[0100] In this embodiment, the image interpolation unit 1040 can associate a series of Scheimpflug images obtained by the slit scan with a series of time-series images obtained by the repeated shooting based on mutual synchronization between the slit scan and the repeated shooting (for example, mutual synchronization control between the first image acquisition unit 1010 and the second image acquisition unit 1020). This association is performed, for example, so as to associate the Scheimpflug images and the time-series images whose acquisition times are small with each other.

[0101] Furthermore, the image interpolation unit 1040 can interpolate the series of Scheimpflug images based further on the correspondence between the series of Scheimpflug images thus obtained and the series of time-series images (i.e., based on this correspondence and the time-series displacement of the slit light).

[0102] According to this aspect, by utilizing the mutual synchronization relationship between the slit scan and the repeated shooting, it is possible to perform image interpolation with higher accuracy and precision in time series, and therefore, it is possible to perform image interpolation with higher accuracy and precision in space.

[0103] An example of the configuration, an example of the functions, and modified examples of the image interpolation unit 1040 will be described later.

[0104] As described above, the first image acquisition unit 1010 in some exemplary embodiments includes a first imaging system and a second imaging system that respectively capture images of the subject's eye from different directions in a slit scan. A plurality of Scheimpflug images collected by the first imaging system in this slit scan is called a first Scheimpflug image group, and a plurality of Scheimpflug images collected by the second imaging system in this slit scan is called a second Scheimpflug image group. A series of Scheimpflug images collected in this slit scan includes a first Scheimpflug image group and a second Scheimpflug image group.

[0105] In the slit scan performed by the first image acquisition unit 1010 of this embodiment, the imaging of the subject's eye by the first imaging system and the imaging of the subject's eye by the second imaging system are performed in parallel with each other. That is, the first image acquisition unit 1010 performs imaging by the first imaging system and the imaging by the second imaging system in parallel while moving the projection position of the slit light with respect to the three-dimensional region of the subject's eye.

[0106] Furthermore, the first image collecting unit 1010 may be configured to perform the photographing by the first photographing system (collection of Scheimpflug images) and the photographing by the second photographing system (collection of Scheimpflug images) in synchronization with each other. By referring to this synchronous relationship, the first Scheimpflug image group and the second Scheimpflug image group can be easily associated with each other without using image processing or the like. This association is performed, for example, so as to associate Scheimpflug images that have a small difference in acquisition time with each other.

[0107] In such an embodiment, the ophthalmic imaging device 1000 can reconstruct a series of Scheimpflug images corresponding to the slit scan from the first Scheimpflug image group and the second Scheimpflug image group by referring to the mutual synchronization relationship between the imaging by the first imaging system and the imaging by the second imaging system.

[0108] The configuration of a modified example of an ophthalmic imaging apparatus 1000 having a function of reconstructing a series of Scheimpflug images is shown in Fig. 4. The first image acquisition unit 1010A of the ophthalmic imaging apparatus 1000A of this example includes a first imaging system 1011 and a second imaging system 1012. The first image acquisition unit 1010A is configured to perform imaging by the first imaging system 1011 and imaging by the second imaging system 1012 in synchronization with each other.

[0109] The ophthalmic imaging apparatus 1000A further includes an image selection unit 1050. The image selection unit 1050 is disposed between the first image acquisition unit 1010A and the image interpolation unit 1040. The image selection unit 1050 is configured to select a new series of Scheimpflug images from the first Scheimpflug image group and the second Scheimpflug image group, which correspond to the slit scan that collected the first Scheimpflug image group and the second Scheimpflug image group, based on a correspondence relationship between the first Scheimpflug image group and the second Scheimpflug image group based on synchronization between the photographing by the first photographing system 1010A and the photographing by the second photographing system 1020A. In short, the image selection unit 1050 is configured to reconstruct a series of Scheimpflug images from the first Scheimpflug image group and the second Scheimpflug image group collected by the first photographing system 1011 and the second photographing system 1012, respectively. In this example, the image interpolator 1040 performs interpolation of the series of Scheimpflug images reconstructed by the image selector 1050 .

[0110] Any method may be used for the image selection processing executed by the image selection unit 1050. For example, the method for the image selection processing may be determined and / or selected based on predetermined conditions and predetermined parameters such as the configuration and / or arrangement of the first imaging system 1011 and the second imaging system 1012, the purpose and / or use of image selection, and the configuration and / or function of the image interpolation unit 1040.

[0111] The configuration of a modified example of the ophthalmic imaging apparatus 1000A having an image selection function is shown in FIG. 5. The first image acquisition unit 1010B of the ophthalmic imaging apparatus 1000B of this example includes an illumination system 1013 in addition to the first imaging system 1011 and the second imaging system 1012 similar to those of the first image acquisition unit 1010A. The illumination system 1013 is configured to project slit light onto a three-dimensional area of ​​the subject's eye. The image selection unit 1050 of this example is disposed between the first image acquisition unit 1010B and the image interpolation unit 1040.

[0112] The first image collecting unit 1010B executes imaging by the first imaging system 1011 and imaging by the second imaging system 1012 in synchronization with each other. In this example, the optical axis of the first imaging system 1011 and the optical axis of the second imaging system 1012 are arranged to be inclined in opposite directions with respect to the optical axis of the illumination system 1013. For example, the optical axis of the first imaging system 1011 is arranged to be inclined leftward with respect to the optical axis of the illumination system 1013, and the optical axis of the second imaging system 1012 is arranged to be inclined rightward with respect to the optical axis of the illumination system 1013. The first imaging system 1011 and the second imaging system 1012 arranged in this manner may be referred to as the left imaging system and the right imaging system, respectively.

[0113] The tilt angle of the optical axis of the first imaging system 1011 with respect to the optical axis of the illumination system 1013 and the tilt angle of the optical axis of the second imaging system 1012 with respect to the optical axis of the illumination system 1013 may be equal to or different from each other. Furthermore, these tilt angles may be fixed or variable.

[0114] The illumination system 1013 of this example is constructed and arranged to project slit light, the longitudinal direction of the cross section of which is oriented in the Y direction, from the front direction onto the subject's eye. The first image acquisition unit 1010B of this example applies slit scanning to a three-dimensional region of the anterior segment of the subject's eye by integrally moving the illumination system 1013, the first imaging system 1011, and the second imaging system 1012 in the X direction.

[0115] The image selection unit 1050 of this example selects a new series of Scheimpflug images corresponding to the slit scan that collected the first Scheimpflug image group and the second Scheimpflug image group by selecting a plurality of Scheimpflug images that do not include artifacts from the first Scheimpflug image group and the second Scheimpflug image group based on the correspondence between the first Scheimpflug image group collected by the first imaging system 1011 and the second Scheimpflug image group collected by the second imaging system 1012. The artifact may be any type of artifact. When performing an anterior eye scan as in this example, the artifact may be an artifact caused by corneal reflection (corneal reflection artifact). Below, some examples of the processing performed by the image selection unit 1050 will be described.

[0116] The projection position (Scheimpflug image, frame) of the slit light where the corneal reflection artifact occurs is different between the left and right imaging systems. For example, as in this example, when slit scanning is performed by moving the illumination system 1013, the first imaging system 1011, and the second imaging system 1012 integrally in the X direction while projecting slit light whose longitudinal direction of the cross section is oriented in the Y direction from the front direction to the subject's eye, the corneal reflection light of the slit light is likely to enter the left imaging system when the slit light is projected to a position to the left of the corneal apex, and is likely to enter the right imaging system when the slit light is projected to a position to the right of the corneal apex.

[0117] Taking such circumstances into consideration, the image selection unit 1050 in some exemplary embodiments can first identify a Scheimpflug image (first corneal apex image) corresponding to the corneal apex from a first Scheimpflug image group collected by the first imaging system 1011 as the left imaging system, and can also identify a Scheimpflug image (second corneal apex image) corresponding to the corneal apex from a second Scheimpflug image group collected by the second imaging system 1012 as the right imaging system.

[0118] In some exemplary embodiments, the process of identifying the corneal apex image may include a process of detecting an image corresponding to the corneal surface from each Scheimpflug image included in the first Scheimpflug image group, a process of identifying a pixel closest to the ophthalmic imaging apparatus 1000B based on the Z coordinates of pixels of the detected images, and a process of setting the Scheimpflug image including the identified pixel as the first corneal apex image. The same applies to the setting of the second corneal apex image.

[0119] Next, the image selection unit 1050 selects a Scheimpflug image group located to the right of the first corneal apex image from the first Scheimpflug image group, and selects a Scheimpflug image group located to the left of the second corneal apex image from the second Scheimpflug image group, and forms a series of Scheimpflug images consisting of the two selected Scheimpflug image groups (as well as the first corneal apex image and / or the second corneal apex image). This provides a series of Scheimpflug images that cover the three-dimensional region of the anterior segment to which the slit scan has been applied and that are (highly likely to) not include corneal reflection artifacts.

[0120] Another example of the process of identifying the corneal apex image will be described. The image selection unit 1050 of some exemplary embodiments determines whether a corneal reflection artifact is included in either of two images acquired substantially simultaneously by the first imaging system 1011 (e.g., the left imaging system) and the second imaging system 1012 (e.g., the right imaging system). This corneal reflection artifact determination process includes a predetermined image analysis, for example, a threshold process on the luminance information assigned to the pixel. Note that the process of determining the two images acquired substantially simultaneously can be executed based on a synchronous relationship between the imaging by the first imaging system 1011 and the imaging by the second imaging system 1012.

[0121] In the threshold processing used in the artifact determination process, for example, pixels assigned with a luminance value exceeding a preset threshold are identified. Typically, the threshold is set higher than the luminance value of the slit light image (projection area of ​​the slit light) in the image. In this way, the image selection unit 1050 is configured to determine an image brighter than the slit light image as an artifact without determining the slit light image as an artifact. An image brighter than the slit light image in the Scheimpflug image is highly likely to be an image caused by regular reflection at the cornea, and therefore the artifact detected by the image selection unit 1050 is highly likely to be a corneal reflection artifact.

[0122] The image selection unit 1050 may perform any image analysis other than threshold processing for artifact determination, such as pattern recognition, segmentation, edge detection, etc. In general, any information processing technique, such as image analysis, image processing, artificial intelligence, cognitive computing, etc., can be applied to artifact determination.

[0123] When it is determined as a result of the artifact determination that one of the two images acquired substantially simultaneously by the first imaging system 1011 and the second imaging system 1012 contains an artifact, the image selection unit 1050 selects the other image. In other words, the image selection unit 1050 selects the image that is not determined to contain an artifact, out of these two images.

[0124] When it is determined that both images contain artifacts, the image selection unit 1050 can, for example, evaluate the magnitude of the adverse effect that the artifact has on observation or diagnosis, and select the image with the smaller adverse effect. This evaluation is performed, for example, based on one or more conditions of the size, intensity, shape, and position of the artifact. Typically, artifacts with large dimensions, artifacts with high intensity, artifacts located in or near a region of interest such as a slit light image, etc. are evaluated as having a large adverse effect.

[0125] In addition, in cases where artifacts are included in both images, the artifact removal disclosed in Patent Document 3 (JP 2019-213733 A) may be applied.

[0126] These variations can provide images of a three-dimensional region of the subject's eye that are free of artifacts that would interfere with observation, analysis, or diagnosis, for example by constructing a three-dimensional image of the subject's eye based on a set of artifact-free images.

[0127] In addition, even if the images are obtained by photographing substantially the same position, the dimensions of the depicted predetermined part may differ between the image obtained by the left photographing system and the image obtained by the right photographing system. For example, the thickness of the depicted cornea may differ between the left image and the right image obtained by photographing substantially the same position with the left photographing system and the right photographing system, respectively. Even in such a case, it is possible to match the dimensions of the predetermined part by using the image selection unit 1050.

[0128] Some configuration examples, some function examples, and some modified examples of the image interpolation unit 1040 will be described below.

[0129] In some exemplary embodiments, the image interpolation unit 1040 may be configured to perform spatial image interpolation using the optical flow described above. That is, the image interpolation unit 1040 may be configured to receive the series of Scheimpflug images collected by the first image collection unit 1010 (or the series of Scheimpflug images reconstructed by the image selection unit 1050), apply the optical flow to the series of Scheimpflug images to represent the displacement of the subject's eye as a vector, and perform interpolation of the series of Scheimpflug images based on the vector representation.

[0130] According to this example, optical flow, which is generally used to analyze the movement (change over time) of an object, can be used to grasp the spatial changes of the object, thereby enabling spatial image interpolation of a series of Scheimpflug images.

[0131] In some example embodiments, the image interpolator 1040 may be configured to perform spatial image interpolation using a pre-constructed inference model, i.e., the image interpolator 1040 may be configured to receive the series of Scheimpflug images collected by the first image collector 1010 (or the series of Scheimpflug images reconstructed by the image selector 1050), input the series of Scheimpflug images to the inference model, and perform interpolation of the series of Scheimpflug images based on the output of the inference model.

[0132] The inference model used for spatial image interpolation is constructed in advance by applying machine learning using training data including at least Scheimpflug images of the eye to a neural network. The inference model constructed in this way is configured to input two or more Scheimpflug images each representing two or more different positions of the same eye, and output an interpolated image obtained by spatially interpolating these two or more Scheimpflug images. The interpolated image output from the inference model may be only an image added to the two or more input Scheimpflug images (an image created by the image interpolation unit 1040), or both the two or more input Scheimpflug images and the added image.

[0133] An example of an image interpolator 1040 configured to perform spatial image interpolation using an inference model is shown in Figure 6A. The image interpolator 1040A in this example includes an inference unit 1041 configured to perform spatial image interpolation using an inference model 1042.

[0134] The inference model 1042 is constructed by applying machine learning using training data including at least a Scheimpflug image of the eye to a neural network. An apparatus for constructing the inference model 1042 (an inference model construction apparatus) may be provided in the ophthalmic imaging apparatus 1000 (1000A, 1000B), in a peripheral device (such as a computer) of the ophthalmic imaging apparatus 1000 (1000A, 1000B), or may be another computer.

[0135] The model construction unit 1060 shown in FIG. 6B is an example of an inference model construction device, and includes a learning processing unit 1061 and a neural network 1062.

[0136] The neural network 1062 typically includes a convolutional neural network (CNN). An example of the structure of a convolutional neural network is shown in FIG. 6B.

[0137] An image is input to the input layer. Behind the input layer, multiple pairs of convolution layers and pooling layers are arranged. In the example shown in Figure 6B, three pairs of convolution layers and pooling layers are provided, but the number of pairs is arbitrary.

[0138] In the convolution layer, a convolution operation is performed to grasp features (such as contours) from an image. The convolution operation is a multiplication and accumulation operation of a filter function (weighting factor, filter kernel) of the same dimension as the input image for the input image. In the convolution layer, the convolution operation is applied to each of multiple parts of the input image. More specifically, in the convolution layer, the value of each pixel of the partial image to which the filter function is applied is multiplied by the value (weight) of the filter function corresponding to that pixel to calculate the product, and the sum of the products is calculated over the multiple pixels of this partial image. The product-accumulation value thus obtained is substituted for the corresponding pixel in the output image. By performing the product-accumulation operation while moving the location (partial image) to which the filter function is applied, the convolution operation result for the entire input image is obtained. According to such a convolution operation, a large number of images in which various features are extracted using a large number of weighting factors are obtained. In other words, a large number of filtered images such as smoothed images and edge images are obtained. The large number of images generated by the convolution layer are called feature maps.

[0139] In the pooling layer, the feature map generated by the immediately preceding convolution layer is compressed (data thinning, etc.). More specifically, in the pooling layer, statistics of a predetermined neighboring pixel of a pixel of interest in the feature map are calculated at a predetermined pixel interval, and an image with a smaller size than the input feature map is output. The statistics applied to the pooling operation are, for example, maximum value (max pooling) or average value (average pooling). The pixel interval applied to the pooling operation is called a stride.

[0140] A convolutional neural network can extract many features from an input image by performing processing using multiple pairs of convolutional layers and pooling layers.

[0141] A fully connected layer is provided after the last pair of convolutional and pooling layers. In the example structure shown in FIG. 6B, two fully connected layers are provided, but the number of fully connected layers is arbitrary. In the fully connected layer, image classification, image segmentation, regression, and other processes are performed using features compressed by a combination of convolution and pooling. After the last fully connected layer, an output layer is provided to provide the output results.

[0142] In some exemplary embodiments, the convolutional neural network may not include a fully connected layer (e.g., a fully layered convolutional network (FCN)), and may include a support vector machine, a recurrent neural network (RNN), or the like. Machine learning for the neural network 1062 may include transfer learning. That is, the neural network 1062 may include a neural network that has already been trained using other training data (training images) and has had its parameters adjusted. The model construction unit 1060 (the learning processing unit 1061) may be configured to be able to apply fine tuning to the trained neural network (the neural network 1062). The neural network 1062 may be constructed using, for example, a known open source neural network architecture.

[0143] The learning processing unit 1061 applies machine learning using training data to the neural network 1062. When the neural network 1062 includes a convolutional neural network, the parameters adjusted by the learning processing unit 1061 include, for example, filter coefficients of a convolutional layer and connection weights and offsets of a fully connected layer.

[0144] As described above, the training data may include one or more Scheimpflug images acquired from one or more eyes. Since the Scheimpflug images of the eyes are the same type of images as the images input to the inference unit 1041, the quality (accuracy, precision, etc.) of the output of the inference unit 1041 can be improved compared to the case where machine learning is performed using training data including only other types of images.

[0145] The training data may include two or more Scheimpflug images each representing two or more different positions of the same eye. In this case, the machine learning is performed, for example, so as to optimize the output (interpolated image obtained by spatially interpolating the two or more Scheimpflug images) when such two or more Scheimpflug images are input. Since the Scheimpflug image group in this example is an image group of the same type as the series of Scheimpflug images input to the inference unit 1041, the quality of the output of the inference unit 1041 can be further improved.

[0146] The types of images included in the training data are not limited to Scheimpflug images, and may include, for example, images acquired by other ophthalmic modalities (fundus cameras, OCT devices, SLO, surgical microscopes, etc.), images acquired by imaging diagnostic modalities of other medical departments (ultrasound diagnostic devices, X-ray diagnostic devices, X-ray CT devices, magnetic resonance imaging (MRI) devices, etc.), images generated by processing actual images, pseudo images, etc. Furthermore, the number of images, etc. included in the training data may be increased by using techniques such as data expansion and data augmentation.

[0147] The training method (machine learning method) for constructing the inference model 1042 may be any method, but may be, for example, any one of supervised learning, unsupervised learning, and reinforcement learning, or a combination of any two or more of them.

[0148] In some exemplary embodiments, supervised learning is performed using training data in which input images are labeled with a final output label. For example, a previously obtained interpolated image is pre-labeled for each set of Scheimpflug images included in the training data (for example, a set of two or more Scheimpflug images each representing two or more different positions of the same eye). The label is generated, for example, by a doctor, a computer, another inference model, or the like. The learning processing unit 1061 can build the inference model 1042 by applying supervised learning using such training data to the neural network 1062.

[0149] The inference model 1042 of this example constructed in this manner is a trained model that takes as input two or more Scheimpflug images, each representing two or more different positions of the same eye, and outputs an interpolated image obtained by spatially interpolating these two or more Scheimpflug images.

[0150] In order to prevent processing from concentrating on a particular unit of the neural network 1062, the learning processing unit 1061 may randomly select and disable some units, and perform learning using the remaining units (dropout).

[0151] The method used to build the inference model is not limited to the examples shown here. For example, any method such as a support vector machine, a Bayesian classifier, boosting, a k-means algorithm, a kernel density estimation, a principal component analysis, an independent component analysis, a self-organizing map, a random forest, or a generative adversarial network (GAN) can be used to build the inference model.

[0152] 6A uses such an inference model 1042 to perform spatial image interpolation of a series of Scheimpflug images collected by slit scanning of a three-dimensional region of the subject's eye (or a series of Scheimpflug images reconstructed by the image selection unit 1050). More specifically, first, the inference unit 1041 inputs the series of Scheimpflug images to the inference model 1042. The inference model 1042 outputs an interpolated image based on the series of Scheimpflug images input.

[0153] In some exemplary embodiments, the image interpolation unit 1040 can use the time series displacement of the slit light acquired by the first image analysis unit 1030 to determine the number of images to be interpolated for the series of Scheimpflug images. An example of the image interpolation unit 1040 of this embodiment is shown in Fig. 7. The image interpolation unit 1040B of this embodiment includes a first interpolation number determination unit 1043.

[0154] The first interpolation number determination unit 1043 is configured to determine the number of images (interpolation number) to be added in spatial image interpolation of the series of Scheimpflug images collected by the first image collection unit 1010 (or the series of Scheimpflug images reconstructed by the image selection unit 1050) based on at least one of the operation information of the first image collection unit 1010 and the time-series displacement of the slit light acquired by the first image analysis unit 1030. The image interpolation unit 1040B interpolates the series of Scheimpflug images using the interpolation number of images determined by the first interpolation number determination unit 1043.

[0155] The operation information of the first image acquisition unit 1010 may include a scan condition of a slit scan for acquiring a series of Scheimpflug images. The scan condition may include any condition of the slit scan, such as a scan length (a moving distance of the slit light), a scan speed (a moving speed of the slit light), and a scan rate (an imaging rate of the imaging system, an imaging repetition frequency, a frame rate, and an image acquisition rate).

[0156] In a mode in which the operation information of the first image collection unit 1010 is referred to, the first interpolation number determination unit 1043 can determine the distance (image interval) between two adjacent Scheimpflug images based on the operation information as described above.

[0157] Although the image intervals in a series of Scheimpflug images may be assumed to be uniform, they may generally be considered to be non-uniform. For example, the slit light moving speed is relatively slow in the section immediately after the start of scanning (acceleration section), the section immediately before the end of scanning (deceleration section), and the section near the turning point of the reciprocating scan (deceleration section and acceleration section), while the slit light moving speed is relatively fast in the section near the center position of the scanning range, so the image intervals are non-uniform.

[0158] The first interpolation number determination unit 1043 can determine the number of interpolations for the series of Scheimpflug images by determining each image spacing in the series of Scheimpflug images by referring to the operation information, and by comparing each image spacing with a default value (default image spacing) required for 3D imaging.

[0159] In a mode in which the time series displacement of the slit light acquired by the first image analysis unit 1030 is referred to, the first interpolation number determination unit 1043 can determine the number of interpolations based on the actual time series displacement of the slit light in the slit scan (e.g., the distance traveled by the slit light, the position at each point in time, the speed of movement at each point in time, the acceleration at each point in time, etc.) in the same manner as when the operation information of the first image collection unit 1010 is referred to.

[0160] In some exemplary embodiments, the image interpolation unit 1040 can use the time-series displacement of the subject's eye acquired by the first image analysis unit 1030 to determine the number of images to be interpolated for the series of Scheimpflug images. An example of the image interpolation unit 1040 of this embodiment is shown in Fig. 8. The image interpolation unit 1040C of this embodiment includes a second interpolation number determination unit 1044.

[0161] The second interpolation number determination unit 1044 is configured to determine the number of images (interpolation number) to be added in spatial image interpolation of the series of Scheimpflug images collected by the first image collection unit 1010 (or the series of Scheimpflug images reconstructed by the image selection unit 1050) based on at least one of the operation information of the first image collection unit 1010 and the time-series displacement of the subject's eye acquired by the first image analysis unit 1030. The image interpolation unit 1040C performs interpolation of the series of Scheimpflug images using the interpolation number of images determined by the second interpolation number determination unit 1044.

[0162] In the aspect of referring to the operation information of the first image collection section 1010, the process executed by the second interpolation number determination section 1044 may be similar to the process executed by the first interpolation number determination section 1043 described above.

[0163] In the embodiment where the time-series displacement of the subject's eye acquired by the first image analysis unit 1030 is referred to, the influence of the eye movement (fixation shift, etc.) of the subject's eye on the slit scan can be cancelled. In this embodiment, the second interpolation number determination unit 1044 can extract the time-series displacement in the X direction from the time-series displacement of the subject's eye, and correct the X-direction position of each Scheimpflug image so as to cancel the X-direction displacement of the subject's eye at the time point corresponding to each Scheimpflug image. This allows a series of Scheimpflug images to be obtained from which the influence of the image position shift in the X direction caused by the movement of the subject's eye during the slit scan has been removed. The second interpolation number determination unit 1044 can determine the number of interpolations for this series of Scheimpflug images by comparing each image interval of the series of Scheimpflug images rearranged in this way with a default value (default image interval) required for three-dimensional imaging.

[0164] In addition, the second interpolation number determination unit 1044 can extract a time series displacement in the Y direction from the time series displacement of the subject's eye, and correct the Y direction position of each Scheimpflug image according to the Y direction displacement of the subject's eye at the time corresponding to the Scheimpflug image. This allows a series of Scheimpflug images to be obtained that are aligned with the image position shift in the Y direction caused by the movement of the subject's eye during the slit scan. When there is a Y direction deviation between two adjacent Scheimpflug images, the image interpolation unit 1040C can position each image that interpolates these two Scheimpflug images in accordance with the Y direction inclination based on the Y direction deviation. The same can be true when a time series displacement in the Z direction is obtained.

[0165] A usage form of the ophthalmologic imaging apparatus 1000 according to the first aspect will be described with reference to Fig. 9. The usage forms described below are merely examples. For example, any of the matters described in the summary of the above-mentioned embodiment or any of the matters described in the first aspect can be combined with the following usage forms.

[0166] It is assumed that various operations (preparatory operations) have been completed before photographing by the ophthalmic imaging apparatus 1000. The preparatory operations include adjustment of a table on which the ophthalmic imaging apparatus 1000 is placed, adjustment of a chair used by the subject, adjustment of a face rest (such as a chin rest or a forehead rest) of the ophthalmic imaging apparatus 1000, alignment of the ophthalmic imaging apparatus 1000 with respect to the subject's eye, adjustment of the light intensity of the slit light, etc.

[0167] Upon receiving an instruction to start photographing, the ophthalmologic photographing apparatus 1000 causes the first image acquisition unit 1010 and the second image acquisition unit 1020 to acquire a series of Scheimpflug images and a series of time-series images in parallel (S11).

[0168] A series of Scheimpflug images collected by the first image collection unit 1010 and a series of time-series images collected by the second image collection unit 1020 are stored, for example, in a memory unit (not shown) provided in the ophthalmic imaging apparatus 1000 and / or a storage device (not shown) connected to the ophthalmic imaging apparatus 1000.

[0169] Next, the ophthalmic imaging apparatus 1000 uses the first image analysis unit 1030 to analyze the series of time-series images collected by the second image collection unit 1020 in step S11, thereby determining the time-series displacement of the slit light during at least a portion of the period during which the series of Scheimpflug images was collected by the first image collection unit 1010 in step S11 (S12).

[0170] Information on the time series displacement of the slit light acquired by the first image analysis unit 1030 is stored in, for example, the storage unit of the ophthalmic imaging apparatus 1000 and / or a storage device connected to the ophthalmic imaging apparatus 1000.

[0171] Next, the ophthalmologic photographing apparatus 1000 causes the image interpolation unit 1040 to interpolate the series of Scheimpflug images collected in step S11 based on the information on the time-series displacement of the slit light acquired in step S12 (S13).

[0172] The images (additional images) created by the image interpolation unit 1040 and added to the series of Scheimpflug images are, for example, stored together with the series of Scheimpflug images.

[0173] In some exemplary aspects, the image interpolation unit 1040 can determine the number of interpolations based on one or more of the operation information of the first image collection unit 1010 in step S11, the time-series displacement information of the slit light obtained in step S12, and the time-series displacement information of the test eye generated from the series of time-series images collected in step S11.

[0174] In an embodiment in which the first image acquisition unit 1010 includes the first imaging system 1011 and the second imaging system 1012, the ophthalmic imaging apparatus 1000 can form a new series of Scheimpflug images (e.g., a plurality of Scheimpflug images not mixed with corneal reflection artifacts) from the first Scheimpflug image group acquired by the first imaging system 1011 and the second Scheimpflug image group acquired by the second imaging system 1012 by the image selection unit 1050, and can further interpolate this new series of Scheimpflug images by the image interpolation unit 1040. The new series of Scheimpflug images and the image group added thereto (additional image group) are stored in, for example, a storage unit of the ophthalmic imaging apparatus 1000 and / or a storage device connected to the ophthalmic imaging apparatus 1000.

[0175] Next, the ophthalmic imaging apparatus 1000 or an image processing apparatus not shown constructs a three-dimensional image (three-dimensional volume) based on a plurality of images including a series of Scheimpflug images and a group of additional images, and further constructs a rendering image based on this three-dimensional image (S14).

[0176] The three-dimensional image and / or the rendered image are stored in, for example, a storage unit of the ophthalmic imaging apparatus 1000 and / or a storage device connected to the ophthalmic imaging apparatus 1000 .

[0177] Next, the ophthalmic imaging apparatus 1000 or a computer not shown reads out the data stored in the memory unit of the ophthalmic imaging apparatus 1000 and / or a storage device connected to the ophthalmic imaging apparatus 1000, and displays various images for observation and diagnosis of the subject's eye on a display device not shown (S15).

[0178] The various images displayed include, for example, the rendering image constructed in step S14. The user can instruct the application of a desired rendering to the three-dimensional image constructed in step S14.

[0179] Also, a series of Scheimpflug images can be selectively or as a list. Similarly, a group of additional images can be selectively or as a list. Here, the display mode of the Scheimpflug image and the display mode of the additional image may be different from each other. For example, when an additional image is displayed, information indicating that the image is an interpolated image, that is, that the image is created by a computer (not obtained by actually photographing the subject's eye) can be additionally displayed. Similarly, when a Scheimpflug image is displayed, information indicating that the image is obtained by actually photographing the subject's eye can be additionally displayed. This allows the user to easily understand whether each displayed image is a Scheimpflug image or an additional image.

[0180] According to the first aspect capable of realizing such a usage form, since a series of Scheimpflug images collected from a three-dimensional region of the test eye can be interpolated with additional images, three-dimensional imaging can be performed without narrowing the intervals between the series of Scheimpflug images collected from the three-dimensional region of the test eye. This makes it possible to obtain a three-dimensional image or a rendering image with an aspect ratio according to the actual shape of the three-dimensional region of the test eye to which slit scanning is applied. Therefore, it is possible to easily and accurately observe the test eye compared to the conventional method, and it is expected that the accuracy and precision of diagnosis will be improved. In addition, it is expected that an analysis result of higher quality than the conventional method can be obtained by performing image analysis using the three-dimensional image or the rendering image obtained in the first aspect. In addition to these effects, the first aspect has various effects described in the present disclosure. In addition, a person skilled in the art would be able to understand effects not explicitly stated in the present disclosure based on the contents of the present disclosure.

[0181] <Second aspect> The second aspect provides an exemplary aspect of an ophthalmologic imaging device capable of performing eye tissue identifier assignment. A specific example (example) of the ophthalmologic imaging device according to this aspect will be described later. Also, detailed explanations of matters similar to those described in the outline of the above-mentioned embodiment and matters similar to those of the first aspect may be omitted.

[0182] 10 shows the configuration of an ophthalmic imaging apparatus according to this embodiment. The ophthalmic imaging apparatus 2000 includes a first image acquisition unit 2010, a second image acquisition unit 2020, a second image analysis unit 2030, and an identifier assignment unit 2040.

[0183] The information obtained through the assignment of the ocular tissue identifier by the identifier assigning unit 2040 (including a plurality of Scheimpflug images, each of which is assigned an ocular tissue identifier) ​​is used in various post-processing operations such as construction of a three-dimensional image, construction of a rendering image, image analysis, image diagnosis, etc. Elements for performing the post-processing operations (processor, user interface, etc.) may be provided in the ophthalmic imaging apparatus 2000 or in a device different from the ophthalmic imaging apparatus 2000.

[0184] The first image collection unit 2010 may have the same functions and configuration as the first image collection unit 1010 of the first embodiment, and the second image collection unit 2020 may have the same functions and configuration as the second image collection unit 1020 of the first embodiment.

[0185] The second image analysis unit 2030 detects a series of images of the slit light (a series of slit light images) by analyzing a series of time-series images collected by the second image collection unit 2020. The process of detecting the slit light images from the time-series images may be performed in the same manner as in the first aspect (the first image analysis unit 1030).

[0186] The identifier assignment unit 2040 assigns to each of the series of Scheimpflug images collected by the first image collection unit 2010 an identifier indicating the ocular tissue of the test eye onto which the slit light was projected when the Scheimpflug image was acquired by the first image collection unit 2010, based on the series of slit light images detected from the series of time-series images by the second image analysis unit 2030.

[0187] In this embodiment, since the slit scan is applied to the three-dimensional region of the anterior part of the subject's eye, the eye tissue indicated by the identifier given to the Scheimpflug image may include tissues and parts of the anterior part of the eye, and may further include tissues in the vicinity of the eye. The tissues and parts of the anterior part of the eye may include, for example, the cornea, corneal apex, anterior corneal surface, posterior corneal surface, limbus, iris, anterior iris surface, posterior iris surface, iris margin, pupil, pupil margin, pupil center, pupil centroid, anterior chamber, iridocorneal angle, ciliary body, Zinn's zonule, crystalline lens, nerve, blood vessel, lesion, treatment scar, artificial object (intraocular lens, MIGS device, etc.) implanted in the anterior part of the subject's eye, any tissue and / or any part. The tissues in the vicinity of the eye may include, for example, the eyelid, meibomian gland, eyelashes, lacrimal puncta, any tissue and / or any part.

[0188] In some exemplary embodiments, the slit scan of the three-dimensional region of the subject's eye by the first image acquisition unit 2010 and the repeated photographing of the subject's eye by the second image acquisition unit 2020 may be performed in synchronization with each other. This allows a correspondence between the series of Scheimpflug images acquired by the first image acquisition unit 2010 and the series of time-series images acquired by the second image acquisition unit 2020 to be obtained. The synchronization between the first image acquisition unit 2010 and the second image acquisition unit 2020 may be performed in the same manner as in the first embodiment. Also, the correspondence between the series of Scheimpflug images and the series of time-series images may be similar to that of the first embodiment.

[0189] In some other exemplary embodiments, post-processing may be performed to determine correspondence between the series of Scheimpflug images collected by the first image collection component 2010 and the series of time-series images collected by the second image collection component 2020. This post-processing may be performed in the same manner as in the first embodiment.

[0190] In a configuration in which the slit scan of the first image collection unit 2010 and the repeated shooting of the second image collection unit 2020 are performed synchronously, the identifier assignment unit 2040 may be configured to assign an identifier further based on the correspondence between a series of Scheimpflug images and a series of time-series images based on this synchronization.

[0191] That is, the identifier assigning unit 2040 in such an exemplary embodiment may be configured to assign, to each Scheimpflug image collected by the first image collecting unit 2010, an identifier indicating the ocular tissue of the subject's eye onto which the slit light was projected when the Scheimpflug image was acquired by the first image collecting unit 2010, based on a series of slit light images detected from a series of time-series images by the second image analyzing unit 2030 and a correspondence relationship between a series of Scheimpflug images and a series of time-series images based on synchronization between the slit scan of the first image collecting unit 2010 and the repeated shooting of the second image collecting unit 2020. The same applies to an exemplary embodiment in which a similar correspondence relationship is established by post-processing.

[0192] According to such an exemplary embodiment, by referring to the correspondence between a series of Scheimpflug images and a series of time-series images, a Scheimpflug image corresponding to each time-series image can be easily and accurately identified, and an ocular tissue identifier determined from each time-series image (the slit light image depicted therein) can be assigned to the Scheimpflug image that accurately corresponds to that time-series image.

[0193] The identifier assigning unit 2040 in such an exemplary embodiment may be configured to execute the following two processes (first process and second process): In the first process, the identifier assigning unit 2040 assigns an eye tissue identifier to each of a series of time-series images based on a slit light image detected from the time-series image.

[0194] For example, in the first process, the identifier assignment unit 2040 may be configured to perform the following series of processes for each of the series of time-series images: a process of identifying image areas (ocular tissue image areas) corresponding to each of the various ocular tissues described above by analyzing the time-series images; a process of determining the positional relationship (overlap relationship, intersection relationship, etc.) between the slit light image detected from the time-series images and each ocular tissue image area; a process of identifying the ocular tissue onto which the slit light was projected when the time-series images were acquired by the second image collection unit 2020, based on the positional relationship obtained for each ocular tissue image area relative to the slit light image; a process of assigning an identifier indicating the identified ocular tissue to the time-series images.

[0195] For example, when an image region corresponding to the cornea at least partially overlaps with the slit light image, an identifier indicating the cornea is assigned to the time-series image. The number of eye tissue identifiers assigned to one time-series image is arbitrary and may be one or more. Furthermore, when no eye tissue onto which the slit light was projected was identified when the time-series image was acquired by the second image acquisition unit 2020, information indicating the result (that no eye tissue was identified) or error information can be assigned to the time-series image.

[0196] In the second process, the identifier assignment unit 2040 assigns an ocular tissue identifier to each of the series of Scheimpflug images based on the series of ocular tissue identifiers assigned to each of the series of time-series images by the first process and the correspondence between the series of Scheimpflug images and the series of time-series images.

[0197] For example, in the second process, the identifier assignment unit 2040 may be configured to perform a process of identifying a Scheimpflug image corresponding to each of the series of time-series images based on the above-mentioned correspondence relationship, and a process of assigning the ocular tissue identifier assigned to the time-series image in the first process to the identified Scheimpflug image.

[0198] This enables the identifier assignment unit 2040 to easily and accurately assign to each of a series of Scheimpflug images an identifier indicating the ocular tissue of the test eye onto which the slit light was projected when the Scheimpflug image was acquired by the first image collection unit 2010.

[0199] Additionally, from the series of Scheimpflug images acquired by the first image acquisition component 2010, a group of Scheimpflug images corresponding to certain ocular tissues may be selected.

[0200] In addition, a first group corresponding to a first ocular tissue, a second group corresponding to a second ocular tissue, ..., Pth group (P is an integer equal to or greater than 2) can be obtained from the series of Scheimpflug images collected by the first image collection unit 2010, where P is an integer equal to or greater than 2. Also, the p1th group and the p2th group may at least partially overlap (p1 and p2 are mutually different integers equal to or greater than 1 and equal to or less than 2).

[0201] In addition to assigning an ocular tissue identifier to the Scheimpflug image based on the time series images, some exemplary embodiments may be configured to assign an ocular tissue identifier to the Scheimpflug image based on the time series images and / or assign an ocular tissue identifier to the time series images based on the Scheimpflug image. According to these exemplary embodiments, it is possible to identify and assign an identifier to ocular tissue that is difficult or impossible to detect in a time series image (such as a frontal image). For example, it is possible to identify and assign an identifier to ocular tissue that is difficult or impossible to detect in a frontal image of the subject's eye, such as the corneal apex, intraocular tissue (such as the lens and the angle), intraocular lesions, and artificial objects implanted in the eye.

[0202] An exemplary embodiment configured to assign an ocular tissue identifier to a Scheimpflug image based on the Scheimpflug image will be described. The second image analysis unit 2030 of this embodiment may be configured to analyze a series of Scheimpflug images collected by the first image collection unit 2010 to detect a series of images of a slit light. The second image analysis unit 2030 detects a slit light image depicted in each Scheimpflug image by analyzing the Scheimpflug image. This process may be performed in the same manner as the process of detecting a slit light image from a time-series image.

[0203] Furthermore, the identifier assigning unit 2040 of this embodiment may be configured to assign an ocular tissue identifier based on a series of slit light images detected from a series of Scheimpflug images collected by the first image collecting unit 2010. The identifier assigning unit 2040 assigns an ocular tissue identifier to each of the series of Scheimpflug images based on the slit light image detected from the Scheimpflug image. The assignment of an ocular tissue identifier to a Scheimpflug image based on a slit light image in the Scheimpflug image may be performed in the same manner as the assignment of an ocular tissue identifier to a time-series image based on a slit light image in the time-series image (described above).

[0204] An exemplary embodiment configured to assign an ocular tissue identifier to the time-series images based on the Scheimpflug images will be described. The second image analysis unit 2030 of this embodiment may be configured to analyze the series of Scheimpflug images collected by the first image collection unit 2010 to detect an image of a site of interest in the subject's eye. The site of interest may be the corneal apex, intraocular tissue, intraocular lesion, artificial object implanted in the eye, etc. The process of detecting the image of the site of interest from the Scheimpflug images may be performed in the same manner as the process of detecting the image of the ocular tissue from the time-series images.

[0205] Furthermore, the identifier assigning unit 2040 of this embodiment may be configured to assign an identifier indicating a site of interest to any one of the series of time-series images collected by the second image collecting unit 2020 based on an image of the site of interest detected from the series of Scheimpflug images collected by the first image collecting unit 2010. In order to assign information obtained from the Scheimpflug images to the time-series images, it is possible to refer to the correspondence between the series of Scheimpflug images and the series of time-series images obtained by using the above-mentioned synchronization or post-processing.

[0206] The assignment of an ocular tissue identifier to a Scheimpflug image may be an assignment of an ocular tissue identifier to the Scheimpflug image itself, an assignment of an ocular tissue identifier to a series of Scheimpflug images, an assignment of an ocular tissue identifier to a group of images selected from the series of Scheimpflug images, or an assignment of an ocular tissue identifier to a subregion of the Scheimpflug image. Similarly, the assignment of an ocular tissue identifier to a time series of images may be an assignment of an ocular tissue identifier to the time series itself, an assignment of an ocular tissue identifier to a series of images selected from the series of images, or an assignment of an ocular tissue identifier to a subregion of the time series of images.

[0207] In some exemplary embodiments, the identifier assigning unit 2040 may be configured to assign, for each of a series of Scheimpflug images, an identifier corresponding to the ocular tissue to a subregion of the Scheimpflug image that corresponds to the ocular tissue onto which the slit light was projected when the Scheimpflug image was acquired by the first image acquisition unit 1010.

[0208] For example, if the eye tissue onto which the slit light was projected when a Scheimpflug image was acquired was the cornea, the identifier assignment unit 2040 can assign an identifier indicating the cornea to a partial image (corneal region) in the Scheimpflug image that corresponds to the cornea.

[0209] As another example, if the eye tissues onto which the slit light was projected when a Scheimpflug image was acquired were the cornea and the angle, the identifier assignment unit 2040 can assign an identifier indicating the cornea and an identifier indicating the angle to the corneal region and the angle region in the Scheimpflug image, respectively.

[0210] According to this aspect, it is possible to easily grasp not only which part of the Scheimpflug image the slit light is projected onto, but also the type, range, shape, etc. of the ocular tissue onto which the slit light is projected. Similarly, it is possible to easily grasp not only which part of the time-series image the slit light is projected onto, but also the type, range, shape, etc. of the ocular tissue onto which the slit light is projected.

[0211] Any of the features described in the first embodiment can be combined with the second embodiment. Some examples are described below. Note that the combination of the first embodiment and the second embodiment is not limited to these, and any of the features of the first embodiment can be at least partially combined with any of the features of the second embodiment.

[0212] In some exemplary embodiments, the first image acquisition unit 2010 may include a first imaging system and a second imaging system that capture images of the subject's eye from different directions in the slit scan. The first imaging system collects a first Scheimpflug image group, and the second imaging system collects a second Scheimpflug image group. The series of Scheimpflug images collected by the first image acquisition unit 2010 of this embodiment includes the first Scheimpflug image group and the second Scheimpflug image group. Although illustrations of the first imaging system and the second imaging system according to this embodiment are omitted, the arrangement, function, and configuration of the first imaging system and the second imaging system in this embodiment will be understood by those skilled in the art from the corresponding parts in the description of the first embodiment, FIG. 4, FIG. 5, and the like.

[0213] Furthermore, in some exemplary aspects, the imaging by the first imaging system and the imaging by the second imaging system can be performed synchronously with each other.

[0214] In addition, some exemplary embodiments may include an image selection unit having the same arrangement, function, and configuration as the image selection unit 1050 of the first embodiment. The image selection unit of this embodiment is configured to select a new series of Scheimpflug images corresponding to the slit scan performed by the first image acquisition unit 2010 from the first Scheimpflug image group and the second Scheimpflug image group based on a correspondence relationship between the first Scheimpflug image group and the second Scheimpflug image group based on the synchronization between the shooting by the first imaging system and the shooting by the second imaging system. Although an illustration of the image selection unit according to this embodiment is omitted, the arrangement, function, and configuration of the image selection unit in this embodiment will be understandable to those skilled in the art from the corresponding parts in the description of the first embodiment, FIG. 4, FIG. 5, etc.

[0215] The identifier assigner 2040 of this embodiment may be configured to assign an ocular tissue identifier to each of the new set of Scheimpflug images formed by the image selector. The assignment of ocular tissue identifiers to each Scheimpflug image selected by the image selector from the first and second set of Scheimpflug images may be performed in the same manner as the assignment of ocular tissue identifiers to the first image collector 2010.

[0216] In some exemplary embodiments, the first image acquisition unit 2010 may further include an illumination system that projects a slit light onto a three-dimensional region of the subject's eye. Furthermore, the optical axis of the first imaging system and the optical axis of the second imaging system may be arranged to be inclined in opposite directions to each other with respect to the optical axis of the illumination system. In addition, the image selection unit may select a new series of Scheimpflug images by selecting a plurality of Scheimpflug images that do not include corneal reflection artifacts from the first Scheimpflug image group and the second Scheimpflug image group based on the correspondence relationship between the first Scheimpflug image group and the second Scheimpflug image group. Note that although illustration of the illumination system according to this embodiment is omitted, those skilled in the art will be able to understand the arrangement, function, and configuration of the illumination system in this embodiment from the corresponding parts in the description of the first embodiment, FIG. 5, and the like.

[0217] In some exemplary embodiments, the first image acquisition unit 2010 may be configured to perform a slit scan of a three-dimensional region of the subject's eye by translating the slit light in a direction perpendicular to the longitudinal direction of the slit light. Here, the longitudinal direction of the slit light may be substantially aligned with the subject's body axis direction. In addition, the dimension of the slit light in the longitudinal direction may be equal to or larger than the corneal diameter in the subject's body axis direction, and the distance of translation of the slit light may be equal to or larger than the corneal diameter in the direction perpendicular to the subject's body axis direction.

[0218] A usage form of the ophthalmologic imaging apparatus 2000 according to the second aspect will be described with reference to Fig. 11. The usage forms described below are merely examples. For example, any of the items described in the outline of the above-mentioned embodiment, any of the items described in the first aspect, and any of the items described in the second aspect can be combined with the usage forms described below. It is assumed that preparatory operations have been completed.

[0219] Upon receiving an instruction to start photographing, the ophthalmologic photographing apparatus 2000 causes the first image acquisition unit 2010 and the second image acquisition unit 2020 to acquire a series of Scheimpflug images and a series of time-series images in parallel (S21).

[0220] A series of Scheimpflug images collected by the first image collection unit 2010 and a series of time-series images collected by the second image collection unit 2020 are stored, for example, in a memory unit (not shown) provided in the ophthalmic imaging apparatus 2000 and / or a storage device (not shown) connected to the ophthalmic imaging apparatus 2000.

[0221] Next, the ophthalmologic photographing apparatus 2000 causes the second image analysis unit 2030 to analyze the series of time-series images collected by the second image collection unit 2020 in step S21 to detect a series of slit light images (S22).

[0222] Next, the ophthalmic imaging apparatus 2000 assigns, to each of the series of Scheimpflug images collected in step S21 based on the series of slit light images detected in step S22, an identifier indicating the ocular tissue onto which the slit light was projected when the Scheimpflug image was acquired (S23).

[0223] The ocular tissue identifier assigned to each Scheimpflug image by the identifier assigning unit 2040 is stored in association with the corresponding Scheimpflug image. For example, the identifier assigning unit 2040 can embed the ocular tissue identifier as an image or code in the Scheimpflug image. In this case, the Scheimpflug image with the embedded ocular tissue identifier is stored. As another example, when managing images using the DICOM (Digital Imaging and COmmunications in Medicine) standard, the identifier assigning unit 2040 can record the ocular tissue identifier in the tag information of the Scheimpflug image. In this case, the ocular tissue identifier is stored as tag information attached to the Scheimpflug image.

[0224] In an embodiment in which the first image collection unit 2010 includes a first imaging system (1011) and a second imaging system (1012), the ophthalmic imaging apparatus 2000 can use the image selection unit (1050) to form a new series of Scheimpflug images (e.g., a plurality of Scheimpflug images not contaminated by corneal reflection artifacts) from a first group of Scheimpflug images collected by the first imaging system and a second group of Scheimpflug images collected by the second imaging system, and further use the identifier assignment unit 2040 to assign an ocular tissue identifier to each of the new series of Scheimpflug images.

[0225] The new set of Scheimpflug images and their associated ocular tissue identifiers are stored, for example, in a memory unit of the ophthalmic imaging device 2000 and / or in a storage device connected to the ophthalmic imaging device 2000 .

[0226] Some embodiments may be configured to perform the following sequence of operations: assigning an ocular tissue identifier to each of the first set of Scheimpflug images; assigning an ocular tissue identifier to each of the second set of Scheimpflug images; and selecting a new set of Scheimpflug images, each of which has been assigned an ocular tissue identifier, from the first set of Scheimpflug images, each of which has been assigned an ocular tissue identifier, and from the second set of Scheimpflug images, each of which has been assigned an ocular tissue identifier. Thus, in the second embodiment, the ocular tissue identifier assignment process may be performed after the image selection process, or may be performed before the image selection process. Note that the image selection process and the ocular tissue identifier assignment process may be performed at least partially in parallel.

[0227] Next, the ophthalmic imaging apparatus 2000 or an image processing device not shown reads out the data stored in the memory unit of the ophthalmic imaging apparatus 2000 and / or a storage device connected to the ophthalmic imaging apparatus 2000, and can display the Scheimpflug image together with the ophthalmic tissue identifier on a display device, or can display the Scheimpflug image on the display device based on the ophthalmic tissue identifier (S24).

[0228] For example, the ophthalmic imaging apparatus 2000 or an image processing device (not shown) can display the name of the ocular tissue (such as a character string) indicated in the ocular tissue identifier together with the Scheimpflug image. In another example, the ophthalmic imaging apparatus 2000 or an image processing device (not shown) can select a group of Scheimpflug images to which the same ocular tissue identifier is assigned from a series of Scheimpflug images, and display the selected group of Scheimpflug images selectively or as a list. In yet another example, the ophthalmic imaging apparatus 2000 or an image processing device (not shown) can select a group of Scheimpflug images to which the same ocular tissue identifier is assigned from a series of Scheimpflug images, construct a three-dimensional image from the selected group of Scheimpflug images, and display a rendering image of the selected Scheimpflug images.

[0229] In an embodiment in which an ocular tissue identifier is assigned to a partial region of a Scheimpflug image corresponding to ocular tissue, the ophthalmic imaging device 2000 or an image processing device not shown can display information indicating the region of ocular tissue indicated in the ocular tissue identifier (such as an image showing the outline of the region) together with the Scheimpflug image.

[0230] In a case where an ocular tissue identifier is not assigned to a partial region of the Scheimpflug image corresponding to ocular tissue, the ophthalmic imaging device 2000 or an image processing device not shown can analyze (segment) the Scheimpflug image by referring to the ocular tissue indicated in the ocular tissue identifier to identify the partial region corresponding to the ocular tissue, and display information indicating the identified partial region together with the Scheimpflug image.

[0231] In addition to or instead of displaying the Scheimpflug image, the ophthalmic imaging apparatus 2000 or an image processing device (not shown) can analyze the Scheimpflug image with reference to the ocular tissue identifier (S24).

[0232] For example, the ophthalmic imaging apparatus 2000 or an image processing device (not shown) can select a subregion in the Scheimpflug image based on the ocular tissue identifier and apply image analysis to the selected subregion. This allows the region to which the image analysis is applied to be smaller, resulting in advantages such as a shorter processing time and a reduced processing resource. In an embodiment in which an ocular tissue identifier is assigned to a subregion of the Scheimpflug image corresponding to an ocular tissue, a similar process can be easily performed.

[0233] In another example, the ophthalmic imaging apparatus 2000 or an image processing device (not shown) can select an image analysis program from a program library based on the ocular tissue identifier and apply image analysis to the Scheimpflug image using the selected image analysis program. This allows accurate selection of an appropriate image analysis program for analyzing the Scheimpflug image from among various image analysis programs, thereby facilitating automation of the analysis of the Scheimpflug image.

[0234] According to the second aspect that can realize such a usage form, it is possible to assign to each Scheimpflug image acquired by slit scanning an identifier of the eye tissue onto which the slit light was projected when the Scheimpflug image was acquired. Therefore, the user can easily and correctly grasp the type of eye tissue depicted in the image used in the action such as observation or diagnosis or the process such as analysis. Also, it is possible to more suitably analyze the Scheimpflug image. In addition to these effects, the second aspect achieves various effects described in the present disclosure. Moreover, a person skilled in the art would be able to understand effects not explicitly stated in the present disclosure based on the contents of the present disclosure.

[0235] <Third aspect> The third aspect provides an embodiment including a combination of the first and second aspects. An example of an ophthalmic imaging apparatus according to the third aspect is shown in Fig. 12. The ophthalmic imaging apparatus of this aspect is a system (slit lamp microscope system 1) that combines a slit lamp microscope and a computer (information processing device).

[0236] In this embodiment, the slit lamp microscope system 1 is used for photographing the anterior segment of the subject's eye E, and includes an illumination system 2, an imaging system 3, a video imaging system 4, an optical path coupling element 5, a moving mechanism 6, a control unit 7, a data processing unit 8, a communication unit 9, and a user interface 10. The cornea of ​​the subject's eye E is indicated by the symbol C, and the crystalline lens is indicated by the symbol CL.

[0237] As a non-limiting example of the arrangement of the elements of the slit lamp microscope system 1, the slit lamp microscope system 1 of some exemplary embodiments includes a microscope body, a computer, and a communication device that handles communication between the microscope body and the computer. The microscope body includes an illumination system 2, an imaging system 3, a video imaging system 4, an optical path coupling element 5, and a moving mechanism 6. The computer includes a control unit 7, a data processing unit 8, a communication unit 9, and a user interface 10. The computer may be installed, for example, near the microscope body or on a network.

[0238] A combination of the illumination system 2, the imaging system 3, and the moving mechanism 6 is an example of the first image acquisition unit 1010 of the first embodiment, and an example of the first image acquisition unit 2010 of the second embodiment. The illumination system 2 is an example of the illumination system 1013 of the first embodiment. The video imaging system 4 is an example of the second image acquisition unit 1020 of the first embodiment, and an example of the second image acquisition unit 2020 of the second embodiment.

[0239] The illumination system 2 projects slit light onto the anterior segment of the subject's eye E. Reference symbol 2a indicates the optical axis (illumination optical axis) of the illumination system 2. The illumination system 2 may have the same configuration as the illumination system of a conventional slit lamp microscope. For example, although not shown, the illumination system 2 includes, in order from the side farthest from the subject's eye E, an illumination light source, a positive lens, a slit forming unit, and an objective lens. The illumination light output from the illumination light source passes through the positive lens and is projected onto the slit forming unit. The slit forming unit generates slit light by passing a part of the illumination light. The slit forming unit has a pair of slit blades. The width of the slit light can be changed by changing the interval (slit width) between these slit blades. In addition, the longitudinal direction of the slit light can be changed by rotating the pair of slit blades. In addition, the slit forming unit can change the longitudinal dimension of the slit light. The slit light generated by the slit forming unit is refracted by the objective lens and projected onto the anterior segment of the subject's eye E.

[0240] The illumination system 2 may include a focusing mechanism for changing the focus position of the slit light. For example, the focusing mechanism moves an objective lens along the illumination optical axis 2a. Alternatively, the focusing mechanism moves a focusing lens disposed between the objective lens and the slit forming portion.

[0241] 12 is a top view, in which the direction along the axis of the subject's eye E is the Z direction, the direction perpendicular to this, that is, the left-right direction for the subject, is the X direction, and the direction perpendicular to both the X and Z directions (up-down direction, body axis direction) is the Y direction. In this embodiment, the slit lamp microscope system 1 can be aligned with the subject's eye E so that the illumination optical axis 2a coincides with the axis of the subject's eye E, and more broadly, the alignment can be performed so that the illumination optical axis 2a is disposed parallel to the axis of the subject's eye E.

[0242] The photographing system 3 photographs the anterior segment of the eye onto which the slit light from the illumination system 2 is projected. Reference symbol 3a indicates the optical axis (photographing optical axis) of the photographing system 3. The photographing system 3 includes an optical system 3A and an image sensor 3B. The optical system 3A guides light from the anterior segment of the subject's eye E onto which the slit light is projected, to the image sensor 3B. The optical system 3A may have a configuration similar to that of the photographing system of a conventional slit lamp microscope. For example, the optical system 3A includes, in order from the side closer to the subject's eye E, an objective lens, a variable magnification optical system, and an imaging lens. The light from the anterior segment of the subject's eye E onto which the slit light is projected passes through the objective lens and the variable magnification optical system, and is imaged on the imaging surface of the image sensor 3B by the imaging lens. The image sensor 3B receives the light guided by the optical system 3A on its imaging surface. The image sensor 3B includes an area sensor having a two-dimensional imaging area. The area sensor may be, for example, a charge-coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor.

[0243] The imaging system 3 may include a focusing mechanism for changing the focus position. For example, the focusing mechanism moves an objective lens along the imaging optical axis 3a. Alternatively, the focusing mechanism moves a focusing lens disposed between the objective lens and the imaging lens along the imaging optical axis 3a.

[0244] The illumination system 2 and the photographing system 3 function as a Scheimpflug camera. That is, the illumination system 2 and the photographing system 3 are configured so that the object plane along the illumination optical axis 2a, the optical system 3A, and the imaging plane of the imaging element 3B satisfy the so-called Scheimpflug condition. More specifically, the YZ plane (including the object plane) passing through the illumination optical axis 2a, the principal plane of the optical system 3A, and the imaging plane of the imaging element 3B intersect on the same straight line. This allows photographing to be performed by focusing on all positions in the object plane (all positions in the direction along the illumination optical axis 2a).

[0245] In this embodiment, the illumination system 2 and the imaging system 3 are configured so that the imaging system 3 is focused at least in the range from the front surface of the cornea C to the rear surface of the crystalline lens CL. That is, the slit lamp microscope system 1 can capture an image of the anterior part of the subject's eye E in a state in which the imaging system 3 is focused on the entire range from the apex (Z=Z1) of the front surface of the cornea C to the apex (Z=Z2) of the rear surface of the crystalline lens CL. The intersection of the illumination optical axis 2a and the imaging optical axis 3a is located at the coordinate Z=Z0. Such a condition is realized according to the configuration and arrangement of the elements included in the illumination system 2, the configuration and arrangement of the elements included in the imaging system 3, and the relative positions of the illumination system 2 and the imaging system 3. The parameters indicating the relative positions of the illumination system 2 and the imaging system 3 include, for example, the angle θ between the illumination optical axis 2a and the imaging optical axis 3a. The angle θ is set to, for example, 17.5 degrees, 30 degrees, or 45 degrees. The angle θ may be variable.

[0246] The moving image capturing system 4 captures a moving image of the anterior segment of the subject's eye E in parallel with the imaging of the subject's eye by the illumination system 2 and the imaging system 3. The moving image capturing system 4 functions as a video camera.

[0247] The optical path combining element 5 combines the optical path (illumination optical path) of the illumination system 2 with the optical path (moving image capturing optical path) of the moving image capturing system 4. The optical path combining element 5 may be, for example, a beam splitter such as a half mirror or a dichroic mirror.

[0248] A specific example of an optical system including an illumination system 2, an imaging system 3, a video imaging system 4, and an optical path coupling element 5 is shown in Fig. 13. In this example, the imaging system 3 includes two imaging systems (a first imaging system and a second imaging system). In some exemplary embodiments, the optical system of the slit lamp microscope system 1 may include other elements (e.g., any element in the first embodiment, any element in the second embodiment) in addition to or instead of any of the elements shown in Fig. 13.

[0249] 13 includes an illumination system 20, a left imaging system 30L, a right imaging system 30R, and a video imaging system 40. The illumination system 20 is an example of the illumination system 2. The combination of the left imaging system 30L and the right imaging system 30R is an example of the imaging system 3, and is an example of the combination of the first imaging system 1011 and the second imaging system 1012 of the first aspect. The video imaging system 40 is an example of the video imaging system 4. The beam splitter 47 is an example of the optical path coupling element 5.

[0250] In FIG. 13, the reference symbol 20a indicates the optical axis (illumination optical axis) of the illumination system 20, the reference symbol 30La indicates the optical axis (left photographing optical axis) of the left photographing system 30L, and the reference symbol 30Ra indicates the optical axis (right photographing optical axis) of the right photographing system 30R. The direction of the left photographing optical axis 30La and the direction of the right photographing optical axis 30Ra are different from each other. The angle between the illumination optical axis 20a and the left photographing optical axis 30La is indicated by θL, and the angle between the illumination optical axis 20a and the right photographing optical axis 30Ra is indicated by θR. The angles θL and θR may be equal to each other or different from each other. Each of the angles θL and θR may be variable. The illumination optical axis 20a, the left photographing optical axis 30La, and the right photographing optical axis 30Ra intersect at one point. As in FIG. 12, the Z coordinate of this intersection is indicated by Z0.

[0251] The moving mechanism 6 in this example is configured to move the illumination system 20, the left imaging system 30L, and the right imaging system 30R in a direction (X direction) indicated by an arrow 49. In some exemplary embodiments, the illumination system 20, the left imaging system 30L, and the right imaging system 30R are placed on a stage that is movable at least in the X direction, and the moving mechanism 6 moves this movable stage in the X direction according to a control signal from the control unit 7.

[0252] The illumination system 20 projects slit light onto the anterior segment of the subject's eye E. Similar to the illumination system of a conventional slit lamp microscope, the illumination system 20 includes, in order from the side farthest from the subject's eye E, an illumination light source 21, a positive lens 22, a slit forming unit 23, and objective lens groups 24 and 25.

[0253] Illumination light (e.g., visible light) output from the illumination light source 21 is refracted by the positive lens 22 and projected onto the slit forming unit 23. A part of the projected illumination light passes through the slit formed by the slit forming unit 23 to become slit light. The generated slit light is refracted by the objective lens groups 24 and 25, and then reflected by the beam splitter 47 and projected onto the anterior segment of the subject's eye E.

[0254] The left photographing system 30L includes a reflector 31L, an imaging lens 32L, and an image sensor 33L. The reflector 31L and the imaging lens 32L guide light from the anterior segment onto which the slit light is projected by the illumination system 20 (light traveling in the direction of the left photographing system 30L) to the image sensor 33L.

[0255] The light traveling from the anterior eye toward the left imaging system 30L is the light from the anterior eye onto which the slit light is projected, and travels in a direction away from the illumination optical axis 20a. The reflector 31L reflects the light in a direction approaching the illumination optical axis 20a. The imaging lens 32L refracts the light reflected by the reflector 31L to form an image on the imaging surface 34L of the imaging element 33L. The imaging element 33L receives the light on the imaging surface 34L.

[0256] The left imaging system 30L repeatedly captures images in parallel with the movement of the illumination system 20, the left imaging system 30L, and the right imaging system 30R by the movement mechanism 6. This allows a plurality of anterior eye images (a series of Scheimpflug images) to be obtained.

[0257] The object plane along the illumination optical axis 20a, the optical system including the reflector 31L and the imaging lens 32L, and the imaging plane 34L satisfy the Scheimpflug condition. More specifically, when the deflection of the optical path of the imaging system 30L by the reflector 31L is taken into consideration, the YZ plane (including the object plane) passing through the illumination optical axis 20a, the principal plane of the imaging lens 32L, and the imaging plane 34L intersect on the same straight line. This allows the left imaging system 30L to focus on all positions within the object plane (for example, in the range from the anterior surface of the cornea to the posterior surface of the crystalline lens) and perform imaging.

[0258] The right photographing system 30R includes a reflector 31R, an imaging lens 32R, and an image sensor 33R. The reflector 31R and the imaging lens 32R guide light (light traveling toward the right photographing system 30R) from the anterior eye segment onto which the slit light is projected by the illumination system 20 to the image sensor 33R. The right photographing system 30R repeatedly captures images in parallel with the movement of the illumination system 20, the left photographing system 30L, and the right photographing system 30R by the moving mechanism 6, thereby acquiring multiple anterior eye images (a series of Scheimpflug images). The object surface along the illumination optical axis 20a, the optical system including the reflector 31R and the imaging lens 32R, and the image sensor 34R satisfy the Scheimpflug condition.

[0259] The Scheimpflug image acquisition by the left imaging system 30L and the Scheimpflug image acquisition by the right imaging system 30R are performed in parallel with each other. A combination of a series of Scheimpflug images acquired by the left imaging system 30L and a series of Scheimpflug images acquired by the right imaging system 30R corresponds to a combination of a first Scheimpflug image group and a second Scheimpflug image group.

[0260] The control unit 7 can synchronize the repeated photographing by the left imaging system 30L and the repeated photographing by the right imaging system 30R. This allows a correspondence relationship to be obtained between a series of Scheimpflug images obtained by the left imaging system 30L and a series of Scheimpflug images obtained by the right imaging system 30R. This correspondence relationship is a temporal correspondence relationship, and more specifically, it is a pairing of images acquired substantially simultaneously.

[0261] Alternatively, the control unit 7 or the data processing unit 8 can execute a process for determining a correspondence relationship between a plurality of anterior-segment images obtained by the left photographing system 30L and a plurality of anterior-segment images obtained by the right photographing system 30R. For example, the control unit 7 or the data processing unit 8 can pair the anterior-segment images sequentially input from the left photographing system 30L and the anterior-segment images sequentially input from the right photographing system 30R based on their input timing.

[0262] The video capture system 40 captures video of the anterior segment of the subject's eye E from a fixed position in parallel with the capture by the left capture system 30L and the capture by the right capture system 30R. Here, the video capture system 40 is not moved by the moving mechanism 6. The video capture system 40 is disposed coaxially with the illumination system 20, but the arrangement is not limited thereto. For example, the video capture system can be disposed non-coaxially with the illumination system 20.

[0263] The light transmitted through the beam splitter 47 is reflected by a reflector 48 and enters the moving image capturing system 40. The light entering the moving image capturing system 40 is refracted by an objective lens 41, and then an image is formed on the imaging surface of an imaging element 43 by an imaging lens 42. The imaging element 43 is an area sensor.

[0264] The video capture system 40 can be used for monitoring, aligning, tracking, etc. the movement of the subject's eye E. Furthermore, the video capture system 40 can be used to process a series of Scheimpflug images.

[0265] Returning to Fig. 12, the movement mechanism 6 is configured to move the illumination system 2 and the imaging system 3 together in the X direction.

[0266] The control unit 7 is configured to control each unit of the slit lamp microscope system 1. For example, the control unit 7 controls elements of the illumination system 2 (illumination light source, slit forming unit, focusing mechanism, etc.), elements of the photography system 3 (focusing mechanism of the optical system 3A, image sensor 3B, etc.), elements of the video photography system 4 (focusing mechanism, image sensor, etc.), the movement mechanism 6, the data processing unit 8, the communication unit 9, the user interface 10, etc.

[0267] The control unit 7 can control the illumination system 2, the imaging system 3, and the moving mechanism 6 in parallel with the control of the video imaging system 4. This makes it possible to execute slit scanning (collection of a series of Scheimpflug images) by the first image acquisition unit 1010 (2010) and repeated imaging (collection of a series of time-series images) by the second image acquisition unit 1020 (2020) in parallel with each other.

[0268] Furthermore, the control unit 7 can execute the controls of the illumination system 2, the imaging system 3, and the moving mechanism 6 in synchronization with the control of the video imaging system 4. This makes it possible to synchronize the slit scan by the first image acquisition unit 1010 (2010) with the repeated imaging by the second image acquisition unit 1020 (2020).

[0269] In a configuration in which the imaging system 3 includes a left imaging system 30L and a right imaging system 30R, the control unit 7 can execute repeated imaging by the left imaging system 30L (collection of a first Scheimpflug image group) and repeated imaging by the right imaging system 30R (collection of a second Scheimpflug image group) in synchronization with each other.

[0270] The control unit 7 includes a processor, a main storage device, an auxiliary storage device, etc. The auxiliary storage device stores computer programs such as various control programs. These computer programs may be stored in a computer or storage device accessible to the slit lamp microscope system 1. The functions of the control unit 7 are realized by cooperation between software such as the control programs and hardware such as a processor.

[0271] In order to scan a three-dimensional area of ​​the anterior segment of the subject's eye E with slit light, the control unit 7 can apply the following controls to the illumination system 2, the imaging system 3, and the moving mechanism 6.

[0272] First, the control unit 7 controls the moving mechanism 6 to place the illumination system 2 and the imaging system 3 at a predetermined scan start position (alignment control). The scan start position is, for example, a position corresponding to an end (first end) of the cornea C in the X direction, or a position further away from the axis of the subject's eye E. The symbol X0 in Fig. 14A indicates the scan start position corresponding to the first end of the cornea C in the X direction. Moreover, the symbol X0' in Fig. 14B indicates a scan start position further away from the axis EA of the subject's eye E than the position corresponding to the first end of the cornea C in the X direction.

[0273] The control unit 7 controls the illumination system 2 to start projecting slit light onto the anterior segment of the subject's eye E (slit light projection control). The control unit 7 also controls the imaging system 3 to start capturing a moving image of the anterior segment of the subject's eye E (imaging control). After executing the alignment control, the slit light projection control, and the imaging control, the control unit 7 controls the moving mechanism 6 to start moving the illumination system 2 and the imaging system 3 (movement control). The illumination system 2 and the imaging system 3 are moved integrally by the movement control. That is, the illumination system 2 and the imaging system 3 are moved while maintaining the relative position (angle θ, etc.) between the illumination system 2 and the imaging system 3 (while satisfying the Scheimpflug condition). The illumination system 2 and the imaging system 3 are moved from the above-mentioned scan start position to a predetermined scan end position. The scan end position is, for example, a position corresponding to the end (second end) of the cornea C on the opposite side of the first end in the X direction, similar to the scan start position, or a position further away from the axis of the subject's eye E than that.

[0274] The slit scan in this example is applied to the range from the scan start position to the scan end position. This slit scan is realized by projecting slit light onto the anterior eye with the X direction as the width direction and the Y direction as the length direction, moving the illumination system 2 and the imaging system 3 in the X direction as a whole, and capturing a moving image by the imaging system 3 in parallel (linked and synchronous). The length of the slit light (i.e., the dimension of the beam cross section of the slit light in the Y direction) is set to, for example, equal to or greater than the diameter of the cornea C on the surface of the subject's eye E. In addition, the moving distance of the illumination system 2 and the imaging system 3 by the moving mechanism 6 is set to equal to or greater than the cornea diameter in the X direction. This makes it possible to apply the slit scan to a three-dimensional area including the entire cornea C.

[0275] By such slit scanning, multiple anterior eye images (a series of Scheimpflug images) with different projection positions of the slit light are obtained. In other words, a moving image is obtained that depicts the movement of the projection position of the slit light in the X direction. An example of such multiple anterior eye images (i.e., a group of frames constituting a moving image) is shown in FIG. 15.

[0276] FIG. 15 shows a plurality of anterior eye images (frame group) F1, F2, F3, . . . , FN. The subscript n of these anterior eye images Fn (n=1, 2, . . . , N) represents the time series order. That is, the anterior eye image acquired n-th is represented by the symbol Fn. The anterior eye image Fn includes a slit light image An. As shown in FIG. 15, the slit light images A1, A2, A3, . . . , AN move rightward along the time series. In the example shown in FIG. 15, the scan start position and the scan end position correspond to both ends of the cornea C in the X direction. Note that the scan start position and / or the scan end position are not limited to this example, and may be, for example, a position farther from the axis of the subject's eye E than the edge of the cornea. In addition, the direction and number of scans can also be set arbitrarily.

[0277] The data processing unit 8 is configured to execute various types of data processing. The data to be processed may be either data acquired by the slit lamp microscope system 1 or data input from the outside.

[0278] The data processing unit 8 includes a processor, a main storage device, an auxiliary storage device, etc. The auxiliary storage device stores computer programs such as various data processing programs. These computer programs may be stored in a computer or storage device accessible to the slit lamp microscope system 1. The functions of the data processing unit 8 are realized by cooperation between software such as the data processing program and hardware such as a processor.

[0279] A configuration example of the data processing unit 8 is shown in Fig. 16. The data processing unit 8A in this example is capable of executing a process combining spatial image interpolation and ocular tissue identifier assignment. The data processing unit 8A includes an image analysis unit 81, an image interpolation unit 82, an identifier assignment unit 83, and a three-dimensional image construction unit 84.

[0280] The image analysis unit 81 has any function related to the first image analysis unit 1030 in the first embodiment and any function related to the second image analysis unit 2030 in the second embodiment. The image analysis unit 81 receives input of a plurality of anterior segment images (a series of Scheimpflug images) collected from a three-dimensional region of the anterior segment of the subject's eye E by slit scanning, and video images (a series of time-series images) acquired by the video imaging system 4 in parallel with the slit scanning.

[0281] For example, as described above, the image analysis unit 81 analyzes a series of time-series images collected by the moving image capture system 4 to detect a series of slit light images, and determines the time-series displacement of the slit light based on the detected series of slit light images. The detection results of the slit light images are sent to the identifier assignment unit 83, and information on the time-series displacement of the slit light is sent to the image interpolation unit 82.

[0282] The image interpolation unit 82 has any function related to the image interpolation unit 1040 in the first embodiment. The image interpolation unit 82 performs interpolation of a series of Scheimpflug images collected by slit scanning, based on the time-series displacement of the slit light acquired by the image analysis unit 81. A plurality of Scheimpflug images obtained by the interpolation process (including the series of collected Scheimpflug images and a group of images added by interpolation) are sent to the identifier assignment unit 83.

[0283] Some example embodiments of the image interpolator 82 may have a configuration similar to the image interpolator 1040A of FIG. 6A and may be configured to perform spatial image interpolation using an inference model.

[0284] The image interpolation unit 82 of some exemplary embodiments includes a configuration similar to that of the image interpolation unit 1040B of Fig. 7 and / or a configuration similar to that of the image interpolation unit 1040C of Fig. 8, and can determine the number of interpolations based on at least one of operation information of the first image acquisition unit (illumination system 2, imaging system 3, and moving mechanism 6), information on the time series displacement of the slit light, and information on the time series displacement of the subject's eye. Furthermore, the image interpolation unit 82 can generate the determined number of images for interpolation and add them to the series of Scheimpflug images.

[0285] The identifier assigning unit 83 has any function related to the identifier assigning unit 2040 of the second embodiment. The identifier assigning unit 83 assigns an ocular tissue identifier to each of the series of Scheimpflug images collected by the slit scan, based on the series of slit light images detected from the series of time-series images collected by the video capturing system 4.

[0286] Furthermore, the identifier assigning unit 83 can assign an ocular tissue identifier to each of the images (additional images) added to the series of Scheimpflug images by the image interpolation unit 82. The process of assigning an ocular tissue identifier to each additional image may be any of the processes described above for assigning an ocular tissue identifier to each Scheimpflug image (each collected image).

[0287] Alternatively, the identifier assigning unit 83 may be configured to assign an ocular tissue identifier to the additional image group based on the ocular tissue identifier assigned to the series of Scheimpflug images. In some exemplary embodiments, the identifier assigning unit 83 may be configured to assign the same ocular tissue identifier to each additional image that is disposed between two adjacent Scheimpflug images (e.g., the two anterior segment images Fn and F(n+1) in FIG. 15) and that are assigned an identifier indicating the same ocular tissue as the ocular tissue identifier assigned to these Scheimpflug images. For example, when an identifier indicating the cornea is assigned to both of the two anterior segment images Fn and F(n+1) in FIG. 15, an identifier indicating the cornea can be assigned to each image inserted between the anterior segment image Fn and the anterior segment image F(n+1) by the image interpolation unit 82.

[0288] In some exemplary embodiments, the identifier assigning unit 83 may first apply a process similar to the process for assigning an ocular tissue identifier to the Scheimpflug image to each additional image (first ocular tissue identifier assigning process). For each additional image to which an ocular tissue identifier is properly assigned in the first ocular tissue identifier assigning process, the process is completed here. For each additional image to which an ocular tissue identifier is not properly assigned in the first ocular tissue identifier assigning process, the identifier assigning unit 83 may assign an ocular tissue identifier based on the ocular tissue identifier assigned to the series of Scheimpflug images (second ocular tissue identifier assigning process). The order of such stepwise processes is not limited to this example, and may be, for example, the reverse order of this example. In addition, any two or more different processes may be performed stepwise.

[0289] In some exemplary embodiments, if an ocular tissue identifier has not been properly assigned to any of the series of Scheimpflug images, the identifier assigning unit 83 may first select one or more additional images located in the vicinity of the Scheimpflug image (e.g., one or more additional images adjacent to the Scheimpflug image). Next, the identifier assigning unit 83 may assign an ocular tissue identifier to the selected additional image by applying a process similar to the process for assigning an ocular tissue identifier to the Scheimpflug image. The identifier assigning unit 83 may then assign an ocular tissue identifier to the target Scheimpflug image based at least on the additional images to which the ocular tissue identifier has been assigned.

[0290] The three-dimensional image construction unit 84 constructs a three-dimensional image based on a plurality of anterior segment images (including a series of Scheimpflug images and a group of additional images) to which an ocular tissue identifier is respectively assigned by the identifier assignment unit 83. A three-dimensional image is an image (image data) in which pixel positions are defined by a three-dimensional coordinate system. An example of a three-dimensional image is volume data (also called voxel data). The three-dimensional image construction unit 84 uses any known three-dimensional image construction technique (such as computer graphics).

[0291] The slit lamp microscope system 1 of this example is configured to perform spatial image interpolation on a series of Scheimpflug images collected from a three-dimensional region of the subject's eye E, and to assign an ocular tissue identifier to each of a plurality of images (e.g., a series of Scheimpflug images and a group of additional images) obtained through the interpolation. According to this example configured in this way, both the effects of the first aspect and the second aspect are achieved. Furthermore, according to this example, since the spatial image interpolation and the assignment of the ocular tissue identifier can be linked, it is possible to further improve the performance of an ophthalmic imaging device capable of performing slit scanning.

[0292] Another configuration example of the data processing unit 8 is shown in Fig. 17. Like the data processing unit 8A in Fig. 16, the data processing unit 8B in this example includes an image analysis unit 81, an image interpolation unit 82, an identifier assignment unit 83, and a three-dimensional image construction unit 84, and further includes an image selection unit 85. Descriptions of elements similar to those in the data processing unit 8A will be omitted.

[0293] The imaging system 3 of this example includes a first imaging system and a second imaging system, such as the left imaging system 30L and the right imaging system 30R in Fig. 13. In this example, the slit scan is performed in parallel to collect a series of Scheimpflug images by the left imaging system 30L and a series of Scheimpflug images by the right imaging system 30R.

[0294] The image selection unit 85 selects a new series of Scheimpflug images corresponding to the slit scan from the two series of Scheimpflug images based on the correspondence between the series of Scheimpflug images collected by the left imaging system 30L and the series of Scheimpflug images collected by the right imaging system 30R (the correspondence between the first Scheimpflug image group and the second Scheimpflug image group). The image selection unit 85 has any function related to the image selection unit 1050 of the first embodiment. The image interpolation unit 82 of this example applies spatial image interpolation to the new series of Scheimpflug images formed by the image selection unit 85.

[0295] The slit lamp microscope system 1 of this example may include an illumination system such as the illumination system 20 of Fig. 13. The image selection unit 85 can select a plurality of Scheimpflug images that do not include corneal reflection artifacts from the two series of Scheimpflug images based on the correspondence between the series of Scheimpflug images collected by the left imaging system 30L and the series of Scheimpflug images collected by the right imaging system 30R. The image interpolation unit 82 of this example applies spatial image interpolation to the new series of Scheimpflug images that do not include corneal reflection artifacts that are thus formed.

[0296] The slit lamp microscope system 1 of this example is configured to perform spatial image interpolation on a new series of Scheimpflug images selected from a group of Scheimpflug images including a series of Scheimpflug images collected by the left imaging system 30L and a series of Scheimpflug images collected by the right imaging system 30R, and to assign an eye tissue identifier to each of a plurality of images (for example, the new series of Scheimpflug images and the additional image group) obtained through the interpolation. According to this example having such a configuration, in addition to the effects of the first and second aspects, the effect of the aspect including the image selection unit 1050 is achieved. Furthermore, according to this example, the Scheimpflug image selection, spatial image interpolation, and eye tissue identifier assignment can be linked, so that it is possible to further improve the performance of an ophthalmic imaging device capable of performing slit scanning.

[0297] The configuration of the data processing unit 8 is not limited to these examples. The data processing unit 8 of some exemplary embodiments may have one or both of the arbitrary data processing function of the first embodiment and the arbitrary data processing function of the second embodiment. In addition, the data processing unit 8 of some exemplary embodiments may have any data processing function related to the technology disclosed by any of the applicants of the present application, such as any data processing function disclosed in Patent Document 3 (JP Patent Publication No. 2019-213733).

[0298] The communication unit 9 performs data communication between the slit lamp microscope system 1 and other devices. That is, the communication unit 9 transmits data to other devices and receives data transmitted from other devices.

[0299] The data communication method executed by the communication unit 9 is arbitrary. For example, the communication unit 9 includes one or more of various communication interfaces such as a communication interface conforming to the Internet, a communication interface conforming to a dedicated line, a communication interface conforming to a LAN, a communication interface conforming to short-distance communication, etc. The data communication may be wired communication or wireless communication.

[0300] The data transmitted and received by the communication unit 9 may be encrypted. In that case, for example, the control unit 7 and / or the data processing unit 8 includes at least one of an encryption processing unit that encrypts the data transmitted by the communication unit 9 and a decryption processing unit that decrypts the data received by the communication unit 9.

[0301] The user interface 10 includes any user interface device such as a display device, an operation device, etc. A user such as a doctor, a subject, or an assistant can operate the slit lamp microscope system 1 and input information to the slit lamp microscope system 1 by using the user interface 10.

[0302] The display device displays various information under the control of the control unit 7. The display device may include a flat panel display such as a liquid crystal display (LCD). The operation device includes a device for operating the slit lamp microscope system 1 and a device for inputting information. The operation device includes, for example, a button, a switch, a lever, a dial, a handle, a knob, a mouse, a keyboard, a trackball, an operation panel, and the like. A device in which the display device and the operation device are integrated, such as a touch screen, may be used.

[0303] At least a part of the user interface may be arranged as a peripheral device of the slit lamp microscope system 1.

[0304] The elements of the slit lamp microscope system 1 are not limited to those described above. The slit lamp microscope system 1 may include any element that can be combined with a slit lamp microscope, or more generally, any element that can be combined with an ophthalmic imaging device. The slit lamp microscope system 1 may also include any element for processing data of the subject's eye acquired by the slit lamp microscope, or more generally, any element for processing any ophthalmic data.

[0305] For example, the slit lamp microscope system 1 may include a fixation system that outputs light (fixation light) for fixating the subject's eye E. The fixation system typically includes at least one visible light source (fixation light source) or a display device that displays an image such as a landscape chart or a fixation target. The fixation system is disposed, for example, coaxially or non-coaxially with the illumination system 2 or the imaging system 3.

[0306] An example of the usage form of the slit lamp microscope system 1 of this embodiment will be described. Fig. 18 is a data flow diagram showing the flow of this example. Each oval in Fig. 18 represents a process or operation, and each rectangle represents data.

[0307] First, the slit lamp microscope system 1 executes video capture 101 using the video capture system 4, and slit scan 102 using the illumination system 2, the capture system 3, and the movement mechanism 6. The control unit 7 executes synchronization control to synchronize the video capture 101 and the slit scan 102 with each other.

[0308] A moving image (a series of time-series images) 103 acquired by moving image shooting 101 is used for eye tracking 104 and detection 106 of a slit light image.

[0309] The eye tracking 104 includes at least a process of analyzing the moving image 103 to detect the movement of the subject's eye E. In other words, the eye tracking 104 includes at least a process of detecting the position of the subject's eye E at each of a plurality of time points. The eye tracking 104 may further include an operation of moving the optical system in real time in accordance with the detected movement of the subject's eye E (an operation of making the optical system follow the movement of the subject's eye E). By the eye tracking 104, a time series displacement 105 of the subject's eye E is obtained.

[0310] Detection 106 of the slit light image includes a process of detecting a series of slit light images by analyzing the moving image 103. By detection 106 of the slit light image, a time series displacement 107 of the slit light image is obtained.

[0311] Artifacts resulting from eye movement of the subject's eye E and artifacts resulting from non-uniform speed of the slit scan can be corrected by the time series displacement 105 of the subject's eye E and the time series displacement 107 of the slit light image. The time series displacement 105 of the subject's eye E and the time series displacement 107 of the slit light image are input to the image interpolation unit 82.

[0312] On the other hand, the slit scan 102 collects a series of Scheimpflug images from a three-dimensional region of the anterior part of the subject's eye E. The slit lamp microscope system 1 of this example includes a left imaging system 30L and a right imaging system 30R shown in FIG. 13 (FIG. 4, FIG. 5). In the slit scan 102, the left imaging system 30L collects a series of Scheimpflug images (left image group 108), and the right imaging system 30R collects a series of Scheimpflug images (right image group 109). In the slit scan 102, the control unit 7 executes synchronization control to synchronize the left imaging system 30L and the right imaging system 30R with each other. Therefore, the control unit 7 executes synchronization control to synchronize the video imaging system 40, the left imaging system 30L, and the right imaging system 30R with each other.

[0313] The left image group 108 and the right image group 109 collected by the slit scan 102 are input to the image selection unit 85. The image selection unit 85 combines the left image group 108 and the right image group 109 (combination of left and right image groups 110). The combination of left and right image groups 110 is a process of selecting a new series of Scheimpflug images corresponding to the slit scan 102 from the left image group 108 and the right image group 109 based on the correspondence between the left image group 108 and the right image group 109.

[0314] By combining the left and right image groups 110, artifacts such as corneal reflection artifacts can be removed. That is, by combining the left and right image groups 110, a new series of Scheimpflug images (composite data 111) that does not include artifacts such as corneal reflection artifacts is formed. The composite data 111 is input to the image interpolation unit 82.

[0315] The image interpolation unit 82 receives input of the time series displacement 105 of the subject's eye E, the time series displacement 107 of the slit light image, and the composite data 111. In the image interpolation 112, the image interpolation unit 82 first determines the conditions of spatial image interpolation based on the time series displacement 105 of the subject's eye E and the time series displacement 107 of the slit light image. The conditions of spatial image interpolation may include, for example, at least one of the number of interpolations, the interpolation position, and the interpolation interval. The image interpolation unit 82 can determine the conditions of spatial image interpolation based on at least one of the conditions of the slit scan 102 (the operation information of the first image collection unit 1010 described above), the time series displacement 105 of the subject's eye E, and the time series displacement 107 of the slit light image.

[0316] In image interpolation 112, the image interpolator 82 further applies spatial image interpolation to the composite data 111 based on the conditions determined in the previous step. This spatial image interpolation can be performed without changing the spacing between the images that make up the composite data 111.

[0317] It should be noted that some exemplary aspects of spatial image interpolation may allow fine adjustment of the spacing between the multiple images that make up the composite data 111. This fine adjustment is performed without substantially changing the aspect ratio in the XY plane of the three-dimensional area represented by the composite data 111 (left image group 108, right image group 109).

[0318] The multiple images (including the synthesis data 111 and the additional images) obtained by the image interpolation 112 are input to the identifier assignment unit 83.

[0319] The identifier assigning unit 83 assigns an eye tissue identifier to each of the multiple images (including the synthetic data 111 and the additional image group) obtained by the image interpolation 112 (identifier assignment 113). The multiple images to which the eye tissue identifiers have been assigned are input to the three-dimensional image constructing unit 84.

[0320] The three-dimensional image constructing unit 84 constructs a three-dimensional image (reconstruction 114 of a three-dimensional image) based on the multiple images (including the composite data 111 and the additional image group) input from the identifier assigning unit 83. This results in a three-dimensional volume 115 expressing a three-dimensional region of the anterior part of the subject's eye E (a three-dimensional region to which the slit scan 102 has been applied). The three-dimensional volume 115 is used for rendering and image analysis.

[0321] By performing image interpolation 112, the aspect ratio of the three-dimensional volume and the aspect ratio of its rendering image can be made similar to that of the actual subject's eye E (the three-dimensional region to which the slit scan 102 is applied).

[0322] Furthermore, in the image interpolation 112, the number of interpolated images can be changed for each interval of the array of a plurality of images constituting the composite data 111, based on the time series displacement 105 of the subject's eye E and / or the time series displacement 107 of the slit light image. This makes it possible to correct artifacts resulting from the eye movement of the subject's eye E that occur when the slit scan 102 is applied to the subject's eye E and artifacts resulting from the non-uniform speed of the slit scan 102.

[0323] Furthermore, by forming the composite data 111, artifacts resulting from the corneal reflection of the slit light can be removed.

[0324] Thus, according to this example, it is possible to generate a three-dimensional image that has an aspect ratio according to the actual shape of the anterior eye and does not include various artifacts such as artifacts caused by corneal reflex, artifacts caused by eye movement, and artifacts caused by non-uniform speed of slit scanning. Furthermore, from such a three-dimensional image, a rendering image that has an aspect ratio according to the actual shape of the anterior eye and does not include various artifacts can be created. In addition, image analysis of the subject's eye E can be performed using a three-dimensional image or a rendering image that has an aspect ratio according to the actual shape of the anterior eye and does not include various artifacts. Therefore, it is possible to improve the performance of slit scanning of the three-dimensional region of the subject's eye.

[0325] Moreover, according to this example, since an ocular tissue identifier can be assigned to each image constituting the composite data 111, the user can easily and correctly grasp the type of ocular tissue depicted in the image used in the action of observation, diagnosis, etc., or in the process of analysis, etc. Also, the Scheimpflug image can be more suitably analyzed.

[0326] The use of the slit lamp microscope system 1 of this embodiment is not limited to this example. For example, some exemplary use modes may include any feature of the first embodiment and / or any feature of the second embodiment.

[0327] <Fourth aspect> The first to third aspects relate to an ophthalmologic imaging apparatus having a function of collecting a series of Scheimpflug images and a function of collecting a series of time-series images, whereas the fourth aspect relates to an ophthalmologic image processing apparatus (computer, information processing apparatus) that receives an image of a subject's eye from outside.

[0328] An ophthalmic image processing device according to some exemplary embodiments includes a receiving unit, a first image analysis unit, and an image interpolation unit.

[0329] The reception unit receives a series of images acquired by scanning a three-dimensional region of the subject's eye with a scanning light, and a series of time-series images collected by repeatedly photographing the subject's eye in parallel with the scanning of the three-dimensional region. The reception unit includes, for example, a communication device and / or a media drive. The communication device may be, for example, the communication unit 9 of the third aspect, and receives data stored in an external storage device. The media drive reads out data recorded on a recording medium.

[0330] The series of images acquired by scanning the three-dimensional region of the subject's eye with the scanning light may be, for example, a series of Scheimpflug images collected by slit scanning, or a new series of Scheimpflug images formed from a first Scheimpflug image group and a second Scheimpflug image group collected by the first imaging system and the second imaging system, respectively. Alternatively, the series of images acquired by scanning the three-dimensional region of the subject's eye with the scanning light may be acquired by a modality other than a slit lamp microscope, and may be, for example, a series of images collected by any ophthalmic scanning modality such as OCT, SLO, etc.

[0331] The first image analysis unit of the ophthalmologic image processing device analyzes the series of time-series images received by the reception unit to obtain the time-series displacement of the scanning light. The first image analysis unit may be, for example, the first image analysis unit 1030 of the first aspect or the image analysis unit 81 of the third aspect.

[0332] The image interpolation unit of the ophthalmologic image processing device performs interpolation of a series of images acquired by scanning with the scanning light, based on the time-series displacement of the scanning light obtained by the first image analysis unit. The image interpolation unit may be, for example, the image interpolation unit 1040 of the first aspect or the image interpolation unit 82 of the third aspect.

[0333] According to the ophthalmologic image processing device of this aspect, it is possible to obtain a plurality of images arranged with an aspect ratio according to the actual shape of the subject's eye, and it is possible to obtain a three-dimensional image or a rendering image with an aspect ratio according to the actual shape of the subject's eye. In addition, any of the features of the present disclosure can be combined with the ophthalmologic image processing device of this aspect.

[0334] An ophthalmic image processing device according to some exemplary embodiments includes a reception unit, a second image analysis unit, and an identifier assignment unit.

[0335] The receiving unit receives a series of images acquired by scanning a three-dimensional region of the test eye with a scanning light, and a series of time-series images collected by repeatedly photographing the test eye in parallel with the scanning of the three-dimensional region. The receiving unit of the ophthalmic image processing device of this aspect may be the same as the receiving unit of the ophthalmic image processing device of the above aspect.

[0336] The second image analysis unit of the ophthalmologic image processing device of this embodiment analyzes the series of time-series images received by the reception unit to detect a series of images of the scanning light. The second image analysis unit may be, for example, the second image analysis unit 2030 of the second embodiment or the image analysis unit 81 of the third embodiment.

[0337] The identifier assigning unit of the ophthalmologic image processing device of this embodiment assigns, to each of a series of images acquired by scanning a three-dimensional region of the subject's eye with the scanning light, an identifier indicating the ocular tissue onto which the scanning light was projected when the image was acquired, based on a series of images of the scanning light detected by the second image analysis unit. The identifier assigning unit may be, for example, the identifier assigning unit 2040 of the second embodiment or the identifier assigning unit 83 of the third embodiment.

[0338] According to the ophthalmologic image processing device of this embodiment, an ocular tissue identifier can be assigned to each of a series of images acquired by scanning a three-dimensional region of a subject's eye with a scanning light, so that a user can easily and correctly grasp the type of ocular tissue depicted in the images used in an action such as observation or diagnosis, or a process such as analysis, etc. Also, it becomes possible to more appropriately analyze a Scheimpflug image.

[0339] The ophthalmologic image processing device of the exemplary embodiment is not limited to the above-mentioned examples. For example, an ophthalmologic image processing device including any of the features of the present disclosure may be configured.

[0340] <Other matters> This disclosure presents some exemplary aspects of the embodiment. These aspects are merely illustrative of the present invention. Therefore, any modification (omission, substitution, addition, etc.) within the scope of the gist of the present invention can be applied to this disclosure.

[0341] It is possible to configure a program that causes a computer to execute any one or more of the processes described in this disclosure. It is also possible to create a recording medium on which such a program is recorded. The recording medium is a non-transitory recording medium that can be read by a computer. Such a recording medium may take any form, and examples of such a recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

[0342] The present invention may include a method including any one or more steps described in this disclosure. For example, a method according to some exemplary embodiments is configured to scan a three-dimensional region of a subject's eye with a slit light to collect a series of Scheimpflug images, repeatedly photograph the subject's eye in parallel with the scanning of the three-dimensional region to collect a series of time-series images, analyze the series of time-series images to determine the time-series displacement of the slit light in the scan, and perform interpolation of the series of Scheimpflug images based on the time-series displacement of the slit light. Any step based on any aspect of this disclosure may be combined with this method. Also, instead of the step of collecting an image, a step of receiving an image from an external source may be provided.

[0343] A method according to some exemplary aspects is configured to collect a series of Scheimpflug images by scanning a three-dimensional region of a subject's eye with a slit light, collect a series of time-series images by repeatedly photographing the subject's eye in parallel with the scanning of the three-dimensional region, analyze the series of time-series images to detect a series of images of the slit light, and assign to each of the series of Scheimpflug images an identifier indicating the ocular tissue to which the slit light was projected when the Scheimpflug image was acquired based on the series of images of the slit light detected from the series of time-series images. Any step based on any aspect of the present disclosure can be combined with this method. Also, instead of the step of collecting images, a step of receiving an image from an external source may be provided. [Explanation of symbols]

[0344] 1000, 1000A Ophthalmic Imaging Device 1010 First Image Acquisition Unit 1011 First camera system 1012 2nd Camera System 1013 Lighting 1020 2nd Image Acquisition Section 1030 First Image Analysis Unit 1040, 1040A, 1040B, 1040C Image Interpolation Unit 1041 Reasoning part 1042 Inference Model 1043 First Interpolation Number Decision Unit 1044 Second Interpolation Number Decision Unit 1050 Image Selection 2020 Ophthalmic imaging equipment 2010 1st Image Acquisition Division 2020 2nd Image Acquisition Division 2030 2nd Image Analysis Division 2040 Identifier Assignment Department

Claims

1. a first image acquisition unit that scans a three-dimensional region of the subject's eye with a slit light to acquire a series of Scheimpflug images; a second image acquisition unit that acquires a series of time-series images by repeatedly capturing images of the subject's eye in parallel with the scanning of the three-dimensional region by the first image acquisition unit; a first image analysis unit that analyzes the series of time-series images to determine a time-series displacement of the slit light in the scan; an image interpolation unit that performs interpolation of the series of Scheimpflug images based on the time-series displacement of the slit light; Including, The first image analysis unit is analyzing the series of time series images to detect a series of images of the slit light; determining the time series displacement of the slit light based on the series of images of the slit light; Ophthalmic imaging equipment.

2. the scanning of the three-dimensional region by the first image acquisition unit and the repeated photographing of the subject's eye by the second image acquisition unit are performed in synchronization with each other; the image interpolation unit performs the interpolation of the series of Scheimpflug images further based on a correspondence relationship between the series of Scheimpflug images and the series of time-series images based on synchronization between the scan and the repeated photographing; 2. An ophthalmologic photographing apparatus according to claim 1.

3. the first image acquisition unit includes a first imaging system and a second imaging system that respectively capture images of the subject's eye from different directions during the scan, the first imaging system acquires a first set of Scheimpflug images; the second imaging system acquires a second set of Scheimpflug images; the series of Scheimpflug images includes the first group of Scheimpflug images and the second group of Scheimpflug images; 3. An ophthalmologic photographing apparatus according to claim 1 or 2.

4. The imaging by the first imaging system and the imaging by the second imaging system are performed in synchronization with each other.

4. An ophthalmologic photographing apparatus according to claim 3.

5. an image selection unit that selects a new series of Scheimpflug images corresponding to the scan from the first Scheimpflug image group and the second Scheimpflug image group based on a correspondence relationship between the first Scheimpflug image group and the second Scheimpflug image group based on synchronization between the photographing by the first photographing system and the photographing by the second photographing system, the image interpolation unit performs an interpolation of the new series of Scheimpflug images.

5. An ophthalmologic photographing apparatus according to claim 4.

6. the first image acquisition unit further includes an illumination system that projects the slit light onto the three-dimensional area; an optical axis of the first imaging system and an optical axis of the second imaging system are arranged to be inclined in opposite directions to each other with respect to an optical axis of the illumination system; the image selection unit selects a plurality of Scheimpflug images that do not include a corneal reflection artifact from the first Scheimpflug image group and the second Scheimpflug image group based on the correspondence relationship between the first Scheimpflug image group and the second Scheimpflug image group, thereby selecting the new series of Scheimpflug images.

6. An ophthalmologic photographing apparatus according to claim 5.

7. the image interpolation unit performs the interpolation of the series of Scheimpflug images using an optical flow that expresses, by a vector, a displacement of the subject's eye depicted in the series of Scheimpflug images.

7. An ophthalmologic photographing apparatus according to claim 1.

8. the image interpolation unit performs the interpolation of the series of Scheimpflug images using a pre-constructed inference model; The inference model is A machine learning method using training data including at least a Scheimpflug image of an eye is applied to a neural network to be constructed in advance; The apparatus is configured to receive two or more Scheimpflug images each representing two or more different positions of the same eye, and to output an interpolated image obtained by spatially interpolating the two or more Scheimpflug images. An ophthalmologic photographing apparatus according to any one of claims 1 to 7.

9. The image interpolation unit is a first interpolation number determination unit that determines a number of interpolations based on at least one of operation information of the first image collection unit and the time-series displacement of the slit light, interpolating the series of Scheimpflug images with the interpolated number of images; The ophthalmologic photographing apparatus according to any one of claims 1 to 8.

10. The first image analysis unit further analyzes the series of time-series images collected by the second image collection unit to obtain a time-series displacement of the subject's eye, The image interpolation unit is a second interpolation number determination unit that determines a number of interpolations based on at least one of operation information of the first image acquisition unit and the time-series displacement of the subject's eye, interpolating the series of Scheimpflug images with the interpolated number of images; 10. An ophthalmologic photographing apparatus according to claim 1.

11. a first image acquisition unit that scans a three-dimensional region of the subject's eye with a slit light to acquire a series of Scheimpflug images; a second image acquisition unit that acquires a series of time-series images by repeatedly capturing images of the subject's eye in parallel with the scanning of the three-dimensional region by the first image acquisition unit; a second image analysis unit that analyzes the series of time-series images to detect a series of images of the slit light; an identifier assigning unit that assigns, to each of the series of Scheimpflug images, an identifier indicating an ocular tissue onto which the slit light was projected when the Scheimpflug image was acquired, based on the series of images of the slit light detected from the series of time-series images; Including, the scanning of the three-dimensional region by the first image acquisition unit and the repeated photographing of the subject's eye by the second image acquisition unit are performed in synchronization with each other; the identifier assigning unit assigns the identifier further based on a correspondence relationship between the series of Scheimpflug images and the series of time-series images based on synchronization between the scan and the repeated photographing, The identifier assignment unit is assigning the identifier to each of the series of time-series images based on the series of images of the slit of light; assigning the identifiers to the series of Scheimpflug images based on a series of identifiers assigned to the series of time-series images, respectively, and the correspondence relationship; Ophthalmic imaging equipment.

12. The second image analysis unit further analyzes the series of Scheimpflug images to detect a series of images of the slit light, the identifier assigning unit further assigns the identifier based on the series of images of the slit light detected from the series of Scheimpflug images.

12. An ophthalmologic imaging apparatus according to claim 11.

13. The second image analysis unit further analyzes the series of Scheimpflug images to detect an image of a site of interest of the subject's eye, The identifier assigning unit further assigns an identifier indicating the region of interest to any one of the series of time-series images collected by the second image collecting unit based on a detection result of the image of the region of interest from the series of Scheimpflug images.

13. An ophthalmologic photographing apparatus according to claim 11 or 12.

14. the identifier assigning unit assigns the identifier to a partial region of the Scheimpflug image corresponding to the eye tissue; An ophthalmologic photographing apparatus according to any one of claims 11 to 13.

15. the first image acquisition unit includes a first imaging system and a second imaging system that respectively capture images of the subject's eye from different directions during the scan, the first imaging system acquires a first set of Scheimpflug images; the second imaging system acquires a second set of Scheimpflug images; the series of Scheimpflug images includes the first group of Scheimpflug images and the second group of Scheimpflug images; An ophthalmologic photographing apparatus according to any one of claims 11 to 14.

16. The imaging by the first imaging system and the imaging by the second imaging system are performed in synchronization with each other.

16. An ophthalmologic imaging apparatus according to claim 15.

17. an image selection unit that selects a new series of Scheimpflug images corresponding to the scan from the first Scheimpflug image group and the second Scheimpflug image group based on a correspondence relationship between the first Scheimpflug image group and the second Scheimpflug image group based on synchronization between the photographing by the first photographing system and the photographing by the second photographing system, the identifier assigning unit assigns the identifier to each of the new series of Scheimpflug images; 17. An ophthalmic imaging apparatus according to claim 16.

18. the first image acquisition unit further includes an illumination system that projects the slit light onto the three-dimensional area; an optical axis of the first imaging system and an optical axis of the second imaging system are arranged to be inclined in opposite directions to each other with respect to an optical axis of the illumination system; the image selection unit selects a plurality of Scheimpflug images that do not include a corneal reflection artifact from the first Scheimpflug image group and the second Scheimpflug image group based on the correspondence relationship between the first Scheimpflug image group and the second Scheimpflug image group, thereby selecting the new series of Scheimpflug images.

18. An ophthalmic imaging apparatus according to claim 17.

19. the first image acquisition unit performs the scan of the three-dimensional area by translating the slit light in a direction perpendicular to a longitudinal direction of the slit light; An ophthalmologic photographing apparatus according to any one of claims 1 to 18.

20. The longitudinal direction of the slit light is substantially aligned with the body axis direction of the subject.

20. An ophthalmic imaging apparatus according to claim 19.

21. a dimension of the slit light in the longitudinal direction is equal to or larger than a diameter of the cornea in the body axis direction; The distance of the translation of the slit light is equal to or greater than the cornea diameter in a direction perpendicular to the body axis direction.

21. An ophthalmic imaging apparatus according to claim 20.

22. a receiving unit that receives a series of images acquired by scanning a three-dimensional region of the subject's eye with a scanning light and a series of time-series images collected by repeatedly photographing the subject's eye in parallel with the scanning of the three-dimensional region; a first image analysis unit that analyzes the series of time-series images to obtain a time-series displacement of the scanning light; an image interpolation unit that performs interpolation of the series of images based on the time-series displacement of the scanning light; Including, The first image analysis unit is analyzing the series of time series images to detect a series of images of the scanning light; determining the time series displacement of the scanning light based on the series of images of the scanning light; Ophthalmic imaging device.

23. a reception unit that receives a series of images acquired by scanning a three-dimensional region of a subject's eye with a scanning light and a series of time-series images collected by repeatedly photographing the subject's eye in synchronization with the scanning of the three-dimensional region; a second image analysis unit that analyzes the series of time-series images to detect a series of images of the scanning light; an identifier assigning unit that assigns, to each of the series of images acquired by scanning the three-dimensional region with the scanning light based on the series of images of the scanning light detected from the series of time-series images, an identifier indicating the eye tissue onto which the scanning light was projected when the image was acquired; Including, the identifier assigning unit assigns the identifier further based on a correspondence relationship between the series of images based on synchronization between the scan and the repeated photographing and the series of time-series images; The identifier assignment unit is assigning the identifier to each of the series of time-series images based on the series of images of the scanning light; assigning the identifiers to each of the series of images based on a series of identifiers assigned to the series of time-series images, respectively, and the correspondence relationship; Ophthalmic imaging device.

Images