Slit Lamp Microscope

Abstract

To provide useful evaluations pertaining to follow-up photographing errors.SOLUTION: A slit lamp microscope 1 of an exemplary aspect includes an image acquisition unit (lighting system 2, photographing system 3, movement mechanism 6), a storage unit 10, and a misalignment information acquisition unit. The image acquisition unit scans the anterior eye part of the eye E to be inspected with slit light and acquires an image. The storage unit 10 store the first image of the anterior eye part of the eye E to be inspected and a second image acquired in follow-up photographing performed by the image acquisition unit referring to the first image. The misalignment information acquisition unit analyzes the first and second images after follow-up photographing is performed and acquires misalignment information between the first and the second images.SELECTED DRAWING: Figure 1A

Description

[Technical Field]

[0001] This invention relates to a slit lamp microscope. [Background technology]

[0002] Diagnostic imaging plays a crucial role in ophthalmology. Various ophthalmic imaging devices are used for diagnostic imaging. These include slit-lamp microscopes, fundus cameras, scanning laser ophthalmoscopy (SLO), and optical coherence tomography (OCT).

[0003] Of these various ophthalmic devices, the most widely and frequently used is the slit lamp microscope. A slit lamp microscope is used to illuminate the eye under examination with a slit beam and to observe or photograph the illuminated cross-section from oblique or lateral angles using a microscope (see, for example, Patent Documents 1 and 2).

[0004] One of the main uses of a slit lamp microscope is anterior segment observation. In anterior segment observation, physicians observe the entire anterior segment by moving the illumination field and focus position of the slit light to determine whether or not there are any abnormalities. Slit lamp microscopes are also sometimes used in prescribing vision correction devices, such as checking the fitting of contact lenses. Furthermore, non-physicians such as optometrists, paramedical staff, and opticians may also use slit lamp microscopes for purposes such as screening and follow-up (monitoring) of eye diseases.

[0005] Furthermore, in light of recent advancements in information and communication technology, research and development related to telemedicine are progressing. Telemedicine is the practice of providing medical care to patients in remote locations using communication networks such as the internet. Patent documents 3 and 4 disclose technology for operating a slit lamp microscope from a remote location.

[0006] Obtaining good images using a slit lamp microscope requires delicate and complex operations, such as adjusting the illumination angle and shooting angle. However, the techniques disclosed in Patent Documents 3 and 4 require examiners in remote locations to perform these difficult operations, even when observing the eyes of a subject who is right in front of them. This leads to problems such as longer examination times and the inability to obtain good images.

[0007] Furthermore, as mentioned above, slit lamp microscopes are effective for screening and follow-up examinations, but there is a shortage of people skilled in operating the equipment, which is hindering the provision of high-quality slit lamp microscopy examinations to a wider audience.

[0008] Furthermore, in telemedicine and screening, the physician performing image interpretation (diagnostic imaging) is often not present at the imaging site. In such cases, there is a possibility that images unsuitable for interpretation may be provided to the physician. Considering the difficulty of operation and the length of imaging time, this problem is expected to occur more frequently with slit lamp microscopes.

[0009] In follow-up examinations, the same area of ​​the eye being examined is repeatedly examined. Follow-up in ophthalmology presents a unique problem: it is difficult to examine the same area repeatedly due to the influence of eye movements. Patent documents 5-7 describe techniques that are considered usable to address this problem.

[0010] The invention described in Patent Document 5 aims to smoothly detect positional displacement between images, including rotational movement, and is configured to detect the amount of positional displacement between images by comparing multiple small region images extracted from each image.

[0011] The invention described in Patent Document 6 aims to continuously measure the same area of ​​the eye under examination. It is configured to detect deviations in the scan position by scanning the eye with a measurement light beam while irradiating the eye with an auxiliary light beam and detecting the deviation, and to correct the scan position of the measurement light beam as needed based on the detection result. Furthermore, the invention described in Patent Document 6 can also utilize template matching based on characteristic regions in the frontal image in detecting deviations in the scan position.

[0012] The invention described in Patent Document 7 aims to accurately irradiate a light beam to a target position regardless of eye movement, and is configured to detect the field of view information of the scan light and the positional displacement information of the eye under examination based on a frontal image, and to correct the scan position based on this information.

[0013] Knowing the accuracy and precision of follow-up imaging, that is, obtaining information on the error in the imaging position (imaging range) during follow-up, is extremely important. In other words, considering whether follow-up imaging was performed properly, and furthermore, to what degree of accuracy and precision it was performed, is crucial for making a correct diagnosis. However, with conventional technology, it is not possible to evaluate the degree of error in follow-up imaging in a way that is useful to the user.

[0014] For example, the technology described in Patent Document 5 can detect the amount of positional misalignment between images, but it cannot provide the user with information about the meaning of that misalignment or whether the follow-up imaging was performed appropriately. Therefore, the user has to make their own judgment based on that misalignment. However, the error in the scan start point during follow-up imaging is usually on the order of millimeters or less, making it difficult to visually determine whether the follow-up imaging was performed appropriately. Such a judgment is not easy for unskilled individuals, nor even for skilled physicians. Furthermore, positional misalignment can manifest in various ways, such as parallel misalignment and rotational misalignment, but it has been impossible to evaluate positional misalignment, at least from the perspective of the appropriateness of the follow-up imaging.

[0015] Since the techniques described in Patent Document 6 or the techniques described in Patent Document 7 detect positional displacement while performing imaging and correct the scanning position, even if this correction is performed, a deviation in the scanning position (error in follow-up imaging) still intervenes. Therefore, these conventional techniques cannot suitably evaluate the degree of error in follow-up imaging.

Prior Art Documents

Patent Documents

[0016]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Patent Document 5

Patent Document 6

Patent Document 7

Summary of the Invention

Problems to be Solved by the Invention

[0017] One object of the present invention is to provide a useful evaluation regarding the error in follow-up imaging.

Means for Solving the Problems

[0018] An exemplary slit lamp microscope includes an image acquisition unit, a storage unit, and a misalignment information acquisition unit. The image acquisition unit scans the anterior segment of the eye to be examined with slit light to acquire an image. The storage unit stores a first image of the anterior segment of the eye to be examined and a second image acquired in follow-up imaging performed by the image acquisition unit with reference to the first image. The misalignment information acquisition unit analyzes the first image and the second image after the follow-up imaging is performed to acquire misalignment information between the first image and the second image.

Effect of the Invention

[0019] According to an exemplary aspect, it becomes possible to provide a useful evaluation regarding the error of follow-up imaging.

Brief Description of the Drawings

[0020] [Figure 1A] It is a schematic diagram showing the configuration of a slit lamp microscope according to an exemplary aspect. [Figure 1B] It is a schematic diagram showing the configuration of a slit lamp microscope according to an exemplary aspect. [Figure 2A] It is a schematic diagram for explaining the operation of a slit lamp microscope according to an exemplary aspect. [Figure 2B] It is a schematic diagram for explaining the operation of a slit lamp microscope according to an exemplary aspect. [Figure 3] It is a schematic diagram for explaining the operation of a slit lamp microscope according to an exemplary aspect. [Figure 4A] It is a schematic diagram showing the configuration of a slit lamp microscope according to an exemplary aspect. [Figure 4B] It is a schematic diagram showing the configuration of a slit lamp microscope according to an exemplary aspect. [Figure 4C] It is a schematic diagram showing the configuration of a slit lamp microscope according to an exemplary aspect. [Figure 4D] It is a schematic diagram showing the configuration of a slit lamp microscope according to an exemplary aspect. [Figure 5A] It is a schematic diagram showing the configuration of a slit lamp microscope according to an exemplary aspect. [Figure 5B] This is a schematic diagram illustrating the construction of an inference model for a slit lamp microscope according to an exemplary embodiment. [Figure 5C] This is a schematic diagram illustrating the configuration of a slit lamp microscope according to an exemplary embodiment. [Figure 5D] This is a schematic diagram illustrating the configuration of a slit lamp microscope according to an exemplary embodiment. [Figure 6] This is a schematic diagram illustrating the configuration of a slit lamp microscope according to an exemplary embodiment. [Figure 7A] This is a schematic diagram illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Figure 7B] This is a schematic diagram illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Figure 7C] This is a schematic diagram illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Figure 7D] This is a schematic diagram illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Figure 8A] This is a schematic diagram illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Figure 8B] This is a schematic diagram illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Figure 8C] This is a schematic diagram illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Figure 8D] This is a schematic diagram illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Figure 9] This is a schematic diagram illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Figure 10] This is a schematic diagram illustrating the configuration of the optical system of a slit lamp microscope according to an exemplary embodiment. [Figure 11] This is a schematic diagram illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Figure 12] This is a flowchart illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Figure 13] This is a flowchart illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Figure 14] This is a flowchart illustrating the operation of a slit lamp microscope according to an exemplary embodiment. [Modes for carrying out the invention]

[0021] Several exemplary embodiments will be described in detail with reference to the drawings. Any prior art, such as that disclosed in the documents cited herein, may be combined with the exemplary embodiments. Furthermore, the entirety of disclosures in other applications made by the applicant may be incorporated into this disclosure. For example, at least a portion of the disclosures in applications relating to slit lamp microscopes, at least a portion of the disclosures in applications relating to systems including slit lamp microscopes, or at least a portion of the disclosures in applications relating to systems connectable to slit lamp microscopes may be incorporated into this disclosure.

[0022] The slit lamp microscope according to the exemplary embodiment may be stationary or portable. The slit lamp microscope according to the exemplary embodiment may be used in situations or environments where a person with expertise in the device or the examination is not present, or in situations or environments where a skilled person is present, or in situations or environments where a skilled person can monitor, instruct, and operate it remotely. Examples of facilities where the slit lamp microscope according to the exemplary embodiment may be installed include opticians, optometrists, medical institutions, health checkup venues, screening venues, patients' homes, welfare facilities, public facilities, and mobile screening units.

[0023] An apparatus or system according to an exemplary embodiment has at least the function of a slit lamp microscope, and may optionally further have imaging functions (modalities) other than a slit lamp microscope, eye characteristic measurement functions, analysis functions, etc.

[0024] An exemplary ophthalmic system (first ophthalmic system) may include one or more slit-lamp microscopes, one or more information processing devices, and one or more image interpretation terminals, and may be used, for example, for telemedicine. The slit-lamp microscope may be a slit-lamp microscope according to any exemplary embodiment, or a slit-lamp microscope equipped with at least a part thereof. The information processing device receives images acquired by the slit-lamp microscope and transmits them to the image interpretation terminal. The information processing device may also have a function for managing images acquired by the slit-lamp microscope. Images may be transmitted from the slit-lamp microscope to the image interpretation terminal without providing an information processing device. The image interpretation terminal is a computer used by a physician (typically an ophthalmologist or radiologist or other specialist) to interpret images acquired by the slit-lamp microscope (observing images and obtaining clinical findings). Information entered by the radiologist into the image interpretation terminal may be converted into an image interpretation report or electronic medical record information by the image interpretation terminal or another computer and transmitted to the information processing device. In other exemplary configurations, the reading terminal may receive information entered by a radiologist, the reading terminal may transmit the received information to an information processing device, the information processing device may receive the information transmitted from the reading terminal, and the information processing device or another computer may generate a reading report or electronic medical record information from the received information. The information processing device may have a function for managing the reading report or electronic medical record information. The information processing device may also have a function for transferring the reading report or electronic medical record information to other devices or other medical systems (e.g., electronic medical record systems).

[0025] Other exemplary ophthalmic systems (second ophthalmic systems) may include one or more slit-lamp microscopes, one or more information processing devices, and one or more image interpretation devices. At least one of the slit-lamp microscope and the information processing device may be the same as that of the first ophthalmic system. Alternatively, images may be transmitted from the slit-lamp microscope to the image interpretation device without providing an information processing device. The image interpretation device is, for example, a computer that performs image interpretation processing of images acquired by the slit-lamp microscope using an image processing processor and / or an artificial intelligence engine (inference engine, inference model, trained model, etc.) that operates according to a predetermined program. The image interpretation device or other computer can generate an image interpretation report or electronic medical record information from the information derived from the image by the image interpretation device. The generated image interpretation report or electronic medical record information may be transmitted to the information processing device. In other examples, the information interpretation device can transmit information derived from the image to the information processing device. In this case, the information processing device or other computer can generate an image interpretation report or electronic medical record information from the information derived from the image by the image interpretation device. In other examples, the image interpretation device can provide physicians with information derived from images, and / or information generated based on this information (such as image interpretation reports and electronic medical record information), as diagnostic support information.

[0026] Furthermore, other exemplary embodiments of an ophthalmic system (the third ophthalmic system) may include one or more slit-lamp microscopes and one or more information processing devices. The slit-lamp microscope may be the same as that of the first ophthalmic system. The information processing device may be the same as that of the first ophthalmic system and / or that of the second ophthalmic system. In other words, the information processing device may be capable of data communication with a reading terminal or similar computer of the first ophthalmic system, and / or with a reading device or similar computer of the second ophthalmic system.

[0027] While the slit-lamp microscope and ophthalmic system according to the exemplary embodiment can be used for telemedicine, as mentioned above, obtaining good images with a slit-lamp microscope is not easy, and a wide range of images of the anterior segment of the eye must be acquired "in advance" for effective interpretation and diagnosis. For these reasons, it can be said that effective telemedicine using a slit-lamp microscope has not yet been realized. The exemplary embodiment can provide technology that can contribute to its realization. However, the applications of the exemplary embodiment are not limited to telemedicine, and it is possible to apply the exemplary embodiment to other applications. In particular, the applications of this disclosure include follow-up using a slit-lamp microscope.

[0028] A slit-lamp microscope according to an exemplary embodiment can be used to acquire a series of images (image group, image set) that represent a wide area of ​​the anterior segment of the eye with good image quality.

[0029] The exemplary embodiment also addresses the following problem: In the application assumed by the exemplary embodiment (such as telemedicine), a series of images (image group, image set) from a sufficiently wide area of ​​the anterior segment are provided to the radiologist directly or indirectly using a slit lamp microscope. Therefore, even if the quality of the provided series of images is poor, it is difficult to retake the images, which may result in problems such as being unable to interpret the images at all or only being able to interpret them insufficiently. Consequently, it is necessary to acquire images of "good" quality "in advance." In other words, it is necessary to acquire a series of images of sufficient quality to perform a diagnosis (such as image interpretation) all at once before providing them to the radiologist. However, considering the difficulty of operating a slit lamp microscope, as well as the occurrence of blinking and eye movements during shooting, it is extremely difficult to obtain a series of images that represent the entire target area for observation and interpretation with good image quality. This disclosure also takes into consideration how to address these problems.

[0030] The following describes several exemplary embodiments. It is possible to combine at least two or more of these embodiments in part. Furthermore, it is possible to combine any known technology with such combinations or to make modifications (substitutions, omissions, etc.) based thereon.

[0031] At least a portion of the functionality of the elements disclosed herein is implemented using a circuitry or processing circuitry. The circuitry or processing circuitry is configured and / or programmed to perform at least a portion of the disclosed functionality and may include general-purpose processors, dedicated processors, integrated circuits, CPUs (Central Processing Units), GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), programmable logic devices (e.g., SPLDs (Simple Programmable Logic Devices), CPLDs (Complex Programmable Logic Devices), FPGAs (Field Programmable Gates)). This includes any of the following: an array, a conventional circuit configuration, and any combination thereof. A processor is considered a processing circuit configuration or circuit configuration, including transistors and / or other circuit configurations. In this disclosure, a circuit configuration, unit, means, or similar terms means hardware that performs at least a portion of the disclosed functions, or hardware programmed to perform at least a portion of the disclosed functions. The hardware may be the hardware disclosed herein, or it may be known hardware programmed and / or configured to perform at least a portion of the described functions. If the hardware is a processor that can be considered a certain type of circuit configuration, then a circuit configuration, unit, means, or similar terms means a combination of hardware and software, the software being used to configure the hardware and / or processor.

[0032] <Configuration of a slit lamp microscope> Figures 1A and 1B show the configuration of a slit lamp microscope according to one exemplary embodiment. Figure 1A shows an example of the overall configuration of the slit lamp microscope 1, and Figure 1B shows an example of the configuration of the data processing unit 8 of the slit lamp microscope 1. The slit lamp microscope 1 can be used for follow-up of the eye E under examination. In Figure 1A, the symbol C represents the cornea, the symbol CL represents the lens, the symbol IR represents the iris, and the symbol CA represents the iridocorneal angle.

[0033] The slit lamp microscope 1 includes an illumination system 2, an imaging system 3, a movement mechanism 6, a control unit 7, a data processing unit 8, a communication unit 9, and a storage unit 10. The slit lamp microscope 1 may be a single device or a system including two or more devices. As an example of the latter, the slit lamp microscope 1 includes a main unit including the illumination system 2, the imaging system 3, and the movement mechanism 6, a computer including the control unit 7, the data processing unit 8, and the communication unit 9, and a communication interface responsible for data communication between the main unit and the computer. This computer may, for example, be installed together with the main unit or installed on a network. Examples of this computer include the above-mentioned information processing device, image interpretation terminal, and image interpretation device.

[0034] <Lighting System 2> Illumination system 2 irradiates the anterior segment of the eye E under examination with slit light. Reference numeral 2a indicates the optical axis (illumination optical axis) of illumination system 2. Illumination system 2 may have the same configuration as the illumination system of a conventional slit lamp microscope. Although not shown in the figure, an exemplary illumination system 2 includes, in order from the side furthest from the eye E under examination, an illumination light source, a positive lens, a slit forming section, and an objective lens.

[0035] The illumination light source outputs illumination light. Illumination system 2 may have multiple illumination light sources. For example, illumination system 2 may include an illumination light source that outputs continuous light and an illumination light source that outputs flash light. Illumination system 2 may also include an illumination light source for the anterior segment and an illumination light source for the posterior segment. Illumination system 2 may also include two or more illumination light sources with different output wavelengths. A typical illumination system 2 includes a visible light source as an illumination light source. Illumination system 2 may also include an infrared light source. The illumination light output from the illumination light source passes through the positive lens and is projected onto the slit forming section.

[0036] The slit-forming unit generates slit light by allowing a portion of the illumination light to pass through. A typical slit-forming unit has a pair of slit blades. By changing the distance between these slit blades (slit width), the width of the area through which the illumination light passes (slit) is changed, thereby changing the width of the slit light. The slit-forming unit may also be configured to allow the length of the slit light to be changed. The length of the slit light is the cross-sectional dimension of the slit light in a direction perpendicular to the cross-sectional width direction of the slit light, which corresponds to the slit width. The width and length of the slit light are typically expressed as the dimensions of the projected image of the slit light onto the anterior eye, but are not limited to this. For example, they can also be expressed as the dimensions of the cross-section of the slit light at any position, or as the dimensions of the slit formed by the slit-forming unit.

[0037] The slit light generated by the slit forming unit is refracted by the objective lens and illuminated onto the anterior segment of the eye E under examination.

[0038] The illumination system 2 may further include a focusing mechanism for changing the focus position of the slit light. The focusing mechanism, for example, moves the objective lens along the illumination optical axis 2a. The movement of the objective lens can be performed automatically and / or manually. Alternatively, a focusing lens may be positioned on the illumination optical axis 2a between the objective lens and the slit forming portion, and the focus position of the slit light may be changed by moving this focusing lens along the illumination optical axis 2a.

[0039] Figure 1A is a top view, and as shown in the figure, in this embodiment, the direction along the axis of the eye E under examination is defined as the Z direction, the direction perpendicular to this, which is the left and right direction for the subject, is defined as the X direction, and the direction perpendicular to both the X and Z directions is defined as the Y direction. Typically, the X direction is the alignment direction of the left and right eyes, and the Y direction is the direction along the subject's body axis (body axis direction).

[0040] <Photography Series 3> The imaging system 3 images the anterior segment of the eye illuminated by slit light from the illumination system 2. Reference numeral 3a indicates the optical axis (imaging optical axis) of the imaging system 3. The imaging system 3 in this embodiment includes an optical system 4 and an image sensor 5.

[0041] The optical system 4 guides light from the anterior segment of the eye E being illuminated by the slit light to the image sensor 5. The image sensor 5 receives the light guided by the optical system 4 on its imaging surface.

[0042] The light guided by the optical system 4 (light from the anterior segment of the eye E under examination) includes the reflected light from the slit light illuminating the anterior segment, and may also include other light. Examples of reflected light include reflected light, scattered light, and fluorescence. Examples of other light include light from the environment (indoor light, sunlight, etc.). If an anterior segment illumination system for illuminating the entire anterior segment is provided separately from the illumination system 2, the reflected light from this anterior segment illumination system is included in the light guided by the optical system 4.

[0043] The optical system 4 may have a configuration similar to that of a conventional slit-lamp microscope imaging system. For example, the optical system 4 includes, in order from the side closest to the eye E under examination, an objective lens, a variable magnification optical system, and an imaging lens. Light from the anterior segment of the eye E under examination, which is illuminated by the slit light, passes through the objective lens and the variable magnification optical system, and is imaged onto the imaging surface of the image sensor 5 by the imaging lens.

[0044] The image sensor 5 is an area sensor having a two-dimensional imaging area, and may be, for example, a charge-coupled device (CCD) image sensor or a complementary metal-oxide-semiconductor (CMOS) image sensor.

[0045] The imaging system 3 may further include a focusing mechanism for changing its focus position. The focusing mechanism, for example, moves the objective lens along the imaging optical axis 3a. The movement of the objective lens can be performed automatically and / or manually. Alternatively, a focusing lens may be positioned on the imaging optical axis 3a between the objective lens and the imaging lens, and the focus position may be changed by moving this focusing lens along the imaging optical axis 3a.

[0046] The illumination system 2 and the imaging system 3 function as a Scheinproof camera. That is, the illumination system 2 and the imaging system 3 are configured such that the object surface along the illumination optical axis 2a, the optical system 4, and the imaging surface of the image sensor 5 satisfy the so-called Scheinproof conditions. More specifically, the YZ plane (including the object surface) passing through the illumination optical axis 2a, the principal surface of the optical system 4, and the imaging surface of the image sensor 5 intersect on the same straight line. This makes it possible to focus and take images at all positions within the object surface (all positions in the direction along the illumination optical axis 2a).

[0047] In this embodiment, for example, imaging is performed with the imaging system 3 in focus on a three-dimensional region including at least a portion of a predetermined part of the anterior segment of the eye. In the Z direction, for example, imaging can be performed with the imaging system 3 in focus over the entire range from the apex of the anterior surface of the cornea C (Z=Z1) to the apex of the posterior surface of the lens CL (Z=Z2). Note that Z=Z0 indicates the Z coordinate of the intersection of the illumination optical axis 2a and the imaging optical axis 3a.

[0048] These conditions are typically achieved by the configuration and arrangement of elements in the illumination system 2, the configuration and arrangement of elements in the imaging system 3, and the relative positions of the illumination system 2 and the imaging system 3. Parameters indicating the relative positions of the illumination system 2 and the imaging system 3 include, for example, the angle θ between the illumination optical axis 2a and the imaging optical axis 3a. The angle θ is set to, for example, 17.5 degrees, 30 degrees, or 45 degrees. Note that the angle θ may be variable.

[0049] <Movement mechanism 6> The moving mechanism 6 moves the lighting system 2 and the imaging system 3. The moving mechanism 6 includes, for example, a movable stage on which the lighting system 2 and the imaging system 3 are mounted, an actuator that operates according to a control signal input from the control unit 7, and a mechanism that moves the movable stage based on the driving force generated by the actuator. In another example, the moving mechanism 6 includes a movable stage on which the lighting system 2 and the imaging system 3 are mounted, and a mechanism that moves the movable stage based on a force applied to an operating device (not shown). The operating device is, for example, a lever. The movable stage is movable in at least the X direction, and may also be movable in the Y direction and / or the Z direction.

[0050] In this embodiment, the moving mechanism 6 moves the illumination system 2 and the imaging system 3 together in the X direction. That is, the moving mechanism 6 moves the illumination system 2 and the imaging system 3 in the X direction while maintaining the state in which the above-described shine-proof conditions are satisfied. In parallel with this movement, the imaging system 3 performs video recording at predetermined time intervals (shooting rate). As a result, the three-dimensional region of the anterior segment of the eye E under examination is scanned with slit light, and multiple images (image group) corresponding to multiple cross-sections within this three-dimensional region are collected.

[0051] In a typical example, the longitudinal direction (length direction) of the slit light illuminating the anterior segment by the illumination system 2 coincides with the Y direction, and the transverse direction (width direction) coincides with the X direction. Furthermore, the movement mechanism 6 moves the illumination system 2 and the imaging system 3 together in the X direction. In other words, the movement mechanism 6 moves the illumination system 2 and the imaging system 3 so that the slit light illuminating the anterior segment by the illumination system 2 moves in the width direction of this slit light. This makes it possible to scan the three-dimensional region of the anterior segment simply and efficiently. Note that the combination of the direction of the slit light and the direction of movement can be arbitrary.

[0052] <Control Unit 7> The control unit 7 controls various parts of the slit lamp microscope 1. For example, the control unit 7 controls elements of the illumination system 2 (illumination light source, slit forming unit, focusing mechanism, etc.), elements of the imaging system 3 (focusing mechanism, image sensor, etc.), the movement mechanism 6, the data processing unit 8, the communication unit 9, etc. The control unit 7 may also be capable of performing controls to change the relative position between the illumination system 2 and the imaging system 3. Furthermore, the control unit 7 may be capable of controlling devices included in the slit lamp microscope 1 (e.g., display device) or devices connected to the slit lamp microscope 1 (e.g., display device).

[0053] The control unit 7 includes a processor, main memory, and auxiliary memory. The auxiliary memory stores control programs and the like. These control programs may be stored in a computer or memory device accessible by the slit lamp microscope 1. The functions of the control unit 7 are realized through the cooperation of software such as control programs and hardware such as the processor.

[0054] The control unit 7 can apply the following controls to the illumination system 2, the imaging system 3, and the movement mechanism 6 in order to scan the three-dimensional region of the anterior segment of the eye E under examination with slit light.

[0055] First, the control unit 7 controls the movement mechanism 6 to position the illumination system 2 and the imaging system 3 at a predetermined scan start position (alignment control). The scan start position is, for example, a position corresponding to the edge (first edge) of the cornea C in the X direction, or a position further away from the axis of the eye E being examined.

[0056] In Figure 2A, the symbol X0 indicates an example of a scan start position corresponding to the first edge of the cornea C in the X direction. In Figure 2B, the symbol X0' indicates an example of a scan start position that is further from axis EA of the eye E than the position corresponding to the first edge of the cornea C in the X direction.

[0057] The control unit 7 controls the illumination system 2 to start irradiating the anterior segment of the eye E under examination with slit light (slit light irradiation control). Note that slit light irradiation control may be performed before or during alignment control. The slit light is typically continuous light, but may also be intermittent light (pulsed light). The control of pulsed light illumination is synchronized with the imaging rate of the imaging system 3. Furthermore, the slit light is typically visible light, but may also be infrared light, or a mixture of visible and infrared light.

[0058] The control unit 7 controls the imaging system 3 to start recording video of the anterior segment of the eye E under examination (imaging control). Note that imaging control may be performed before or during alignment control. Typically, imaging control is performed simultaneously with or after slit light irradiation control.

[0059] After the alignment control, slit light irradiation control, and imaging control are performed, the control unit 7 controls the movement mechanism 6 to start moving the illumination system 2 and the imaging system 3 (movement control). The illumination system 2 and the imaging system 3 are moved together by the movement control. In other words, the illumination system 2 and the imaging system 3 are moved while maintaining their relative positions (angle θ, etc.). Typically, the illumination system 2 and the imaging system 3 are moved while maintaining the state in which the aforementioned shine-proof conditions are satisfied. The movement of the illumination system 2 and the imaging system 3 is performed from the aforementioned scan start position to a predetermined scan end position. The scan end position is, for example, a position corresponding to the end of the cornea C opposite the first end (second end) in the X direction, similar to the scan start position, or a position further away from the axis of the eye E under examination. In such a case, the range from the scan start position to the scan end position becomes the scan range.

[0060] Typically, a slit beam with the X direction as the width and the Y direction as the length is projected onto the anterior eye region, while the illumination system 2 and the imaging system 3 are moved in the X direction, and video recording is performed by the imaging system 3.

[0061] Here, the length of the slit light (i.e., the dimension of the slit light in the Y direction) is set to be greater than or equal to the diameter of the cornea C on the surface of the eye E being examined. In other words, the length of the slit light is set to be greater than or equal to the corneal diameter in the Y direction. Also, as mentioned above, the movement distance of the illumination system 2 and imaging system 3 by the moving mechanism 6 (i.e., the scan range) is set to be greater than or equal to the corneal diameter in the X direction. This allows at least the entire cornea C to be scanned with the slit light.

[0062] By performing such a scan, multiple anterior segment images are obtained with different slit light illumination positions. In other words, a moving image is obtained that depicts the movement of the slit light illumination position in the X direction. An example of such multiple anterior segment images (i.e., a group of frames that make up a moving image) is shown in Figure 3.

[0063] Figure 3 shows multiple anterior segment images (frame groups, image groups) F1, F2, F3, ..., FN. The subscript n in these anterior segment images Fn (n=1, 2, ..., N) indicates the chronological order. That is, the nth acquired anterior segment image is represented by the code Fn. The anterior segment image Fn includes the slit light illumination region An. As shown in Figure 3, the slit light illumination regions A1, A2, A3, ..., AN move to the right along the time series.

[0064] In the example shown in Figure 3, the longitudinal direction of the slit light coincides with the Y direction (vertical direction, body axis direction), the transverse direction (width direction) coincides with the X direction (horizontal direction, sideways direction), and the direction of movement of the slit light coincides with the X direction. Furthermore, the scan start position is outside (to the left) of the first edge (left edge) of the cornea C, and the scan end position is outside (to the right) of the second edge (right edge) of the cornea C. With this type of scan, a three-dimensional region including the entire cornea C can be captured. Note that the scan start position and / or scan end position are not limited to this example; for example, the scan start position and / or scan end position may be at the edges of the cornea C in the X direction. In addition, various scan parameters such as the direction of the slit light, the direction of movement of the slit light, and the number of scans can be arbitrarily set.

[0065] <Data Processing Unit 8> The data processing unit 8 performs various data processing operations. The data to be processed may be either data acquired by the slit lamp microscope 1 or data input from an external source. The data processing unit 8 can process images acquired by the imaging system 3.

[0066] The data processing unit 8 includes a processor, main memory, and auxiliary memory. The auxiliary memory stores data processing programs, etc. These data processing programs may include models constructed by machine learning (trained models, inference models, etc.). The data processing programs may be stored in a computer or memory device accessible by the slit lamp microscope 1. The functions of the data processing unit 8 are realized through the cooperation of software such as data processing programs and hardware such as the processor.

[0067] The exemplary data processing unit 8 shown in Figure 1B includes an image group processing unit 80, a processed image construction unit 85, a positional misalignment information acquisition unit 86, an evaluation value calculation unit 87, and a quality determination unit 88.

[0068] The slit lamp microscope 1 may be capable of performing follow-up imaging and post-processing. In follow-up imaging, the same position as in the past is captured by referring to the image of the eye E acquired in the past. Post-processing may include a process to evaluate the error in the imaging position in the follow-up imaging. More specifically, post-processing may include a process to evaluate the discrepancy between the region of the eye E imaged in the past imaging and the region of the eye E imaged in the follow-up imaging based on the image obtained in the past imaging.

[0069] Follow-up imaging using the slit lamp microscope 1 reproduces the scan position (scan range) from previous imaging (anterior segment scan with slit light). As preparation for this, for example, the control unit 7 can display a patient selection screen (not shown) on a display device (not shown). The patient selection screen includes, for example, a function unit for selecting a patient to whom follow-up imaging is applied (patient selection unit), a function unit for displaying patient information (patient information display unit), and a function unit for displaying imaging information (imaging information display unit). The patient selection screen may also be provided with various operation units (software keys). The slit lamp microscope 1 may also be equipped with an operation device for performing operations and / or inputting information related to the patient selection screen.

[0070] The patient selection section may include an input space for search queries and a calendar for selecting the shooting date (such as the last shooting date in the past). When input is made to the patient selection section, the control unit 7 searches for patient information stored in the storage device (such as the storage unit 10 or a storage device accessible by the slit lamp microscope 1) and displays the retrieved patient information on the patient information display unit. The patient information may include, for example, patient identification information (patient ID, subject ID), name, gender, date of birth, and electronic medical record information. When a calendar is used to select the shooting date, the control unit 7 may display a list of multiple patient information entries on the patient information display unit. The user can select the desired patient from the displayed list.

[0071] When a patient is selected, the control unit 7 can retrieve imaging information for that patient from the storage device and display it on the imaging information display unit. If multiple imaging sessions have been performed in the past, the control unit 7 may, for example, display a list in which the imaging information for each imaging session is enumerated in chronological order. The imaging information may include, for example, the date of imaging, the time of imaging, the data storage location (file number, etc.), whether it is a follow-up imaging session or not, the scan mode (scan conditions, etc.), left eye / right eye identification information, fixation position, and information related to the analysis process (analysis content, analysis conditions, etc.).

[0072] When imaging information to be referenced in follow-up imaging is selected, the control unit 7 can retrieve images obtained from past imaging corresponding to the selected imaging information from the storage device and display them on the display device. At this time, the display screen switches from the patient selection screen to the image display screen (not shown). The user can refer to the displayed image and decide whether or not to use this image for follow-up imaging. If other images are to be referenced, the user can observe various past images and select the desired image. Once the image to be referenced is determined, the control unit 7 can display this image on the display device and transition to a state waiting for the instruction to start follow-up imaging.

[0073] When a predetermined instruction to start imaging is received, the control unit 7 can perform actions such as alignment control, slit light irradiation control, imaging control, and movement control. In this way, control is performed to image the same body part under the same conditions as in previous imaging sessions.

[0074] The control unit 7 can store images obtained through such follow-up imaging in the storage unit 10, associating them with patient information, scan position information (scan range information), imaging information, and information about previously referenced imaging.

[0075] <Image group processing unit 80> The image group processing unit 80 processes at least a portion of the images collected by the anterior segment scan using slit light. Several examples of the image group processing unit 80 are described below. Figures 4A, 4B, 4C, and 4D show the first, second, third, and fourth examples of the image group processing unit 80, namely image group processing unit 80A, 80B, 80C, and 80D, respectively. Note that the configuration of the image group processing unit 80 is not limited to these. For example, the image group processing unit 80 may include any two or more combinations of the four image group processing units 80A, 80B, 80C, and 80D. Furthermore, it is possible to provide the image group processing unit 80 with any elements necessary to obtain the same or similar type of results.

[0076] In this embodiment, the slit lamp microscope 1 applies one or more scans to the anterior segment of the eye E under examination. Each scan yields a group of images such as the multiple anterior segment images F1 to FN shown in Figure 3. The image group processing unit 80 can process one or more of these acquired image groups.

[0077] The slit lamp microscope 1 may be capable of applying two or more scans to the anterior segment. For example, if a good set of images is not obtained in the first scan, a second scan is performed. More generally, if a good set of images is not obtained in the first to v scans, an (v+1) scan is performed (v is an integer greater than or equal to 1). In some exemplary embodiments, an (v+1) scan is performed if a good set of images is not obtained from the v image sets obtained in the first to v scans. In some other exemplary embodiments, an (v+1) scan is performed if none of the v image sets obtained in the first to v scans are good image sets. Note that the embodiments of performing two or more scans are not limited to these.

[0078] When performing two or more scans, the start and end positions (i.e., scan ranges) of these scans may be the same, or one or both of the start and end positions may be different. Typically, two or more scans are applied to the same scan range. This results in acquiring a set of images, such as the multiple anterior segment images F1-FN shown in Figure 3, the same number of images as the number of scans.

[0079] Considering the eye movements of the eye under examination E, the scan range (scan start position, scan end position) is typically defined not by the position or range in the anterior segment of the eye, but by the operation of the slit lamp microscope 1. For example, the starting point of the movement of the illumination system 2 and imaging system 3 during the scan is defined as the scan start position, and the ending point is defined as the scan end position.

[0080] On the other hand, it is also possible to define the scan range by its position and extent in the anterior segment of the eye. In this case, tracking is applied to move the illumination system 2 and the imaging system 3 to follow the movement of the eye under examination E. The slit lamp microscope 1 in this example is equipped with a tracking function similar to that of conventional microscopes. The tracking function includes, for example, video recording of the anterior segment of the eye, extraction of landmarks from each frame, calculation of the displacement between frames using the landmarks, and control of the movement of the illumination system 2 and the imaging system 3 to cancel out the displacement.

[0081] Another example of defining (setting) the scan range is that it is possible to set the scan range for images that have already been collected. In other words, the scan range can be configured to be set retrospectively (after scanning). In this example, the scan range represents the area of ​​the image that will be provided for subsequent processing.

[0082] The first example of such retrospective scan range setting is described below. The slit lamp microscope in this example is capable of anterior segment imaging and is capable of recognizing the positional relationship between the anterior segment imaging optical system and the scanning optical system (illumination system 2 and imaging system 3). In this example, first, anterior segment imaging is performed while scanning a sufficiently wide area of ​​the anterior segment (especially in the X and Y directions) to collect a group of images (wide-area image group). Next, a scan range is set for the anterior segment images obtained from the anterior segment imaging. This setting may be done manually or automatically. Automatic setting includes, for example, a process of analyzing the anterior segment images to detect landmarks and a process of setting the scan range by referring to these landmarks. These landmarks may be, for example, the corneal margin, the iridocorneal angle, or the pupillary margin. Once the scan range is set, a sub-region of each wide-area image corresponding to this scan range is identified based on the positional relationship of the optical systems described above. Finally, an image corresponding to the scan range is formed by cropping this identified sub-region from the wide-area image. This yields a group of images corresponding to the set scan range. In this example, if the fixation of the eye under examination is stable (or if it can be assumed to be so or is possible to assume so), the anterior segment radiography may be still image capture. If the anterior segment radiography is video capture, for example, based on the control of scanning and video capture (synchronization information, etc.), a group of wide-area images and a group of frames in the video are associated, and the same processing as above is performed for each pair of associated wide-area images and frames.

[0083] A second example of retrospective scan range setting is described. In this example, parallel anterior segment imaging is not required, and a wide-area image set is collected by scanning a sufficiently wide area of ​​the anterior segment (especially in the X and Y directions). Next, a sub-region corresponding to the desired scan range is specified for each wide-area image. This specification may be done manually or automatically. Automatic specification includes, for example, a process of analyzing the wide-area image to detect landmarks (e.g., corneal margin or angle) and a process of setting the scan range by referring to these landmarks. Alternatively, a landmark may be manually specified for any wide-area image (reference wide-area image), and then the reference wide-area image and its adjacent wide-area images are analyzed to specify the landmark for the adjacent wide-area image. By applying such processes sequentially, landmarks can be specified for all wide-area images. Furthermore, a sub-region of the wide-area image corresponding to the scan range can be identified based on the landmark, and an image corresponding to the scan range can be formed by cropping this identified sub-region from the wide-area image. This yields a set of images corresponding to the set scan range.

[0084] A first example of the image group processing unit 80 will be described. The image group processing unit 80A shown in Figure 4A includes an image group evaluation unit 81. The image group evaluation unit 81 evaluates the quality of the image group collected in a single scan of the eye E under examination. In some exemplary embodiments, the image group evaluation unit 81 is configured to determine whether the image group satisfies predetermined conditions, typically, whether each image included in the image group satisfies predetermined conditions.

[0085] The specified conditions (image group evaluation conditions) are, for example, conditions related to image quality necessary for effective interpretation and diagnosis, such as conditions related to the eye E under examination, conditions related to the slit lamp microscope 1, and environmental conditions. The image group evaluation conditions may include, for example, either one or both of the following: "the image to be evaluated is not affected by blinking (blinking condition)" and "the image to be evaluated is not affected by eye movement (eye movement condition)." Furthermore, the image group evaluation conditions may be any index related to image quality evaluation, such as brightness, contrast, noise, signal-to-noise ratio (SNR), tone reproduction, dynamic range, sharpness, vignetting, aberrations (spherical aberration, astigmatism, coma aberration, field curvature, distortion, chromatic aberration, etc.), color accuracy, exposure accuracy, lens flare, moiré, and artifacts. The image selection conditions are not limited to these examples and may be selected arbitrarily.

[0086] The blinking conditions are explained below. For each image included in the image group, the image group evaluation unit 81 determines, for example, whether the reflected image of the slit light irradiated onto the anterior segment of the eye E under examination is included in the image. This determination is a process that utilizes the fact that the reflected image of the slit light is not depicted in images taken during blinking, and that the reflected image of the slit light is represented as significantly brighter than other areas. This is performed, for example, based on the brightness distribution in the image.

[0087] As an example, the image group evaluation unit 81 creates a luminance histogram from the image and determines whether there are pixels with a luminance above a predetermined threshold. If it is determined that there are pixels with a luminance above the predetermined threshold, it is determined that the image contains a reflected image of slit light. This example has the advantage of being extremely simple to process, but it may misdetect high-luminance noise or reflections of ambient light.

[0088] As another example, the image group evaluation unit 81 creates a luminance histogram from the image and determines whether there are a predetermined number of pixels with a luminance above a predetermined threshold. If it is determined that there are a predetermined number of pixels with a luminance above a predetermined threshold, it is determined that the image contains a reflected image of slit light. This example has the advantage of preventing the above-mentioned false detection through simple processing.

[0089] The eye movement conditions will now be explained. For each image included in the image group, the image group evaluation unit 81 determines, for example, whether or not eye movement has had an effect by comparing the image in question with an adjacent image. This determination is a process that utilizes the fact that "image jumps" occur when eye movement occurs during video recording.

[0090] As an example, the image group evaluation unit 81 detects landmarks from the image in question and adjacent images, calculates the displacement of these landmarks, and determines whether this displacement is greater than or equal to a predetermined threshold. If it is determined that the displacement is greater than or equal to the predetermined threshold, it is determined that eye movement has occurred. Here, the landmarks may be, for example, the cornea, iris, pupil, or iridocorneal angle. The threshold is calculated based on predetermined scanning conditions, such as the shooting rate of the shooting system 3 or the movement speed of the movement mechanism 6.

[0091] In other examples, the image group evaluation unit 81 may be configured to determine the presence or absence of eye movement from a single image. For example, when the shooting rate of the shooting system 3 is slow, rapid eye movement may cause "blur" in the image. The image group evaluation unit 81 can determine the presence or absence of eye movement by utilizing blur detection. Blur detection is typically performed using known techniques such as edge detection.

[0092] The image group evaluation unit 81 may include an artificial intelligence engine for determining whether the input images satisfy the above-mentioned image selection conditions. This artificial intelligence engine typically includes a convolutional neural network (CNN), which is pre-trained using training data that includes a large number of images acquired by a slit lamp microscope and the determination results of whether each image satisfies the image selection conditions. The images included in the training data are not limited to images acquired by a slit lamp microscope, but may also include, for example, images acquired by other ophthalmic modalities (fundus cameras, OCT devices, SLOs, surgical microscopes, etc.), images acquired by other medical imaging modalities (ultrasound diagnostic devices, X-ray diagnostic devices, X-ray CT devices, magnetic resonance imaging (MRI) devices, etc.), images generated by processing actual images, or pseudo-images. Furthermore, the methods and technologies used in the artificial intelligence engine (e.g., hardware, software, machine learning methods, types of neural networks, etc.) are arbitrary.

[0093] A series of images constituting a set of images collected in a single scan may be associated with multiple positions within the scan range. This association is performed, for example, by the image group processing unit 80. A specific example is described below. The scan range along the X direction is divided into (N-1) sections, the scan start position is set to the first position, and the scan end position is set to the Nth position (N is an integer greater than or equal to 2). This sets N positions within the scan range. The N positions are represented by B1, B2, B3, ..., BN (not shown). The intervals between the N positions B1 to BN may be equal or different. The number of positions B1 to BN (N) is set based on, for example, the scan start position, the scan end position, the slit light movement speed (scan speed), the frame rate of the imaging system 3, etc. In this example, the number of positions B1 to BN (N) set is equal to the number of images collected in a single scan. Note that the number of positions B1 to BN (N) is not limited to this example, nor is the method of setting them limited to this example. Assume that a single scan is applied to the anterior segment of the eye E under examination, and the image group F1, F2, F3, ..., FN shown in Figure 3 is obtained. The image group processing unit 80 can assign image Fn to position Bn. This results in N images Fn (n=1, 2, ..., N) corresponding to N positions Bn (n=1, 2, ..., N).

[0094] The image group may consist only of a series of images, or it may also include additional information. Examples of information included in the image group along with the series of images include various types of supplementary information such as subject information, eye information, date and time of shooting, and shooting conditions. It is also possible to include images obtained with other modalities or examination data acquired by the examination device in the image group. The configuration and operation of the image group evaluation unit 81 will be described later with reference to several examples in Figures 5A to 5D.

[0095] A second example of the image group processing unit 80 will be described. The effectiveness of this example is particularly evident when two or more scans are applied to the eye E under examination. The image group processing unit 80B shown in Figure 4B includes an image set creation unit 82 in addition to the image group evaluation unit 81 similar to that in Figure 4A. The image group processing unit 80B is provided with two or more image groups collected by two or more scans applied to the eye E under examination. The image group evaluation unit 81 in this example may perform quality evaluation on each of these image groups, or it may perform quality evaluation only on the image group collected in the first scan (first scan). The image set creation unit 82 selects a series of images corresponding to the scan range from the two or more image groups input to the image group processing unit 80B and creates an image set.

[0096] The range represented by the series of images constituting the image set may be, for example, a three-dimensional region to which any one of two or more scans is applied, or a three-dimensional region defined based on the scan ranges of at least two of the two or more scans. As an example of the former, the largest or smallest scan range can be adopted from two or more scan ranges to which two or more scans are applied. As an example of the latter, the union or intersection of at least two scan ranges can be adopted.

[0097] Furthermore, the image set may contain only the series of images in question, or it may also contain other information. Examples of information included in the image set along with the series of images include various types of supplementary information such as subject information, eye information, date and time of shooting, and shooting conditions. It is also possible to include images obtained with other modalities or examination data acquired by the examination device in the image set.

[0098] A third example of the image group processing unit 80 will be described. Similar to the second example, the effectiveness of this example is particularly evident when two or more scans are applied to the eye E under examination. The image group processing unit 80C shown in Figure 4C includes an image group evaluation unit 81 similar to that in Figure 4A, as well as an image set creation unit 82 similar to that in Figure 4B, but the image set creation unit 82 in this example includes a selection unit 821.

[0099] The image group processing unit 80C is provided with two or more image groups collected from two or more scans applied to the eye E under examination. The image group evaluation unit 81 in this example may perform a quality evaluation on each of these image groups, or it may perform a quality evaluation only on the image group collected in the first scan (first scan). The image set creation unit 82 selects a series of images corresponding to the scan range from the two or more image groups input to the image group processing unit 80B to create an image set. In this image set creation, the selection unit 821 selects images that satisfy predetermined conditions from among the two or more image groups.

[0100] The predetermined conditions (image selection conditions) may be the same as or different from the image group evaluation conditions described above. For example, if the system is configured such that two or more image groups are provided to the selection unit 821 after the image group evaluation unit 81 has evaluated the quality of each individual image, the selection unit 821 may be configured to select images while taking into account conditions related to the image arrangement (e.g., blinking conditions, eye movement conditions, etc.). Note that the image selection conditions are not limited to these examples, nor is the relationship between the image group evaluation conditions and the image selection conditions limited to these examples.

[0101] The following describes the cases in which the selection unit 821 considers blinking conditions and eye movement conditions. For cases where other conditions are considered, the selection unit 821 can perform the same processing as described later in the specific examples relating to the image group evaluation unit 81 (see Figures 5A to 5D).

[0102] This section describes image selection that takes blinking conditions into account. The blinking condition is a condition to confirm that "the image to be evaluated is not affected by blinking." For images included in a group of two or more images, the selection unit 821 determines, for example, whether the reflected image of the slit light irradiated onto the anterior segment of the eye E under examination is included in the image. This determination is a process that utilizes the fact that the reflected image of the slit light is not depicted in images taken during blinking, and that the reflected image of the slit light is represented as significantly brighter than other areas, and is performed, for example, based on the brightness distribution in the image.

[0103] As an example, the selection unit 821 creates a luminance histogram from the image and determines whether there are pixels with a luminance above a predetermined threshold. If it is determined that there are pixels with a luminance above the predetermined threshold, it is determined that the image contains a reflected image of slit light. This example has the advantage of being extremely simple to process, but it may misdetect high-luminance noise or reflections of ambient light.

[0104] As another example, the selection unit 821 creates a luminance histogram from the image and determines whether there are a predetermined number of pixels with a luminance above a predetermined threshold. If it is determined that there are a predetermined number of pixels with a luminance above a predetermined threshold, it is determined that the image contains a reflected image of slit light. This example has the advantage of preventing the above-mentioned false detection through simple processing.

[0105] This section describes image selection that takes eye movement conditions into consideration. The eye movement condition is a condition to confirm that "the image being evaluated is not affected by eye movement." For images included in a group of two or more images, the selection unit 821 determines the presence or absence of eye movement influence, for example, by comparing the image in question with an adjacent image. This determination utilizes the fact that "image jumps" occur when eye movement occurs during video recording.

[0106] As an example, the selection unit 821 detects landmarks from the image in question and adjacent images, calculates the displacement of these landmarks, and determines whether this displacement is greater than or equal to a predetermined threshold. If it is determined that the displacement is greater than or equal to the predetermined threshold, it is determined that eye movement has occurred. Here, the landmarks may be, for example, the cornea, iris, pupil, or iridocorneal angle. The threshold is calculated based on predetermined scanning conditions, such as the shooting rate of the shooting system 3 or the movement speed of the movement mechanism 6.

[0107] In other examples, the selection unit 821 may be configured to determine the presence or absence of eye movement from a single image. For example, when the imaging rate of the imaging system 3 is slow, rapid eye movement may cause "blur" in the image. The selection unit 821 can determine the presence or absence of eye movement by utilizing blur detection. Blur detection is typically performed using known techniques such as edge detection.

[0108] Similar to the image group evaluation unit 81, the selection unit 821 may include an artificial intelligence engine for determining whether the input image satisfies the above-mentioned image selection conditions.

[0109] The series of images included in the image set created by the image set creation unit 82 may be associated with multiple locations within the scan range. For example, the selection unit 821 may be configured to select images so that one or more images are assigned to each of the multiple locations within the scan range.

[0110] Let's explain with a specific example. As mentioned earlier, the scan range along the X direction is divided into (N-1) intervals, the scan start position is set to the 1st position, and the scan end position is set to the Nth position (where N is an integer greater than or equal to 2). This sets N positions within the scan range. These N positions are represented by B1, B2, B3, ..., BN.

[0111] Assume that two or more scans were performed on the anterior segment of the eye E under examination, resulting in the acquisition of the image group F1, F2, F3, ..., FN shown in Figure 3. While the N images F1-FN shown in Figure 3 are described above as a group obtained from a single scan, for the sake of simplicity, this specification may refer to the N images F1-FN shown in Figure 3 as an arbitrary group of images. For example, in this example, the N images F1-FN are referred to as a series of images included in an image set.

[0112] For these N positions B1 to BN and N images F1 to FN, the selection unit 821 can assign an image Fn to position Bn. This yields N images Fn (n=1, 2, ..., N) corresponding to N positions Bn (n=1, 2, ..., N), and an image set is created in which the image group Fn is a "series of images".

[0113] In some exemplary embodiments, if the image group evaluation unit 81 evaluates that the quality of the image group obtained in the first scan is not good, a second scan is performed automatically or at the user's instruction. If the number of scans applied to the anterior segment is two or more, the slit lamp microscope 1 performs the operation of applying two or more scans to the anterior segment and the operation of selecting a series of images from the two or more image groups collected thereby, but the manner in which these operations are performed is arbitrary. As a first example, the slit lamp microscope 1 may be configured to alternately repeat scanning of the anterior segment and selecting images from the image group acquired by this scan in response to the evaluation that the quality of the image group obtained in the first scan is not good. As a second example, the slit lamp microscope 1 may be configured to perform two or more scans together and then select a series of images from the two or more image groups collected thereby in response to the evaluation that the quality of the image group obtained in the first scan is not good. These two examples will be described below. In addition to or instead of selecting images from an image group, the image group evaluation unit 81 may perform an evaluation of the image group. As mentioned above, image group evaluation and image selection may be similar, analogous, or interchangeable processes. In some exemplary embodiments, the matters described for image group evaluation can be applied to image selection, and conversely, the matters described for image selection can be applied to image group evaluation.

[0114] A first example of a scan and image selection method performed after the image group evaluation unit 81 evaluates that the quality of the image group obtained in the first scan is not good is an alternating iteration of scan and image selection. More specifically, in the first example, for example, the pair of applying a scan to the anterior segment and selecting an image from the image group obtained in this scan is performed a predetermined number of times. That is, in the first example, U pairs of operations are performed in the order of the first pair of operations (scan and image selection), the second pair of operations (scan and image selection), ..., the Uth pair of operations (scan and image selection) (U is an integer of 1 or more). The pair of the first scan performed before this alternating iteration and the quality evaluation of the image group obtained in the first scan (image group evaluation unit 81) will be called the 0th pair of operations.

[0115] Here, the number of scans in the u-th set can be any number greater than or equal to 1 (u=0, 1, ..., U). Also, the number of scans in the u1-th set and the number of scans in the u2-th set can be equal or different (u1=0, 1, ..., U; u2=0, 1, ..., U; u1≠u2).

[0116] In the first example, the selection unit 821 may be configured to create a provisional image set by selecting images from two or more image sets collected in two or more scans already performed. That is, the slit lamp microscope 1 may be configured to create a provisional image set from two or more image sets obtained in two or more scans performed up to that point in time while alternating scans and image selections are being performed. For example, after a scan in the u-th set has been performed, the selection unit 821 is configured to create a provisional image set from all images obtained from the 0th set to the u-th set. With such a configuration, a provisional image set can be created from the images obtained up to that point in order to construct the final image set.

[0117] When the above configuration for creating a provisional image set is applied, the following configurations can be combined. When a new scan is applied to the anterior segment of the eye E under examination, the selection unit 821 first selects images from the new set of images acquired in this new scan. Subsequently, the selection unit 821 creates a new provisional image set by adding the images selected from this new set of images to a provisional image set based on one or more scans performed prior to this new scan. For example, after the scan in the (u+1)th set is performed, the selection unit 821 can first select images from the set of images obtained in the (u+1)th set. Furthermore, the selection unit 821 can create a new provisional image set by adding the images selected from the set of images obtained in the (u+1)th set to a provisional image set based on the set of images obtained from the 0th set to the uth set. With such a configuration, each time a scan is applied to the anterior segment, the provisional image set can be sequentially updated based on the set of images obtained in this scan. This makes it possible to construct the final image set reliably and efficiently.

[0118] When the above configuration for creating (and updating) a provisional image set is applied, the following configurations can be combined. The control unit 7 (or image set creation unit 82 (selection unit 821)) includes an image counter that counts the number of images included in the provisional image set. When the number of images included in the provisional image set reaches a predetermined number, the control unit 7 controls the scanning unit (illumination system 2, shooting system 3, moving mechanism 6) and the selection unit 821 to terminate the alternating repetition of scanning and image selection. Here, the predetermined number is the number of images in the final image set and is set in advance or based on the processing status. The determination of whether the number of images included in the provisional image set has reached the predetermined number is performed by the control unit 7. This determination may be based solely on a comparison of the numbers. Alternatively, if multiple positions in the scan range are associated with a series of images (as described above), it may be determined whether corresponding images have been assigned to all of the multiple positions. With such a configuration, once the number required for the final image set is obtained, the alternating repetition of scanning and image selection can be automatically terminated.

[0119] When the above configuration for creating (and updating) a provisional image set is applied, the following configurations can be further combined. The control unit 7 includes an iteration counter that counts the number of alternating iterations of scanning and image selection. The count may be defined in units of scan and image selection pairs (the 1st pair to the Uth pair), or in units of scan counts. When the number of iterations reaches a predetermined number, the control unit 7 controls the scanning unit (illumination system 2, imaging system 3, moving mechanism 6) and the selection unit 821 to terminate the alternating iterations of scanning and image selection. When the number of iterations is defined in units of scan and image selection pairs, the predetermined number is equal to the total number of pairs (U+1) set in advance. When the number of iterations is defined in units of scan counts, the predetermined number is equal to the total number of scans set in advance. The control unit 7 also determines whether or not the number of iterations has reached a predetermined number. With such a configuration, scanning and image selection can be automatically terminated when they have been repeated a predetermined number of times. Without this configuration, scanning and image selection are repeated until the required number of images for the final image set are selected, which fatigues the subject and reduces imaging efficiency. In particular, when imaging multiple subjects sequentially, the imaging throughput is significantly impaired.

[0120] As described above, this embodiment may be configured to automatically terminate the alternating repetition of scanning and image selection. The conditions for this automatic termination are not limited to the two examples described above, and may also be, for example, instructions from the user. Alternatively, the elapsed time from the start of the alternating repetition of scanning and image selection may be measured, and the alternating repetition may be terminated when a predetermined time is reached. Note that if the repetition rate of scanning and image selection is constant, the automatic termination control based on elapsed time is equivalent to the automatic termination control based on the number of repetitions described above. The image set creation unit 82 can create an image set based on the provisional image set when the alternating repetition of scanning and image selection is terminated. The provisional image set is included in the image set as, for example, a series of images corresponding to the scan range. The slit lamp microscope 1 is separately input with predetermined subject information such as subject ID, public ID, name, age, and gender. The image set creation unit 82 can create an image set by configuring such subject information, subject eye information (information indicating left eye / right eye, etc.), shooting date and time, shooting conditions, etc., as supplementary information for a series of images. The image set may also include other images obtained with the slit lamp microscope 1, images obtained with other modalities, and inspection data acquired by the inspection device. This concludes the explanation of the first example of the scanning and image selection execution method.

[0121] Next, a second example of a scan and image selection method performed after the image group evaluation unit 81 evaluates that the quality of the image group obtained in the first scan is not good will be described. In this example, two or more scans are performed together, and then a series of images are selected from the two or more image groups collected therefrom. Then, images are selected from the two or more image groups collected in these two or more scans and the image group obtained in the first scan performed earlier, and an image set including the selected series of images is created.

[0122] Let's explain a specific example of this process. First, the selection unit 821 associates the group of images corresponding to each scan with multiple locations (as described above) within the scan range. As a result, two or more images corresponding to different scans are assigned to each of the multiple locations within the scan range.

[0123] Next, for each of the multiple locations within the scan range, the selection unit 821 selects one image from among the two or more images assigned to that location. The image selection conditions applied in this example may be, for example, the blinking condition and eye movement condition described above. As a result, one image is assigned to each of the multiple locations within the scan range. The multiple images associated with the multiple locations in this way are adopted as a series of images included in the image set. This concludes the explanation of the second example of the scanning and image selection execution mode.

[0124] A fourth example of the image group processing unit 80 will be described. Similar to the second and third examples, the effectiveness of this example is particularly evident when two or more scans are applied to the eye E under examination. The image group processing unit 80D shown in Figure 4D includes an image group evaluation unit 81 similar to that in Figure 4A, an image set creation unit 82 similar to that in Figure 4B or Figure 4C, and an image set evaluation unit 83.

[0125] The image set evaluation unit 83 evaluates the quality of the image set created by the image set creation unit 82. This evaluation determines whether the image set has sufficient quality to effectively perform diagnosis (image interpretation), and evaluation items and criteria are determined from this perspective. The evaluation items and criteria may be common to at least some of the image group evaluation conditions and / or at least some of the image selection conditions, but are not limited to them.

[0126] The image set evaluation unit 83 may be configured to perform different evaluations depending on the nature of the image set. For example, the evaluation applied to an image set created in response to the number of images in a provisional image set reaching a predetermined number may be different from the evaluation applied to an image set created in response to the number of alternating scan and image selection iterations reaching a predetermined number. Alternatively, the same evaluation may be applied regardless of the nature of the image set.

[0127] Examples of image set quality evaluation include evaluating the quality of each image (which may be the same as the evaluation of the image group), as well as evaluating the "arrangement order" of the series of images, the "missing images," and the "positional misalignment." Problems in image sets, such as changes in arrangement order, missing images, and positional misalignment, can be caused by eye movements or fixation errors.

[0128] The evaluation of the sequence order of a series of images is described below. In some examples, the aforementioned correspondence (one-to-one correspondence) is established between a series of images and multiple positions within the scan range. The image set evaluation unit 83 can use this correspondence to evaluate the sequence order.

[0129] Here, multiple positions within the scan range are assigned an order corresponding to their positional relationships in real space. Let's explain with an example. As mentioned above, the scan range along the X direction is divided into (N-1) sections, and N positions B1, B2, B3, ..., BN are set sequentially from the scan start position to the scan end position. In other words, the N positions B1 to BN are assigned an order corresponding to their positional relationships in real space. Furthermore, N images F1 to FN (a series of images) are associated with the N positions B1 to BN in a one-to-one relationship.

[0130] Under these premises, the image set evaluation unit 83 arranges, for example, N images F1 to FN according to the arrangement order (relative positional relationship) of N positions B1 to BN. This process is realized, for example, by setting the coordinates of N positions B1 to BN in a certain 3D coordinate system and arranging (embedding) the N images F1 to FN according to the set N coordinates. More specifically, the image set evaluation unit 83 can, for example, extract slit light irradiation regions A1 to AN (2D cross-sectional images) from each of the N images F1 to FN, set the coordinates of N positions B1 to BN in a 3D coordinate system, and embed the N 2D cross-sectional images A1 to AN according to the set N coordinates.

[0131] The image set evaluation unit 83 can evaluate whether the arrangement order is appropriate by analyzing the images F1 to FN (2D cross-sectional images A1 to AN) embedded in a 3D coordinate system. For example, the image set evaluation unit 83 can detect areas of interest (image areas corresponding to areas of interest such as the anterior surface of the cornea, posterior surface of the cornea, iris, pupil, anterior surface of the lens, posterior surface of the lens, and anterior chamber angle) from the images F1 to FN (2D cross-sectional images A1 to AN) and perform an evaluation based on the morphology (connectivity, continuity, etc.) of the areas of interest in the arrangement direction (X direction in this example) of the images F1 to FN (2D cross-sectional images A1 to AN). For example, if a gap of a predetermined size or larger exists in the areas of interest, it is determined that the arrangement order is inappropriate (there is a change in the arrangement order).

[0132] In another example, the image set evaluation unit 83 constructs a cross-sectional image along the X direction from images F1 to FN (2D cross-sectional images A1 to AN) embedded in a 3D coordinate system. Furthermore, the image set evaluation unit 83 can perform an evaluation based on the morphology (connectivity, continuity, etc.) of this cross-sectional image.

[0133] The evaluation of image jumps and positional shifts can be performed in the same manner as the evaluation of array order.

[0134] Similar to the image group evaluation unit 81 and the selection unit 821, the image set evaluation unit 83 may include an artificial intelligence engine for evaluating whether the input image set has sufficient quality for a valid diagnosis.

[0135] If the image set evaluation unit 83 evaluates the image set to be of good quality, the control unit 7 may be configured to control the communication unit 9 to transmit this image set. For example, the control unit 7 prepares transmission information including this image set and controls the communication unit 9 to transmit this transmission information to a predetermined external device.

[0136] The methods of outputting image sets, etc., from the slit lamp microscope 1 are not limited to transmission. Examples of output methods other than transmission include saving to a storage device (such as a database), recording to a recording medium, and printing to a printing medium.

[0137] If the image set evaluation unit 83 evaluates that the quality of the image set is not good, the control unit 7 may be configured to perform a new scan of the eye E under examination, that is, to perform a control to acquire a new group of images (a new image set). For example, the control unit 7 may be configured to display and / or output predetermined output information. The predetermined output information may include, for example, information prompting the user to retake the image, such as that the image was not captured or that a retake is necessary.

[0138] Alternatively, if the image set evaluation unit 83 evaluates that the quality of the image set is not good, the control unit 7 may be configured to send commands to at least the scanning unit (illumination system 2, shooting system 3, moving mechanism 6) and the image set creation unit 82 to automatically start reshooting (re-executing scanning and image set creation).

[0139] Furthermore, several examples of the image group evaluation unit 81 will be explained with reference to Figures 5A to 5D. Note that the image group evaluation unit 81 is not limited to these examples, and any modifications (addition, substitution, omission, etc.) are possible. Also, it is possible to combine at least two or more of these examples or modifications in part.

[0140] A first example of the image group evaluation unit 81 will be described with reference to Figures 5A and 5B. This example uses artificial intelligence technology to evaluate the quality of an image group. The image group evaluation unit 81A shown in Figure 5A includes an inference unit 811A that performs quality evaluation of the image group using an inference model 812A.

[0141] The inference model 812A is pre-built by machine learning using training data that includes multiple anterior segment images. The device for building the inference model 812A (inference model building device) may be located in the slit lamp microscope 1 (data processing unit 8, etc.) or its peripheral equipment (computer, etc.), or it may be another computer.

[0142] The model building unit 90 shown in Figure 5B is an example of such an inference model building device and is provided in a slit lamp microscope 1 or its peripheral equipment. The model building unit 90 includes a learning processing unit 91 and a neural network 92.

[0143] Neural networks typically include convolutional neural networks (CNNs). An example of the structure of a convolutional neural network is shown in Figure 5B.

[0144] The input layer receives an image as input. Behind the input layer, multiple pairs of convolutional and pooling layers are arranged. In the example shown in Figure 5B, there are three pairs of convolutional and pooling layers, but the number of pairs is arbitrary.

[0145] In a convolutional layer, a convolution operation is performed to extract features (such as contours) from an image. A convolution operation is a sum-of-products operation of a filter function (weight coefficients, filter kernel) of the same dimension as the input image. In a convolutional layer, the convolution operation is applied to multiple parts of the input image. More specifically, in a convolutional layer, the value of each pixel in the part of the image to which the filter function is applied is multiplied by the value of the filter function corresponding to that pixel (weight), and the product is calculated. The sum of these products is then calculated across multiple pixels in this part of the image. The resulting sum of products is then substituted for the corresponding pixels in the output image. By performing the sum-of-products operation while moving the part of the image to which the filter function is applied, the result of the convolution operation on the entire input image is obtained. This type of convolution operation yields many images in which various features have been extracted using a large number of weight coefficients. In other words, many filtered images such as smoothed images and edge images are obtained. The many images generated by the convolutional layer are called feature maps.

[0146] In the pooling layer, the feature map generated by the preceding convolutional layer is compressed (e.g., data decimation). More specifically, the pooling layer calculates statistical values ​​for predetermined neighboring pixels of a pixel of interest within the feature map at predetermined pixel intervals, and outputs an image with dimensions smaller than the input feature map. The statistical values ​​applied to the pooling operation are, for example, the maximum value (max pooling) or the average value (average pooling). The pixel interval applied to the pooling operation is called the stride.

[0147] Convolutional neural networks can extract many features from an input image by processing it using multiple pairs of convolutional and pooling layers.

[0148] A fully connected layer is provided behind the last pair of convolutional and pooling layers. In the example shown in Figure 2, there are two fully connected layers, but the number of fully connected layers is arbitrary. The fully connected layer uses the features compressed by the combination of convolution and pooling to perform processes such as image classification, image segmentation, and regression. An output layer is provided behind the last fully connected layer to provide the output result.

[0149] In some exemplary embodiments, the convolutional neural network may not include fully connected layers (e.g., a fully convolutional network (FCN)), or it may include support vector machines, recurrent neural networks (RNNs), etc. Furthermore, the machine learning applied to the neural network 92 may be transfer learning. In other words, the neural network 92 may include a neural network that has already been trained using other training data (training images) and whose parameters have been tuned. The model building unit 90 (learning processing unit 91) may also be configured to allow fine-tuning to be applied to the trained neural network (92). The neural network 92 may be constructed using, for example, a known open-source neural network architecture.

[0150] The learning processing unit 91 applies machine learning using training data to the neural network 92. If the neural network 92 includes a convolutional neural network, the parameters adjusted by the learning processing unit 91 include, for example, the filter coefficients of the convolutional layer and the connection weights and offsets of the fully connected layer.

[0151] As mentioned above, the training data includes at least multiple anterior segment images. These multiple anterior segment images are typically, but not limited to, images acquired by a slit-lamp microscope, and may include, for example, images acquired by other ophthalmic modalities (fundus cameras, OCT devices, SLOs, surgical microscopes, etc.), images acquired by imaging diagnostic modalities of other medical departments (ultrasound diagnostic devices, X-ray diagnostic devices, X-ray CT devices, magnetic resonance imaging (MRI) devices, etc.), images generated by processing actual images, and pseudo-images. Furthermore, the number of training data may be increased using techniques such as data augmentation.

[0152] The training method for constructing the inference model may be arbitrary, but may include, for example, supervised learning, unsupervised learning, and reinforcement learning, or a combination of two or more of these.

[0153] In some exemplary embodiments, supervised learning is performed using training data in which input images are labeled with the final output. For example, each of the multiple anterior segment images included in the training data is pre-labeled to indicate whether it is interpretable or not. The labels are generated, for example, by a physician or another inference model. The learning processing unit 91 can construct an inference model 812A by applying supervised learning using such training data to the neural network 92.

[0154] The inference model 812A constructed in this manner is a trained model that takes images obtained by scanning the anterior segment of the eye with a slit light as input and outputs the possibility of interpretation. The possibility of interpretation, which is the output of the inference model 812A, may be any parameter indicating whether the group of images to be evaluated is suitable for interpretation, for example, it may be a determination of whether it is possible or impossible to interpret, the probability that interpretation can be performed, or the validity of the result if interpretation is performed (e.g., accuracy, precision).

[0155] To prevent processing from becoming concentrated on specific units of the neural network 92, the learning processing unit 91 may randomly select and disable several units and perform learning using the remaining units (dropout).

[0156] The methods used to construct inference models are not limited to the examples shown here. For example, any method such as support vector machines, Bayesian classifiers, boosting, k-means algorithms, kernel density estimation, principal component analysis, independent component analysis, self-organizing maps, random forests, and generative adversarial networks (GANs) can be used to construct inference models.

[0157] The inference unit 811A shown in Figure 5A uses such an inference model 812A to evaluate the quality of the image set acquired in a single scan of the anterior segment of the eye E under examination. More specifically, first, the inference unit 811A inputs the image set or each image contained therein into the inference model 812A. The inference model 812A derives information indicating the possibility of interpretation from the input image set or images.

[0158] The inference unit 811A may use the image interpretation possibility information output from the inference model 812A as the inference result, or it may generate an inference result based on the image interpretation possibility information. As an example of the latter, the inference unit 811A may generate display information from the image interpretation possibility information, or it may perform predetermined statistical processing.

[0159] A second example of the image group evaluation unit 81 will be described with reference to Figure 5C. This example performs quality evaluation by converting the image group into a three-dimensional image. The image group evaluation unit 81B shown in Figure 5C includes a three-dimensional image construction unit 811B, a comparison unit 812B, and an evaluation processing unit 813B.

[0160] The 3D image construction unit 811B constructs a 3D image from a set of images acquired in a single scan of the anterior segment of the eye E under examination. For example, the 3D image construction unit 811B may be configured to construct stack data by embedding the image set into a single 3D coordinate system. This stack data is constructed, for example, by embedding N 2D cross-sectional images A1 to AN, as shown in Figure 3, into a 3D coordinate system according to the aforementioned arrangement (relative positional relationship) of N positions B1 to BN.

[0161] The 3D image construction unit 811B may be configured to construct volume data by applying a voxelization process to stack data constructed from a group of images. Furthermore, the 3D image construction unit 811B can apply a predetermined rendering to the stack data or volume data. Examples of rendering include volume rendering and surface rendering.

[0162] The comparison unit 812B compares the 3D image (stack data, volume data, rendered image, etc.) constructed by the 3D image construction unit 811B with a predetermined reference 3D image. One or any number of reference 3D images are prepared.

[0163] The reference 3D image may include one or more reference 3D images corresponding to a normal eye. The reference 3D image corresponding to a normal eye may, for example, be an image obtained by photographing a normal eye (an eye in which no disease was observed (healthy eye)). The imaging modality for obtaining this reference 3D image may be arbitrary, but typically it is a slit lamp microscope 1 or a similar slit lamp microscope. Furthermore, the reference 3D image corresponding to a normal eye may be an image obtained by photographing a model of a normal eye (such as a model eye), or an image created by computer graphics from a model of a normal eye or a clinical case.

[0164] The reference 3D image may include one or more reference 3D images corresponding to the affected eye. The reference 3D image corresponding to the affected eye may be, for example, an image obtained by photographing an eye that has been definitively diagnosed with a specific disease. The imaging modality for obtaining this reference 3D image may be arbitrary, but is typically a slit lamp microscope 1 or a similar slit lamp microscope. The reference 3D image corresponding to the affected eye may also be an image obtained by photographing a model of the affected eye (such as a model eye), or an image created by computer graphics from a model of the affected eye or a clinical case.

[0165] The comparison unit 812B performs matching between the 3D image constructed by the 3D image construction unit 811B and a reference 3D image to calculate the values ​​of predetermined parameters. This image matching may utilize any method, such as image correlation, feature-based matching, region-based matching, or machine learning (a trained model). The calculated parameters may be any parameters, such as correlation values, matching parameters (angle, scale, similarity, goodness of fit, etc.), or output parameters of a trained model.

[0166] This type of image matching typically involves quantitatively (i.e., numerically) representing how similar the shape and structure of tissues and parts (such as the cornea, iris, pupil, and iridocorneal angle) depicted in a three-dimensional image are to the shape and structure of a standard normal eye, and / or to the shape and structure of a standard affected eye.

[0167] The evaluation processing unit 813B evaluates the quality of the image group based on the parameter values ​​calculated by the comparison unit 812B. For example, the evaluation processing unit 813B may be configured to evaluate the quality of the image group by comparing the parameter values ​​calculated by the comparison unit 812B with a predetermined threshold. Alternatively, the evaluation processing unit 813B may be configured to evaluate the quality of the image group by determining whether the parameter values ​​calculated by the comparison unit 812B fall within a predetermined range. The processing methods performed by the evaluation processing unit 813B are not limited to these, and any method that can be used to derive an evaluation result from a certain parameter value may be used.

[0168] The quality evaluation of an image group using 3D image construction is not limited to this example. For example, the image group evaluation unit 81 may be configured to perform one or more evaluation processes, such as evaluating the arrangement order of a series of images constituting the image group, evaluating image gaps, and evaluating positional misalignment, in the same manner as the image set evaluation unit 83.

[0169] A third example of the image group evaluation unit 81 will be described with reference to Figure 5D. This example evaluates quality by quantitatively evaluating the image quality of the image group. The image group evaluation unit 81C shown in Figure 5D includes an evaluation data generation unit 811C and an evaluation processing unit 812C.

[0170] The evaluation data generation unit 811C obtains image quality evaluation data from the images included in the image set collected in a single scan of the anterior segment of the eye E under examination. The image quality evaluation data quantitatively represents the image quality of the image set.

[0171] Several examples of image quality evaluation processing performed by the evaluation data generation unit 811C are described below. In some exemplary embodiments, the image quality evaluation processing performed by the evaluation data generation unit 811C may be any processing, such as processing using any known technique, including signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), root mean square (RMS) granularity, Wiener spectrum, modulation transfer function (MTF), and quality index (QI).

[0172] For example, the evaluation data generation unit 811C calculates a predetermined image quality evaluation index value (image quality evaluation value) as image quality evaluation data for the image group. The image quality evaluation value may be any parameter that quantitatively expresses the quality of the image, and typically, the higher the image quality, the larger the image quality evaluation value.

[0173] As an example of a method for calculating image quality evaluation values, the method for calculating the Image Quality value (IQ value) used for evaluating the image quality of OCT images is described below. First, the evaluation data generation unit 811C detects image regions corresponding to the tissue (part) of the anterior segment (tissue image region) and other image regions (background region, non-tissue image region) by applying a predetermined analysis process (e.g., segmentation) to the evaluation region set in the image to be evaluated. Next, the evaluation data generation unit 811C generates a histogram of brightness in the tissue image region and a histogram of brightness in the background region. Subsequently, the evaluation data generation unit 811C calculates the image quality evaluation value (IQ value) from the degree of overlap between these two histograms. For example, the IQ value is defined in the range [0,100] such that the IQ value is 0 when both histograms completely overlap, and 100 when both histograms completely separate. This image quality evaluation calculation may include, for example, normalization of two histograms, generation of a probability distribution function, and calculation of an IQ value using a predetermined calculation formula.

[0174] Thus, the evaluation data generation unit 811C may be configured to perform the following processes: identify a tissue image region corresponding to the anterior segment tissue and a background region in the image to be evaluated; create a first histogram showing the frequency distribution of brightness in the tissue image region; create a second histogram showing the frequency distribution of brightness in the background region; and calculate an image quality evaluation value (IQ value) as image quality evaluation data based on the first and second histograms.

[0175] The evaluation processing unit 812C evaluates the quality of the image group based on the image quality evaluation data generated by the evaluation data generation unit 811C. Several methods for quality evaluation based on image quality evaluation data are described below, but the quality evaluation method is not limited to these and may be arbitrary.

[0176] The first example of quality evaluation based on image quality evaluation data is described below. When an IQ value is obtained for each image constituting a group of images, the evaluation processing unit 812C compares each of the multiple IQ values ​​obtained for this group of images with a predetermined threshold. If all of the multiple IQ values ​​are equal to or greater than the threshold, the evaluation processing unit 812C determines that the quality of this group of images is good. On the other hand, if any of the multiple IQ values ​​are below the threshold, the evaluation processing unit 812C determines that the quality of this group of images is not good.

[0177] A second example of quality evaluation based on image quality evaluation data is described below. When an IQ value is obtained for each image constituting a group of images, the evaluation processing unit 812C selects the lowest IQ value from among the multiple IQ values ​​obtained for this group of images and compares this lowest IQ value with a predetermined threshold. If the lowest IQ value is equal to or greater than the threshold, the evaluation processing unit 812C determines that the quality of this group of images is good. On the other hand, if the lowest IQ value is less than the threshold, the evaluation processing unit 812C determines that the quality of this group of images is not good.

[0178] A third example of quality evaluation based on image quality evaluation data is described below. When an IQ value is obtained for each image constituting the image group, the evaluation processing unit 812C calculates a statistical value by applying a predetermined statistical operation to the multiple IQ values ​​obtained for this image group. The type of this statistical value can be arbitrary, for example, the mean, minimum, maximum, mode, median, etc. Note that the case where the statistical value is the minimum corresponds to the second example described above. The evaluation processing unit 812C compares the calculated statistical value with a predetermined threshold. If the statistical value is greater than or equal to the threshold, the evaluation processing unit 812C determines that the quality of this image group is good. On the other hand, if the statistical value is less than the threshold, the evaluation processing unit 812C determines that the quality of this image group is not good.

[0179] The processing performed by the evaluation processing unit 812C is not limited to processing based on image quality evaluation data generated by the evaluation data generation unit 811C. For example, the evaluation processing unit 812C may be configured to perform one or more of the following evaluation processes: evaluation of the arrangement order of a series of images constituting a group of images, evaluation of image gaps, and evaluation of positional shifts (all of which have been described above).

[0180] <Processed Image Construction Unit 85> The processed image construction unit 85 constructs a processed image from the image set collected by the anterior segment scan using slit light. At least one image included in the image set is used for processed image construction. Processed image construction may include any image processing. Several examples are described below, but the processing included in processed image construction is not limited to these.

[0181] In some exemplary embodiments, the processed image construction unit 85 may be capable of performing 3D image construction, which constructs a 3D image from a group of images. The 3D image constructed from the group of images may be, for example, stack data or volume data. The processed image construction unit 85 in this example may have a configuration similar to that of the 3D image construction unit 811B described above. For example, the combination of hardware and software constituting the processed image construction unit 85 and the combination of hardware and software constituting the 3D image construction unit 811B may be at least partially common.

[0182] In some exemplary embodiments, the processed image construction unit 85 may be capable of performing rendering processes to construct a rendered image of a 3D image. The processed image construction unit 85 can, for example, apply a predetermined rendering process to stack data or volume data obtained by 3D image construction. Examples of rendering processes include volume rendering and surface rendering. The processed image construction unit 85 in this example may have a configuration similar to that of the 3D image construction unit 811B.

[0183] In some exemplary embodiments, the processed image construction unit 85 may be capable of performing processes such as identifying a sub-region of an image (segmentation) and extracting the identified sub-region (cropping, trimming). For example, the processed image construction unit 85 may be capable of extracting a region of interest corresponding to a predetermined part of the eye E under examination from an image containing a group of images, a three-dimensional image, or a rendered image.

[0184] <Position deviation information acquisition unit 86> The positional displacement information acquisition unit 86 operates after follow-up imaging using the slit lamp microscope 1. Follow-up imaging is performed by referring to previously acquired images of the anterior segment of the eye E (first image, reference image), and a new image of the anterior segment of the eye E (second image, follow-up image) is acquired. The reference image and follow-up image are stored in the storage unit 10.

[0185] The control unit 7 reads the reference image and follow-up image from the storage unit 10 and sends them to the positional displacement information acquisition unit 86. The positional displacement information acquisition unit 86 analyzes the reference image referenced in the follow-up imaging and the follow-up image acquired in this follow-up imaging to acquire positional displacement information between the reference image and the follow-up image. Positional displacement information is information that quantitatively indicates how much the position (range, region) of the anterior segment depicted in the follow-up image is shifted from that of the reference image.

[0186] The positional displacement information acquisition unit 86 calculates, for example, translational displacement and rotational displacement between the reference image and the follow-up image as positional displacement information. Translational displacement includes, for example, information indicating the amount of translational displacement (translational deviation) of the follow-up image relative to the reference image in the XY coordinate system. Translational displacement is not limited to this and may also include translational deviation information in any of the YZ coordinate system, ZX coordinate system, and XYZ coordinate system. Similarly, rotational displacement includes, for example, information indicating the amount of rotational displacement (rotational displacement) of the follow-up image relative to the reference image in the XY coordinate system centered on a predetermined position in the XY coordinate system. Rotational displacement is not limited to this and may also include rotational deviation information in any of the YZ coordinate system, ZX coordinate system, and XYZ coordinate system. Translational displacement and rotational displacement are expressed, for example, as an affine transformation between the coordinates of the reference image and the coordinates of the follow-up image. Note that only one of translational displacement or rotational displacement may be calculated.

[0187] Examples of methods for calculating translational and rotational displacement are described below. In some exemplary embodiments, the reference image may include a first scan image obtained by scanning the anterior segment of the eye E under examination with a slit lamp. The first scan image is an image previously obtained by a slit lamp microscope 1 or other device. On the other hand, the follow-up image may include a second scan image obtained in a follow-up imaging performed with reference to the first scan image. The positional displacement information acquisition unit 86 can acquire positional displacement information based on at least the first scan image and the second scan image.

[0188] The follow-up image may include at least one image (a two-dimensional cross-sectional image) from the image group collected by follow-up imaging (anterior segment scan using slit light). In this case, the positional displacement information acquisition unit 86 performs a process to identify, for example, a region in the reference image (a cross-section, or a three-dimensional region spanning two or more images) corresponding to the cross-section represented by the two-dimensional cross-sectional image as a follow-up image. If the reference image includes a group of images (or at least one image (a two-dimensional cross-sectional image) included therein), the positional displacement information acquisition unit 86 can identify a two-dimensional cross-sectional image in the group of reference images that corresponds to the position of the cross-section represented by the two-dimensional cross-sectional image as a follow-up image. Here, the positional displacement information acquisition unit 86 can select a two-dimensional cross-sectional image from the group of reference images that corresponds to the cross-section closest to the cross-sectional position of the two-dimensional cross-sectional image as a follow-up image. Alternatively, the positional displacement information acquisition unit 86 can select two or more two-dimensional cross-sectional images located near the cross-sectional position of the two-dimensional cross-sectional image used as a follow-up image from a group of reference images, synthesize the two or more selected two-dimensional cross-sectional images to construct a new two-dimensional cross-sectional image, and use this new two-dimensional cross-sectional image as a reference image.

[0189] When the follow-up images include at least a portion of the image group, the reference image may be a 3D image (stack data, volume data, etc.) or a rendered image. In this case, the positional displacement information acquisition unit 86 can, for example, perform a process to identify a region (cross-section, 3D region, etc.) in the reference image that corresponds to the cross-section represented by the 2D cross-sectional image as the follow-up image. In another example, the positional displacement information acquisition unit 86 can construct a processed image (3D image, rendered image, etc.) from multiple 2D cross-sectional images as follow-up images, and construct a reference image corresponding to the cross-section represented by the 2D cross-sectional image as the follow-up image based on the processed image as the reference image and the processed image as the follow-up image.

[0190] The follow-up image may include a processed image constructed by the processed image construction unit 85. As mentioned above, the processed image may be, for example, a 3D image (stack data, volume data, etc.), a rendered image, etc. If the follow-up image includes a processed image, the positional shift information acquisition unit 86 performs a process to identify, for example, a region (cross section, 3D region, etc.) in the reference image that corresponds to the region represented by the processed image as the follow-up image. If the reference image includes a group of images (or at least one image included therein (a 2D cross-sectional image)), the positional shift information acquisition unit 86 can identify a region in the group of images as the reference image that corresponds to the position of the region represented by the processed image as the follow-up image. On the other hand, if the reference image includes a processed image (3D image, rendered image, etc.), the positional shift information acquisition unit 86 can construct a reference image corresponding to the region represented by the processed image as the follow-up image based on the processed image as the reference image and the processed image as the follow-up image.

[0191] The comparative analysis of the reference image and the follow-up image may include any image processing, such as feature point extraction, segmentation, labeling, pattern matching, template matching, thresholding, binarization, edge detection, image correlation, and masking.

[0192] An example of the process for generating positional displacement information is described below. First, the positional displacement information acquisition unit 86 analyzes a reference image to identify image positions (pixel positions) corresponding to predetermined feature areas in the anterior segment of the eye E under examination. Similarly, the positional displacement information acquisition unit 86 analyzes a reference image to identify image positions (pixel positions) corresponding to the feature areas. The feature areas may be, for example, the corneal apex, corneal margin, pupillary center, pupillary margin, feature points of the iris pattern, iridocorneal angle, lesion area, etc.

[0193] Next, the positional shift information acquisition unit 86 determines the displacement between the coordinates of the image position identified in the reference image and the coordinates of the image position identified in the follow-up image. In this process, for example, for each of the image positions of multiple feature regions, the components of the affine transformation matrix are determined by substituting the coordinates of the image position in the reference image and the coordinates of the image position in the follow-up image into the transformation formula of the two-dimensional affine transformation. Here, the coordinates of the image position are address information pre-assigned to each pixel. The affine transformation matrix obtained in this way contains information on both translational and rotational shifts between the reference image and the follow-up image.

[0194] <Evaluation Value Calculation Unit 87> The evaluation value calculation unit 87 calculates an evaluation value of the scan position error in follow-up imaging based on the positional misalignment information acquired by the positional misalignment information acquisition unit 86. Examples of evaluation values ​​include an evaluation value calculated based on the area of ​​a predetermined image region defined by the scan position of the reference image and the scan position of the follow-up image, and an evaluation value based on the deviation between the scan position of the reference image and the scan position of the follow-up image. Note that the factors for calculating the evaluation value are not limited to these, and it is also possible to calculate the evaluation value based on other factors. Furthermore, the evaluation value may be calculated by combining different factors.

[0195] Figure 6 shows an example of the configuration of the evaluation value calculation unit 87. In this example, the evaluation value calculation unit 87A includes a relative position calculation processing unit 871, a judgment processing unit 872, an area calculation processing unit 873, a displacement calculation processing unit 874, and an evaluation value calculation processing unit 875. The calculation of the evaluation value based on area is performed by the relative position calculation processing unit 871, the judgment processing unit 872, and the area calculation processing unit 873. The calculation of the evaluation value based on displacement is performed by the relative position calculation processing unit 871, the displacement calculation processing unit 874, and the evaluation value calculation processing unit 875.

[0196] The follow-up imaging information (described later) stored in the memory unit 10 includes reference scan position information indicating the scan position corresponding to the reference image (reference scan position) and follow-up scan position information indicating the scan position corresponding to the follow-up image acquired in this follow-up imaging (follow-up scan position). The relative position calculation processing unit 871 is configured to calculate the relative position information between the reference scan position and the follow-up scan position based on the position shift information acquired by the position shift information acquisition unit 86.

[0197] As mentioned above, follow-up imaging involves performing a new scan by reproducing the position where a scan was previously applied. Therefore, ideally, the reference scan position and the follow-up scan position should be the same. However, considering the effects of eye movements and the imaging environment, it is practically difficult to perfectly reproduce the scan position. On the other hand, if these effects are absent, the reference scan position and the follow-up scan position can be matched. Therefore, the relative position between these scan positions corresponds to the positional displacement of the eye E due to eye movements, etc., that is, the positional displacement between the reference image and the follow-up image. This positional displacement is acquired as positional displacement information by the positional displacement information acquisition unit 86.

[0198] The relative position calculation processing unit 871 determines the position corresponding to the reference scan position in the follow-up image, that is, the ideal scan position achieved when the reference scan position is perfectly reproduced, by, for example, shifting the reference scan position by the amount of positional shift indicated in the positional shift information. The relative position information indicates the relative position between this ideal scan position and the scan position (follow-up scan position) actually performed during follow-up imaging. Thus, the relative position information is essentially equivalent to the positional shift information. Note that positional shift information represents the displacement between images, while relative position information represents the displacement between scan positions. The relative position calculation processing unit 871 sends the acquired relative position information to the judgment processing unit 872 and the displacement calculation processing unit 874.

[0199] The relative position between the reference scan position and the follow-up scan position can take the form shown in Figures 7A to 7D, for example. In these figures, the symbol R indicates the follow-up scan position, and the symbol R0 indicates the reference scan position. The reference scan position R0 is obtained by displacement based on positional misalignment information (relative position information). As can be seen from the above explanation of anterior segment scanning using slit light and from Figure 3, the position (region) to which the scan is applied is a three-dimensional region of the anterior segment, and the shape of this three-dimensional region in the XY coordinate system, that is, the shape of the projection image of this three-dimensional region onto the XY plane, is substantially rectangular. In this example, the follow-up scan position R indicates the major axis (axis along the scan direction, axis in the X direction) of the XY projection image of the three-dimensional region of the anterior segment to which the follow-up scan is applied. Similarly, the reference scan position R0 indicates the major axis of the XY projection image of the three-dimensional region to which the reference scan is applied.

[0200] The definitions of the reference scan position and follow-up scan position are not limited to these. For example, the scan position can be defined based on the top edge, bottom edge, side edges, diagonals, vertices, etc., of the XY projection image. Furthermore, the definitions of the reference scan position and follow-up scan position are not limited to the two-dimensional definitions shown in this example, but may also be three-dimensional definitions based on a three-dimensional region, for example.

[0201] Figure 7A shows an example where only translational displacement is present. Figure 7B shows an example where only rotational displacement is present and both scan positions intersect near the center. Figure 7C shows an example where only rotational displacement is present and both scan positions intersect near the scan start point. Figure 7D shows an example where both translational and rotational displacement are present. Note that Figures 7A to 7D are merely examples of the relative positions of the scan positions, and the relative positions of the scan positions are not limited to these.

[0202] The decision processing unit 872 determines, based on the positional misalignment information (relative positional information), whether there is a common position between the reference scan position and the follow-up scan position. A common position refers to a position (region) where the reference scan position and the follow-up scan position overlap, that is, a position (region) where the reference scan position and the follow-up scan position intersect.

[0203] The common location can be any of the following: a 0-dimensional region (point), a 1-dimensional region (line), a 2-dimensional region (plane), or a 3-dimensional region (solid). In Figures 7A to 7D, both the reference scan location and the follow-up scan location are represented by line segments (arrows). If these intersect, the common location is a point (intersection); if they do not intersect, there is no common location. Although not shown in the illustration, if the deviation of the follow-up scan location relative to the reference scan location has only a component in the length direction of these scan locations, and the amount of this deviation is less than the length of these scan locations, then the common location of these two scan locations is a 1-dimensional region.

[0204] The determination of whether or not a common location exists is performed based on the coordinates of the reference scan location and the follow-up scan location in the follow-up image. For example, the determination processing unit 872 compares the coordinates of the reference scan location and the coordinates of the follow-up scan location, and determines that a common location exists if there are common coordinates between these scan locations, and determines that no common location exists otherwise. Through this process, the coordinates of the common location in the follow-up image are also obtained. These coordinates are defined, for example, in a coordinate system that is predefined in the follow-up image.

[0205] The decision processing unit 872 sends the decision result regarding the common location (whether it exists or not, the coordinates of the common location, etc.) to the area calculation processing unit 873.

[0206] The area calculation processing unit 873 calculates the area of ​​a two-dimensional region defined by the reference scan position and the follow-up scan position based on the positional displacement information (relative positional information). As mentioned above, positional displacement information and relative positional information are substantially equivalent. The two-dimensional region for which the area is to be calculated is a subregion of the follow-up image. The method for defining the two-dimensional region from the two scan positions is predetermined. The area calculation process can be performed in any manner. The area calculation process may include, for example, a process for counting the number of pixels in the two-dimensional region, a process for obtaining a mathematical expression (formula, etc.) that indicates the boundary region of the two-dimensional region, and integral calculations. Furthermore, the calculated area only needs to be a numerical value that uniquely represents the dimensions of the two-dimensional region, such as the perimeter or cross-section.

[0207] The area calculation processing unit 873 performs different calculations in the area calculation process depending on the judgment result of the judgment processing unit 872. For example, the area calculation processing unit 873 performs different calculations depending on whether or not there is a common position between the reference scan position and the follow-up scan position. Specific examples are described below. In these specific examples, when the two scan positions are defined by line segments, the cases where the two scan positions intersect and the cases where they do not are considered separately. That is, the cases where a common position exists and the cases where it does not exist are considered separately. Furthermore, when the two scan positions intersect, the cases where the intersection point is an endpoint of the scan position and the cases where it is not an endpoint are considered separately. Determining whether or not the intersection point is an endpoint can be easily done by referring to the coordinates of the two scan positions and the coordinates of the intersection point. Note that the endpoints of the scan positions refer to the start point and end point of scanning for a line segment scan position. Also, in the following specific examples, since the scan positions are line segments, there is only one intersection point, but if the scan positions are curved, there may be two or more intersection points.

[0208] As shown in Figures 7B and 7C, when two scan locations intersect, the area calculation processing unit 873 calculates the area of ​​the two-dimensional region defined by the endpoints of the two scan locations and the intersection point of the two scan locations. More specifically, the area calculation processing unit 873 calculates the sum of the area of ​​the triangle formed by the endpoints of the two scan locations located on one side of the intersection point and the intersection point of the two scan locations, and the area of ​​the triangle formed by the endpoints of the two scan locations located on the other side and the intersection point of the two scan locations, and defines this as the area of ​​the two-dimensional region.

[0209] For example, as shown in Figure 7B, if an intersection point exists at a location other than the endpoint of the scan position, the area calculation processing unit 873 calculates the area of ​​triangle TR1, whose vertices are the endpoints (end points of scan) R0E and RE of the two scan positions R0 and R located to the right of the intersection point C of the two scan positions R0 and R, and the intersection point C, as shown in Figure 8A. The area calculation processing unit 873 also calculates the area of ​​triangle TR2, whose vertices are the endpoints (start points of scan) R0S and RS of the two scan positions R0 and R located to the left of the intersection point C, and the intersection point C. Furthermore, the area calculation processing unit 873 adds the area of ​​triangle TR1 and the area of ​​triangle TR2. This sum is the area of ​​the target two-dimensional region.

[0210] As another example, if two scan positions intersect at their respective endpoints (scan start points) as shown in Figure 7C, the area calculation processing unit 873 calculates the area of ​​a triangle TR whose vertices are the intersection point C of the two scan positions R0 and R (both scan start points), the scan end point of the reference scan position R0, and the scan end point of the follow-up scan position R, as shown in Figure 8B.

[0211] If two scan locations intersect at a point other than the endpoint of one scan location and the endpoint of the other scan location, for example, the area of ​​a triangle whose vertices are the intersection point of the two scan locations, the opposite endpoint of one scan location, and one endpoint of the other scan location (for example, an endpoint located on the same side as the opposite endpoint) can be calculated.

[0212] As shown in Figures 7A and 7D, if the two scan positions R0 and R do not intersect, the area calculation processing unit 873 can calculate the area of ​​the quadrilateral defined by the two scan positions R0 and R, and define the area of ​​this quadrilateral as the area of ​​the target two-dimensional region. More specifically, when the two scan positions R0 and R do not intersect, the area calculation processing unit 873 can find the area of ​​a quadrilateral QU, as shown in Figures 8C and 8D, where the reference scan position R0 and the follow-up scan position R are two sides, and the other two sides are two line segments (two non-intersecting line segments) connecting the endpoints of the two scan positions R0 and R.

[0213] The area calculation processing unit 873 sends the area obtained as described above to the evaluation value calculation processing unit 875. In this example, the evaluation value is calculated using area, but the dimensional parameter used in calculating the evaluation value is not limited to area and may be any parameter (for example, volume, length, etc.).

[0214] The deviation calculation processing unit 874 calculates the deviation between the reference scan position and the follow-up scan position based on the positional deviation information (relative positional information) generated by the positional deviation information acquisition unit 86. The method for calculating this deviation is arbitrary. An example of a deviation calculation method will be explained with reference to Figure 9.

[0215] Figure 9 shows the reference scan position R0 and the follow-up scan position R, which are linear segments. Note that, in calculating the deviation, it is not necessary to consider whether the two scan positions R0 and R intersect.

[0216] The deviation calculation processing unit 874 first identifies predetermined positions on the reference scan position R0 and predetermined positions on the follow-up scan position R. This identification process can be performed based on the coordinates of each point at the scan position R0(R). Examples of predetermined positions to be identified include the scan start point, scan end point, and midpoint. When considering scan positions in forms other than line segments, predetermined positions corresponding to those forms can be used. Furthermore, the same type of predetermined position is applied to the two scan positions being compared. For example, if the scan start point is used as the predetermined position, the scan start point is identified from each scan position. Alternatively, multiple types of predetermined positions may be identified from each scan position. In that case, each identified predetermined position is associated with its respective type.

[0217] Next, the displacement calculation processing unit 874 calculates the displacement between the coordinates of a predetermined position at the reference scan position R0 and the coordinates of a predetermined position at the follow-up scan position R. This displacement calculation process can be performed, for example, using a predefined two-dimensional coordinate system in the follow-up image. Alternatively, the displacement may be determined by counting the number of pixels between the two predetermined positions. The displacement calculated here is defined as the displacement between predetermined positions of the same type. For example, the calculated displacement is the displacement between the two scan start points, and not the displacement between the scan start point and the midpoint.

[0218] Figure 9 shows the cases where three types of predetermined positions are applied: the scan start point, the scan end point, and the midpoint. In Figure 9, the deviation between the scan start point R0S of the reference scan position R0 and the scan start point RS of the follow-up scan position R is indicated by the symbol DS. The deviation between the scan end point R0E of the reference scan position R0 and the scan end point RE of the follow-up scan position R is indicated by the symbol DE. The deviation between the midpoint R0M of the reference scan position R0 and the midpoint RM of the follow-up scan position R is indicated by the symbol DM.

[0219] The deviation calculation processing unit 874 sends the deviation obtained in the manner described above to the evaluation value calculation processing unit 875.

[0220] The evaluation value calculation processing unit 875 calculates a value (evaluation value) to evaluate the error in scan position during follow-up imaging. The area information obtained by the area calculation processing unit 873 and the displacement information obtained by the displacement calculation processing unit 874 are input to the evaluation value calculation processing unit 875 as information used in this evaluation value calculation process. Below, examples of evaluation value calculation processing based on area, examples of evaluation value calculation processing based on displacement, and examples of evaluation value calculation processing combining both will be described.

[0221] The types of information (parameters) used in the evaluation value calculation process are not limited to these, and may be any type of information that reflects the error in the scan position during follow-up imaging. The evaluation value calculation processing unit 875 calculates the evaluation value by performing predetermined processing according to the type of information used.

[0222] As an example of the process of obtaining an evaluation value from the area of ​​the image region calculated by the area calculation processing unit 873, the evaluation value calculation processing unit 875 can perform a calculation that subtracts the product of the area of ​​this image region and a predetermined weight from a predetermined maximum value of the evaluation value.

[0223] The calculation in this example can be expressed as follows: S1 = S 1,max -a × (Area). Here, S1 represents the evaluation value based on area, S 1,max S represents the maximum value of the evaluation value set in advance, a represents the weight set in advance, and Area represents the area calculated by the area calculation processing unit 873. Maximum value S 1,max The value can be set arbitrarily, and the weight a can also be set arbitrarily. Maximum value S 1,max For example, this is set to 100. The weight a is set based on, for example, the magnitude of the maximum value setting, the magnitude of the area value, etc.

[0224] According to the calculation process in this example, the smaller the area, that is, the smaller the error in the scan position during follow-up imaging, the larger the evaluation value S1 becomes.

[0225] As an example of the process of obtaining an evaluation value from the deviation calculated by the deviation calculation processing unit 874, the evaluation value calculation processing unit 875 can perform a calculation that subtracts the product of this deviation and a predetermined weight from a predetermined maximum value of the evaluation value.

[0226] The calculation in this example can be expressed as follows: S² = S 2,max -(b×DM+c×DS+d×DE). Here, S2 represents the evaluation value based on deviation, S 2,max represents the maximum value of the evaluation value set in advance, b, c, and d represent the weights set in advance, and DM, DS, and DE represent the deviations of the midpoint, scan start point, and scan end point, respectively, calculated by the deviation calculation processing unit 874. Maximum value S 2,max And the weights b to d can each be set arbitrarily. For example, the maximum value S 2,max The value is set to 100, and weights b to d are set based on the magnitude of the maximum value, the magnitude of the area value, etc.

[0227] According to the calculation process in this example, the smaller the deviation, that is, the smaller the error in the scan position during follow-up imaging, the larger the evaluation value S2 becomes. In this example, the evaluation value is calculated considering three points: the midpoint, the scan start point, and the scan end point. However, it is also possible to calculate the evaluation value considering only one or two of these points. Furthermore, it is also possible to calculate the deviation at points other than these and consider them.

[0228] As an example of a process to obtain an evaluation value based on both the area of ​​the image region calculated by the area calculation processing unit 873 and the displacement calculated by the displacement calculation processing unit 874, the evaluation value calculation processing unit 875 can use a calculation formula that integrates each of the above cases.

[0229] The formula in this example can be expressed as follows: S=Smax -a × (Area) - (b × DM + c × DS + d × DE). Here, S represents an evaluation value based on area and displacement, and S max represents the maximum value of the preset evaluation value, a to d represent preset weights, Area represents the area calculated by the area calculation processing unit 873, and DM, DS, and DE respectively represent the displacements of the midpoint, scan start point, and scan end point calculated by the displacement calculation processing unit 874. The maximum value S max and the weights a to d can be arbitrarily set respectively. For example, the maximum value S max is set to 100, and the weights a to d are set respectively based on the magnitude of the set value of this maximum value, the magnitude of the numerical value of the area, etc.

[0230] According to the arithmetic processing of this example, the smaller the area and / or displacement, that is, the smaller the error of the scan position in the follow-up shooting, the larger the value of the evaluation value S.

[0231] The information of the evaluation value calculated by the evaluation value calculation processing unit 875 (evaluation value calculation unit 87) in this way is sent to the pass / fail determination unit 88.

[0232] <Pass / Fail Determination Unit 88> The pass / fail determination unit 88 determines the suitability (whether it was performed well, whether it was appropriately implemented) of the follow-up shooting based on the evaluation value calculated by the evaluation value calculation unit 87. This determination process is performed, for example, by comparing the evaluation value calculated by the evaluation value calculation unit 87 with a preset numerical range.

[0233] A specific example of the processing performed by the quality determination unit 88 will be explained. In cases where the evaluation value increases as the error decreases, as in some of the examples above, the quality determination unit 88 compares the calculated evaluation value with a predetermined threshold and determines whether the evaluation value is equal to or greater than the threshold. If it is determined that the evaluation value is equal to or greater than the threshold, the quality determination unit 88 obtains a result indicating that the follow-up was performed in a good condition. Conversely, if it is determined that the evaluation value is less than the threshold, the quality determination unit 88 obtains a result indicating that the follow-up imaging was not performed in a good condition.

[0234] While the example described here uses a single threshold, two or more thresholds may be used to determine the quality of follow-up imaging in stages.

[0235] The pass / fail determination unit 88 sends the acquired determination result to the control unit 7. The control unit 7 can store this determination result in the storage unit 10. The control unit 7 can display this determination result on a display device. The control unit 7 can control the communication unit 9 to transmit this determination result to another device.

[0236] If a configuration is adopted that allows outputting (display, audio output, etc.) the evaluation value calculated by the evaluation value calculation unit 87, it is not necessary to provide a pass / fail judgment unit 88. In this case, the user can determine the pass / fail status of the follow-up imaging from the outputted evaluation value.

[0237] <Communications Department 9> The communication unit 9 performs data communication between the slit lamp microscope 1 and other devices (external devices). Specifically, the communication unit 9 transmits data to other devices and receives data transmitted from other devices.

[0238] For example, the communication unit 9 can transmit at least some of the images from the image set collected by scanning the anterior segment of the eye E under examination with a slit light to another device (first external device). The communication unit 9 can also transmit data generated from this image set by the data processing unit 8 to another device (second external device). The first external device and the second external device may be the same device or different devices. If the first external device and the second external device are the same device, this external device may be, for example, a radiograph interpretation terminal, a radiograph interpretation device, or a computer that performs processing different from the data processing unit 8. On the other hand, if the first external device and the second external device are different, for example, the first external device may be a radiograph interpretation terminal, a radiograph interpretation device, a computer that performs processing similar to the data processing unit 8, or a computer that performs processing different from the data processing unit 8, and the second external device may be a radiograph interpretation terminal, a radiograph interpretation device, or a computer that performs processing different from the data processing unit 8. If the first external device and the second external device are different, the control unit 7 may be configured to select the destination external device according to the type of data to be transmitted.

[0239] The communication unit 9 can transmit the processed image constructed by the processed image construction unit 85, the positional shift information generated by the positional shift information acquisition unit 86, the evaluation value calculated by the evaluation value calculation unit 87, and the judgment result obtained by the pass / fail judgment unit 88 to an external device.

[0240] Before follow-up imaging is performed, the communication unit 9 can receive anterior segment images previously acquired from the patient (subject) from an external device. The communication unit 9 can also receive imaging condition information from the external device at the time these anterior segment images were acquired. These external devices are typically any medical system, such as a medical image management system like a Picture Archiving and Communication System (PACS) or an electronic medical record system. The control unit 7 stores the anterior segment images and imaging condition information received from the external device in the storage unit 10. The anterior segment images can be used as reference images in follow-up imaging. The imaging condition information can be used to set conditions for follow-up imaging.

[0241] The data communication method performed by the communication unit 9 is arbitrary. For example, the communication unit 9 includes one or more of the following communication interfaces: an internet-compliant communication interface, a dedicated line-compliant communication interface, a LAN-compliant communication interface, a short-range communication-compliant communication interface, etc. Data communication may be wired or wireless.

[0242] Data transmitted and / or received by the communication unit 9 may be encrypted. When encryption is used, for example, the control unit 7 and / or data processing unit 8 include at least one of an encryption processing unit that encrypts the data transmitted by the communication unit 9 and a decryption processing unit that decrypts the data received by the communication unit 9.

[0243] <Storage section 10> The memory unit 10 stores various types of data. For example, the memory unit 10 stores a group of images collected by scanning the anterior segment of the eye E under examination with a slit lamp, an image set created from two or more image groups, analysis data generated by image analysis, processed images constructed by the processed image construction unit 85, positional displacement information generated by the positional displacement information acquisition unit 86, evaluation values ​​calculated by the evaluation value calculation unit 87, and judgment results obtained by the pass / fail judgment unit 88. The control unit 7 performs the process of storing data in the memory unit 10 and the process of reading data from the memory unit 10. The storage and reading processes are performed as needed during various operations of the slit lamp microscope 1.

[0244] The memory unit 10 stores follow-up imaging information containing information of a predetermined type. Follow-up imaging information is information related to imaging performed in the past (follow-up imaging and / or non-follow-up imaging). For example, the follow-up imaging information includes at least an image of the anterior segment of the eye E examined (first image, reference image) that was referenced in a previously performed follow-up imaging, and an image of the anterior segment of the eye E examined (second image, follow-up image) acquired in this follow-up imaging.

[0245] Furthermore, the follow-up imaging information may include reference scan position information indicating the scan position corresponding to the reference image (first scan position, reference scan position), and follow-up scan position information indicating the scan position corresponding to the follow-up image acquired in this follow-up imaging (second scan position, follow-up scan position). This scan position information may include, for example, control information for the movement mechanism 6 for scanning, and control information for the illumination system 2 and / or imaging system 3 for scanning (for example, control information for the slit forming section, control information for the image sensor 5). Note that when performing an anterior segment scan with slit light and acquiring a frontal image of the anterior segment in real time (described later), the scan position information may also include the coordinates of the scan position (scan trajectory) reflected in the frontal image.

[0246] The storage unit 10 includes any storage device, and includes at least one of a non-volatile storage device and a volatile storage device. The storage unit 10 includes, for example, at least one of a hard disk drive and a semiconductor memory.

[0247] <Other elements> In addition to the elements shown in Figures 1A and 1B, the slit lamp microscope 1 may be equipped with a display device and an operating device. Alternatively, the display device and operating device may be peripheral devices for the slit lamp microscope 1.

[0248] The display device displays various information under the control of the control unit 7. The display device may include a flat panel display such as a liquid crystal display (LCD).

[0249] The operating devices include devices for operating the slit lamp microscope 1 and devices for inputting information. Examples of operating devices include buttons, switches, levers, dials, handles, knobs, mice, keyboards, trackballs, and control panels.

[0250] A device that integrates a display device and an operating device, such as a touchscreen, may also be used.

[0251] The subject and assistant can operate the slit lamp microscope 1 using the display device and the control device.

[0252] <Alignment> The alignment of the slit lamp microscope 1 with respect to the eye E under examination will be described. Generally, alignment is the operation of positioning the device's optical system in a suitable location for imaging and measuring the eye E under examination. In this embodiment, the alignment is the operation of positioning the illumination system 2 and the imaging system 3 in a suitable location for acquiring multiple anterior segment images (a series of images, a moving image, a group of images, or a set of images, etc.) as shown in Figure 3.

[0253] There are various methods for aligning ophthalmic devices. Several alignment methods are exemplified below, but the methods applicable to this embodiment are not limited to these.

[0254] Stereo alignment is an alignment method applicable to this embodiment. Stereo alignment is applicable to ophthalmic devices capable of photographing the anterior segment from two or more different directions, and a specific method is disclosed in Japanese Patent Application Publication No. 2013-248376 by the present applicant, etc. Stereo alignment includes, for example, the following steps: acquiring two or more images by having two or more anterior segment cameras photograph the anterior segment from different directions; a processor analyzing these images to determine the three-dimensional position of the eye under examination; and a processor controlling the movement of the optical system based on the determined three-dimensional position. As a result, the optical system (illumination system 2 and imaging system 3 in this example) is positioned in a suitable position relative to the eye under examination. In a typical stereo alignment, the position of the pupil of the eye under examination (center or centroid of the pupil) is used as the reference.

[0255] In addition to this stereo alignment method, any known alignment method can be used, such as methods utilizing Purkinje images obtained by alignment light, methods utilizing optical levers, or methods utilizing alignment indices. In methods utilizing Purkinje images, optical levers, or alignment indices, the position of the corneal apex of the eye being examined is used as the reference point.

[0256] While conventional alignment methods, including the examples above, aim to align the axis of the eye under examination with the optical axis of the optical system, in this embodiment, it is possible to perform alignment so that the illumination system 2 and the imaging system 3 are positioned at a location corresponding to the scan start position.

[0257] As a first example of alignment in this embodiment, after performing alignment based on the pupil or corneal apex of the eye E to be examined by applying one of the alignment methods described above, the illumination system 2 and imaging system 3 can be moved (in the X direction) by a distance equivalent to a preset standard value of the corneal radius, or a distance equivalent to a preset standard value of the corneal radius plus a predetermined value. Alternatively, instead of using the standard value, a previously acquired measurement of the corneal radius of the eye E to be examined may be used.

[0258] As a second example, after performing alignment based on the pupil or corneal apex of the eye E to be examined by applying one of the alignment methods described above, the image of the anterior segment of the eye E to be examined is analyzed to measure the corneal radius, and the illumination system 2 and imaging system 3 can be moved (in the X direction) by a distance equivalent to this measurement, or a distance equivalent to this measurement plus a predetermined value. The image of the anterior segment analyzed in this example is, for example, an anterior segment image obtained by imaging system 3, or another image. The other image may be any image, such as an image obtained by an anterior segment camera or an image obtained by an anterior segment OCT.

[0259] As a third example, by analyzing the anterior segment image obtained by the anterior segment camera or imaging system 3 for stereo alignment to determine the first edge of the cornea, and applying stereo alignment, the illumination system 2 and imaging system 3 can be moved to a position corresponding to this first edge, or to a position shifted outward by a predetermined distance from this first edge.

[0260] Alternatively, one of the alignment methods described above may be applied to perform alignment based on the pupil or corneal apex of the eye E under examination, and the scan with slit light may be started from the position determined by this alignment. In this case as well, the scan sequence can be set to scan a three-dimensional region including the cornea C (for example, a three-dimensional region including the entire cornea C). For example, the scan sequence may be set to scan to the left from the position determined by alignment based on the pupil or corneal apex, and then scan to the right.

[0261] <Other Matters> The slit lamp microscope 1 may include a fixation system that outputs light (fixation light) for fixing the examined eye E. The fixation system typically includes at least one visible light source (fixation light source) or a display device that displays images such as a landscape chart or a fixation target. The fixation system is arranged coaxially or non-coaxially with, for example, the illumination system 2 or the imaging system 3. The fixation system may include an internal fixation system that presents a fixation target to the subject through the optical path of the device optical system, and / or an external fixation system that presents a fixation target to the subject from outside the optical path.

[0262] The types of images that can be acquired by the slit lamp microscope 1 are not limited to a plurality of anterior eye images as shown in FIG. 3. For example, the slit lamp microscope 1 includes a three-dimensional image based on a plurality of anterior eye images, a rendering image based on this three-dimensional image, a retroillumination image, a moving image representing the movement of a contact lens worn on the examined eye, an image representing the gap between the contact lens and the corneal surface by applying a fluorescent agent, and the like. The rendering image will be described in other aspects. The retroillumination image is an image obtained by the retroillumination method that depicts turbidity and foreign bodies in the eye using the retinal reflection of illumination light. Note that fundus imaging, corneal endothelial cell imaging, meibomian gland imaging, etc. may also be possible.

[0263] The slit lamp microscope 1 (data processing unit 8) may be capable of executing analysis processing. For example, the slit lamp microscope 1 may be capable of calculating any anterior eye parameter. The anterior eye parameter is a value representing the form of the anterior eye. Examples of anterior eye parameters include the anterior corneal curvature radius, the posterior corneal curvature radius, the anterior lens curvature radius, the posterior lens curvature radius, the corneal diameter (vertical diameter, horizontal diameter (White-to-White)), the corneal thickness (central thickness, peripheral thickness), the lens thickness, the anterior chamber depth, the anterior chamber volume, the pupil diameter, the pupil center (eccentricity), etc. Further, the anterior eye parameter may be shape distribution data, for example, various corneal shape maps such as an axial curvature map (axial curvature map), a tangential curvature map (tangential curvature map), an elevation map (elevation map), a refractive power map, a thickness map (pachymetry map), a wavefront aberration map, etc. Further, the anterior eye parameter may include a corner parameter. Examples of corner parameters include AOD (angle opening distance), ACA (anterior chamber angle), TISA (trabecular iris space area), ARA (angle recess area), AtA (angle-to-angle distance), etc. are known. The method for calculating the anterior eye parameter is well-known. Typically, the anterior eye parameter calculation includes processing for specifying a site and / or position to be measured (for example, segmentation, feature point detection, etc.) and measurement processing for obtaining the value of the anterior eye parameter based on the specified site and / or position (for example, distance measurement, ratio calculation, angle calculation, etc.).

[0264] <Other aspects of the optical system> In the slit lamp microscope of the embodiment described so far, both the reference image and the follow-up image are images (scanned images) obtained by scanning the anterior segment of the eye with slit light, and positional displacement information is generated from these two scanned images. However, the reference image and / or follow-up image do not have to be scanned images. For example, if the slit lamp microscope can acquire not only scanned images but also frontal images, positional displacement information can be generated from the frontal image. An example of such an optical system is shown in Figure 10. In addition to the elements shown in Figure 10, the slit lamp microscope of this embodiment includes, for example, the control unit 7, data processing unit 8, communication unit 9, storage unit 10, display device, operating device, etc. Unless otherwise specified, the matters relating to the slit lamp microscope 1 described above can be applied to this embodiment.

[0265] The illumination system 20 shown in Figure 10 is an example of illumination system 2 of the slit lamp microscope 1, and the left imaging system 30L and the right imaging system 30R are examples of imaging system 3. Hereinafter, illumination system 20 may be referred to as illumination system 2, and the left imaging system 30L and / or the right imaging system 30R may be referred to as imaging system 3. In some embodiments, only one of the left imaging system 30L and the right imaging system 30R may be provided and configured as an example of imaging system 3. Reference numeral 20a indicates the optical axis (illumination optical axis) of illumination system 20, reference numeral 30La indicates the optical axis (left imaging optical axis) of the left imaging system 30L, and reference numeral 30Ra indicates the optical axis (right imaging optical axis) of the right imaging system 30R. The left imaging optical axis 30La and the right imaging optical axis 30Ra are arranged in different directions from each other. The angle between the illumination optical axis 20a and the left imaging optical axis 30La is denoted by θL, and the angle between the illumination optical axis 20a and the right imaging optical axis 30Ra is denoted by θR. The angles θL and θR may be equal or different. The illumination optical axis 20a, the left imaging optical axis 30La, and the right imaging optical axis 30Ra intersect at a single point. As in Figure 1A, the Z-coordinate of this intersection point is denoted by Z0.

[0266] The moving mechanism 6 is capable of moving the lighting system 20, the left imaging system 30L, and the right imaging system 30R in the direction indicated by the arrow 49 (X direction). Typically, the lighting system 20, the left imaging system 30L, and the right imaging system 30R are mounted on a stage that is movable at least in the X direction, and the moving mechanism 6 moves this movable stage according to a control signal from the control unit 7.

[0267] The illumination system 20 irradiates the anterior segment of the eye E under examination with slit light. Similar to the illumination system of a conventional slit lamp microscope, the illumination system 20 includes, in order from the side furthest from the eye E under examination, an illumination light source 21, a positive lens 22, a slit forming section 23, and objective lens groups 24 and 25.

[0268] Illumination light (typically visible light) output from the illumination light source 21 is refracted by the positive lens 22 and projected onto the slit forming section 23. A portion of the projected illumination light passes through the slit formed by the slit forming section 23 to become slit light. The generated slit light is refracted by the objective lens groups 24 and 25, reflected by the beam splitter 47, and irradiated onto the anterior segment of the eye E under examination.

[0269] The left imaging system 30L includes a reflector 31L, an imaging lens 32L, and an image sensor 33L. The reflector 31L and the imaging lens 32L guide the light from the anterior segment (light traveling in the direction of the left imaging system 30L), which is illuminated by slit light from the illumination system 20, to the image sensor 33L.

[0270] Light traveling from the anterior segment toward the left imaging system 30L is light from the anterior segment illuminated by the slit light, and is traveling away from the illumination optical axis 20a. The reflector 31L reflects this light toward the illumination optical axis 20a. The imaging lens 32L refracts the light reflected by the reflector 31L and forms an image on the imaging surface 34L of the image sensor 33L. The image sensor 33L receives this light on its imaging surface 34L.

[0271] Similar to the slit lamp microscope 1 described above, the left imaging system 30L repeatedly takes images in parallel with the movement of the illumination system 20, the left imaging system 30L, and the right imaging system 30R by the movement mechanism 6. This results in obtaining multiple anterior segment images (image groups).

[0272] Furthermore, similar to the slit lamp microscope 1 described above, the object surface along the illumination optical axis 20a, the optical system including the reflector 31L and imaging lens 32L, and the imaging surface 34L satisfy the Scheinproof condition. More specifically, considering the deflection of the optical path of the imaging system 30L by the reflector 31L, the YZ plane (including the object surface) passing through the illumination optical axis 20a, the principal surface of the imaging lens 32L, and the imaging surface 34L intersect on the same straight line. As a result, the left imaging system 30L can focus on and image all positions within the object surface (for example, from the anterior surface of the cornea to the posterior surface of the lens).

[0273] The right imaging system 30R includes a reflector 31R, an imaging lens 32R, and an image sensor 33R. Similar to the left imaging system 30L, the right imaging system 30R guides light from the anterior segment, which is illuminated by slit light from the illumination system 20, to the imaging surface 34R of the image sensor 33R via the reflector 31R and the imaging lens 32R. Furthermore, similar to the left imaging system 30L, the right imaging system 30R acquires multiple anterior segment images (image groups) by repeatedly taking images in parallel with the movement of the illumination system 20, the left imaging system 30L, and the right imaging system 30R by the movement mechanism 6. Similar to the left imaging system 30L, the object surface along the illumination optical axis 20a, the optical system including the reflector 31R and the imaging lens 32R, and the imaging surface 34R satisfy the Scheinproof condition.

[0274] The control unit 7 can synchronize repeated imaging by the left imaging system 30L and repeated imaging by the right imaging system 30R. This allows for a correspondence between multiple anterior segment images obtained by the left imaging system 30L and multiple anterior segment images obtained by the right imaging system 30R. This correspondence is a temporal correspondence, and more specifically, it pairs images that were acquired substantially simultaneously.

[0275] Alternatively, the control unit 7 or the data processing unit 8 can perform a process to determine the correspondence between multiple anterior segment images obtained by the left imaging system 30L and multiple anterior segment images obtained by the right imaging system 30R. For example, the control unit 7 or the data processing unit 8 can pair anterior segment images sequentially input from the left imaging system 30L with anterior segment images sequentially input from the right imaging system 30R based on their input timing.

[0276] This embodiment further includes a video recording unit 40. The video recording unit 40 records video of the anterior segment of the eye E under examination from a fixed position in parallel with the recording by the left recording system 30L and the right recording system 30R. "Recording video from a fixed position" means that the video recording unit 40 does not move when the illumination system 20, the left recording system 30L, and the right recording system 30R are moved for anterior segment scanning. The video recording unit 40 is also capable of capturing still images.

[0277] In this embodiment, the video recording unit 40 is arranged coaxially with the illumination system 20, but its arrangement is not limited thereto. For example, the video recording unit can be arranged non-coaxially with the illumination system 20. Furthermore, an optical system may be provided that illuminates the anterior eye portion with illumination light in a frequency band to which the video recording unit 40 is sensitive.

[0278] Light that has passed through the beam splitter 47 is reflected by the reflector 48 and incident on the video recording unit 40. The light incident on the video recording unit 40 is refracted by the objective lens 41 and then imaged onto the imaging surface of the image sensor 43 by the imaging lens 42. The image sensor 43 is an area sensor. The image sensor 43 is sensitive to either or both of the visible light and infrared light bands, for example.

[0279] If a video recording unit 40 is provided, it is possible to monitor and track the movement of the eye E under examination. It is also possible to perform alignment using the video recording unit 40.

[0280] Depending on the output wavelength of the lighting system 20 and the detection wavelength of the video recording unit 40, the beam splitter 47 is, for example, a dichroic mirror or a half mirror.

[0281] Some features, functions, and effects of the slit lamp microscope of this aspect will be described.

[0282] As an example of the imaging system 3 of the slit lamp microscope 1 described above, this aspect provides a left imaging system 30L and a right imaging system 30R. The left imaging system 30L includes a reflector 31L and an imaging lens 32L (first optical system) that guide light from the anterior eye part irradiated with slit light, and an imaging element 33L (first imaging element) that receives the guided light on an imaging surface 34L (first imaging surface). Similarly, the right imaging system 30R includes a reflector 31R and an imaging lens 32R (second optical system) that guide light from the anterior eye part irradiated with slit light, and an imaging element 33R (second imaging element) that receives the guided light on an imaging surface 34R (second imaging surface).

[0283] The optical axis of the left imaging system 30L (left imaging optical axis 30La) and the optical axis of the right imaging system 30R (right imaging optical axis 30Ra) are arranged in different directions from each other. Further, the object plane along the optical axis of the illumination system 20 (illumination optical axis 20a), the reflector 31L and the imaging lens 32L, and the imaging surface 34L satisfy the Schlein-proof condition. Similarly, the object plane, the reflector 31L and the imaging lens 32L, and the imaging surface 34L satisfy the Schlein-proof condition.

[0284] The left imaging system 30L acquires a first image group by repeatedly imaging in parallel with the movement of the illumination system 20, the left imaging system 30L, and the right imaging system 30R by the movement mechanism 6. Similarly, the right imaging system 30R acquires a second image group by repeatedly imaging in parallel with the movement of the illumination system 20, the left imaging system 30L, and the right imaging system 30R by the movement mechanism 6.

[0285] The control unit 7 can control the illumination system 20, the left imaging system 30L, the right imaging system 30R, and the movement mechanism 6 so that, for example, the left imaging system 30L and the right imaging system 30R apply a single scan to the anterior segment of the eye E being examined in parallel. The data processing unit 8 can select a series of images corresponding to the scan range from the pair of left and right image groups acquired in this scan to create a single image group. The image group evaluation unit 81 can evaluate the quality of the created image group. If the quality of this image group is evaluated as good, the control unit 7 can control the communication unit 9 to transmit this image group to a predetermined external device.

[0286] If the quality of this image set is evaluated as unsatisfactory, the control unit 7 can control the illumination system 20, the left imaging system 30L, the right imaging system 30R, and the movement mechanism 6 to apply a new scan (one or more scans) to the anterior segment of the eye E under examination. This additional scan causes the left imaging system 30L to collect one or more first image sets corresponding to one or more scans, and the right imaging system 30R to collect one or more second image sets corresponding to one or more scans. The image set creation unit 82 can, for example, create a first image set by selecting a first series of images corresponding to the scan range from two or more first image sets obtained from the first scan and the additional scan, and create a second image set by selecting a second series of images corresponding to the scan range from two or more second image sets obtained from the first scan and the additional scan. Alternatively, the image set creation unit 82 may be configured to create an image set by selecting a series of images corresponding to the scan range from two or more first image sets and two or more second image sets obtained from the first scan and the additional scan. Any aspect of the slit lamp microscope 1 described above (configuration, control, processing, operation, function, etc.) can be combined with this embodiment.

[0287] According to this embodiment, the anterior segment illuminated by the slit light can be captured as a video from two different directions. Even if an image acquired by one imaging system contains artifacts, an image acquired substantially simultaneously with that image by the other imaging system may not contain artifacts. Furthermore, even if both images acquired substantially simultaneously by both imaging systems contain artifacts, and the artifact in one image overlaps the region of interest (e.g., the slit light illumination region), the artifact in the other image may not overlap the region of interest. Therefore, the likelihood of acquiring a suitable image increases. Consequently, the probability of obtaining a group and / or set of images of good quality can be further improved.

[0288] Furthermore, imaging system 3 may include, in addition to the left imaging system 30L (first imaging system) and the right imaging system 30R (second imaging system), a third imaging system, ..., and a Kth imaging system (K being an integer greater than or equal to 3) with a similar configuration. This complicates the structure of the optical system, but makes it possible to further improve the probability of obtaining a good quality image group and / or image set. Those who intend to implement this embodiment can design a slit lamp microscope taking into account these trade-offs (complexity of the optical system and the probability of obtaining a high-quality image set).

[0289] The left imaging system 30L in this embodiment includes a reflector 31L and an imaging lens 32L. The reflector 31L reflects light from the anterior eye portion illuminated by the slit light, which is traveling away from the illumination optical axis 20a, in a direction approaching the illumination optical axis 20a. Furthermore, the imaging lens 32L forms an image of the light reflected by the reflector 31L on the imaging surface 34L. Here, the imaging lens 32L includes one or more lenses.

[0290] Similarly, the right imaging system 30R includes a reflector 31R and an imaging lens 32R. The reflector 31R reflects light from the anterior eye segment illuminated by the slit light, which is traveling away from the illumination optical axis 20a, in a direction toward the illumination optical axis 20a. Furthermore, the imaging lens 32R images the light reflected by the reflector 31R onto the imaging surface 34R. Here, the imaging lens 32R includes one or more lenses.

[0291] This configuration makes it possible to miniaturize the device. Specifically, the image acquired by the image sensor 33L (33R) is output through a cable extending from the opposite side of the imaging surface 34L (34R). With this configuration, the cable can be routed from the back of the image sensor 33L (33R), which is located relatively close to the illumination optical axis 20a, in the direction opposite to the eye under examination E. Therefore, the cable can be routed efficiently, making it possible to miniaturize the device.

[0292] Furthermore, this configuration allows for larger angles θL and θR, thereby increasing the likelihood that, if an image acquired by one imaging system contains artifacts, an image acquired substantially simultaneously with that image by the other imaging system will not contain artifacts. Additionally, if both images acquired substantially simultaneously by both imaging systems contain artifacts, and the artifact in one image overlaps the area of ​​interest (e.g., the slit light irradiation area), the likelihood that the artifact in the other image overlaps the area of ​​interest can be reduced.

[0293] This embodiment includes a video recording unit 40. The left recording system 30L and the right recording system 30R repeatedly record the anterior eye region in parallel with the movement of the illumination system 20, the left recording system 30L, and the right recording system 30R by the movement mechanism 6. In parallel with this repeated recording, the video recording unit 40 records video of the anterior eye region from a fixed position.

[0294] With this configuration, by recording video from a fixed position (e.g., the front) in parallel with scanning the anterior segment of the eye using a slit light, it is possible to understand the state of the eye being examined E during the scan and to perform control according to the state of the eye being examined E.

[0295] For example, if the slit lamp microscope of this embodiment includes the image group evaluation unit 81 (image set evaluation unit 83) of the slit lamp microscope 1 described above, the slit lamp microscope of this embodiment can evaluate whether the image group (image set) acquired by the left imaging system 30L and / or the right imaging system 30R is of sufficient quality for interpretation. The image group evaluation unit 81 (image set evaluation unit 83), in combination with the video recording unit 40, can perform the following operations. It is also possible to perform similar operations using two or more anterior segment cameras for stereo alignment or similar imaging means.

[0296] The video recording unit 40 records video of the anterior segment of the eye E from a fixed position in parallel with applying a scan to the anterior segment of the eye E under examination. This video recording is performed, for example, under the control of the control unit 7. In other words, the control unit 7 can coordinately control the illumination system 2 (illumination system 20), the imaging system 3 (left imaging system 30L and / or right imaging system 30R), the movement mechanism 6, and the video recording unit 40 for scanning the anterior segment of the eye E under examination.

[0297] In this linked control, the control unit 7 can synchronize the shooting rate of the shooting system 3 with the shooting rate of the video shooting unit 40. For example, the shooting rate of the shooting system 3 and the shooting rate of the video shooting unit 40 are set to be equal, and the shooting timing of the shooting system 3 and the shooting timing of the video shooting unit 40 are matched. This makes it possible to temporally associate the group of frames acquired by the shooting system 3 and the group of frames acquired by the video shooting unit 40 during scanning.

[0298] Furthermore, even if one or both of the shooting rate and / or shooting timing are different, it is possible to temporally associate the frame group acquired by the shooting system 3 and the frame group acquired by the video shooting unit 40 in the scan by, for example, allowing a time difference within a predetermined range.

[0299] A pair of time-correlated frames (a pair of frames obtained by the shooting system 3 and frames obtained by the video shooting unit 40) can be considered to have been acquired substantially simultaneously. Therefore, when considering a pair of time-correlated frames, positional shifts due to eye movements can be ignored.

[0300] Under these premises, the image group evaluation unit 81 (image set evaluation unit 83) may be configured to evaluate the quality of a series of images included in an image group (image set) based on the moving images (frame group) acquired by the video shooting unit 40.

[0301] In this case, the image group evaluation unit 81 (image set evaluation unit 83) may be configured to evaluate the quality of the image group (image set) based on the correspondence between a series of images included in the image group (image set) and a series of frames included in the moving image acquired by the moving image unit 40. That is, the image group evaluation unit 81 (image set evaluation unit 83) may be configured to evaluate the quality of the image group (image set) based on the temporal correspondence between a group of frames obtained by the shooting system 3 and a group of frames acquired by the moving image unit 40. Furthermore, the image group evaluation unit 81 (image set evaluation unit 83) may be configured to evaluate the quality of the image group (image set) based on landmarks in the series of frames and the said correspondence.

[0302] Let's explain one specific example. In this example, we assume that frame Fn and frame Dn are associated with each other (n=1, 2, ..., N) between the frame group F1~FN (mentioned above) obtained by the shooting system 3 and the frame group D1~DN acquired in parallel by the video shooting unit 40.

[0303] The image group evaluation unit 81 (image set evaluation unit 83) identifies landmarks in each of the frame groups D1 to DN. The landmarks may be, for example, image regions corresponding to the iris (iris region).

[0304] Next, the image group evaluation unit 81 (image set evaluation unit 83) determines the arrangement order (spatial arrangement order) of the frame groups D1 to DN based on the positions (e.g., changes in spatial position) of the N landmarks identified in each of the frame groups D1 to DN.

[0305] As mentioned above, the frame group D1-DN is arranged in this order temporally (temporal arrangement order). If the temporal arrangement order differs from the spatial arrangement order, it is possible that the order has been rearranged or that positional shifts have occurred due to the influence of eye movements. Also, it is possible that frames have been skipped due to the influence of blinking.

[0306] If such a defect is detected, that is, if the temporal arrangement order and the spatial arrangement order are different, the image group evaluation unit 81 (image set evaluation unit 83) determines that the quality of this image group (image set) is not good.

[0307] With this configuration, it is possible to acquire a video image that broadly depicts the anterior segment (at least in the X and Y directions) in parallel with scanning the anterior segment with a slit light, and to use this video image to detect defects in a series of images included in the image group (image set).

[0308] Each image in the image set is an image that extends in the depth direction (Z direction). Therefore, in order to recognize the arrangement and sequence of the images in the X and Y directions, which are orthogonal to the Z direction, from the images themselves, it is necessary to use image processing (image analysis) similar to that used in the slit lamp microscope 1 described above.

[0309] This example demonstrates how to evaluate the quality of a series of images by using video footage acquired separately in parallel with the scan, without applying image analysis to the series of images themselves. The evaluation criteria in this example may be the sequence order of the images, any missing images, or any positional misalignment, similar to the slit-lamp microscope 1 described above. Furthermore, it is possible to combine the evaluation process in this example with any of the evaluation processes described for the slit-lamp microscope 1.

[0310] Other applications of the video recording unit 40 will be described. This example aims to prevent displacement of the eye E between scans by adjusting the start timing of the two or more additional scans, for example, when the quality of the image set collected in the first scan is evaluated as not being good and two or more scans are performed.

[0311] In this example as well, the video recording unit 40 captures the anterior segment of the eye E under examination from a fixed position. The control unit 7 is configured to cause the scanning unit to start a second scan when the video recording unit 40 acquires an image that is substantially the same as the reference image acquired by the video recording unit 40 in response to the start of the first scan of two or more additional scans.

[0312] Let's explain in more detail. The first scan is one of two or more scans applied to the anterior segment to create an image set. In this example, the first scan performed among the two or more scans is considered the first scan.

[0313] First, the control unit 7 stores an image (reference image) acquired by the video recording unit 40 at the start timing of the first scan. The reference image may be, for example, a frame acquired at the time closest to the start of the first scan from a group of video frames whose capture began before the start of the first scan. Alternatively, the video recording unit 40 may acquire a reference image by capturing a still image of the anterior eye at the start of the first scan (immediately before, simultaneously with, or immediately after) under the control of the control unit 7.

[0314] Any scan performed after the first scan (second scan) is started while the video recording unit 40 is capturing anterior segment video. The control unit 7 (or data processing unit 8) compares the frames acquired sequentially by the video recording unit 40 with a reference image. This comparison may include any image processing, such as segmentation to identify landmarks, image matching, or image correlation.

[0315] When it is determined that a frame nearly identical to the reference image has been obtained, the control unit 7 controls the illumination system 2, the imaging system 3, and the movement mechanism 6 to start the second scan.

[0316] In this example, the slit lamp microscope repeatedly performs the above-described series of processes until the number of scans applied to the anterior segment reaches the number of scan iterations required to create an image set. This reduces the displacement of the eye E between scans and prevents degradation of image set quality due to eye movements. It also makes image selection more efficient and easier.

[0317] Images acquired by the video recording unit 40 can be used to determine the positional displacement that occurred during follow-up imaging. In some exemplary embodiments, the video recording unit 40 is positioned to capture the anterior segment of the eye E from the front. That is, the video recording unit 40 may be capable of acquiring a frontal image of the anterior segment.

[0318] The configuration for acquiring a frontal image is not limited to the video recording unit 40. For example, a frontal image can be constructed by combining two or more anterior segment images acquired by two or more anterior segment cameras, as disclosed in Japanese Patent Application Publication No. 2013-248376, etc. The frontal image constructed from two or more anterior segment images is a virtual viewpoint image generated using image processing techniques such as viewpoint transformation. Even when using such a virtual frontal image, it is possible to perform the same processing as when acquiring a frontal image with the video recording unit 40.

[0319] In this example, anterior segment scanning (acquisition of image sets) using slit light is performed using the illumination system 20, left imaging system 30L, right imaging system 30R, and movement mechanism 6, along with anterior segment imaging (acquisition of frontal images) using the video recording unit 40.

[0320] The timing of acquiring the frontal image is arbitrary. For example, anterior segment imaging using the video recording unit 40 may be performed in parallel with the anterior segment scan using slit light. In other words, at least a portion of the execution period for anterior segment imaging may overlap with at least a portion of the execution period for anterior segment scanning. Alternatively, anterior segment imaging may be performed immediately before or after the anterior segment scan. In this case, the time difference between the anterior segment scan and anterior segment imaging is set to be small enough that, for example, the effects of eye movements can be ignored.

[0321] Before follow-up imaging is performed, the slit lamp microscope of this embodiment acquires a frontal image (first frontal image, reference frontal image) of the anterior segment of the eye E under examination that was acquired during a previous imaging session. The acquired first frontal image is stored in the memory unit 10. At this time, imaging conditions can also be acquired and saved. Follow-up imaging is performed using the first frontal image as the reference image. In this follow-up imaging, the video recording unit 40 acquires a frontal image (second frontal image, follow-up frontal image) of the anterior segment of the eye E under examination. Follow-up imaging is performed, for example, using past imaging conditions (scan conditions, etc.).

[0322] In this example, the positional displacement information acquisition unit 86 acquires positional displacement information between the two frontal images by analyzing the first frontal image referenced in the follow-up imaging and the second frontal image acquired in the same follow-up imaging.

[0323] If the shooting conditions for the previous scan and the current scan (follow-up scan) are the same, the positional shift between the first and second frontal images can be considered to correspond to a deviation in the scan position. By comparing these frontal images, information indicating the positional error in the follow-up scan can be obtained.

[0324] Past and current shooting conditions do not need to be the same. For example, if past and current shooting conditions are comparable, positional shift information can be generated by considering the differences between the two shooting conditions. For instance, if the current scan start position differs from that of past scans, the image group positioning can be performed by considering the shift amount of the scan start position. Also, if the current scan interval (frame rate) differs from that of past scans, the image group arrangement (such as stack data construction) can be determined by considering the difference in scan intervals.

[0325] In this embodiment, the evaluation value calculation unit 87 can calculate an evaluation value of the scan position error in follow-up imaging based on the positional misalignment information acquired by the positional misalignment information acquisition unit 86. Furthermore, the pass / fail determination unit 88 can determine whether the follow-up imaging was performed successfully based on the evaluation value calculated by the evaluation value calculation unit 87.

[0326] The slit lamp microscope of this embodiment can display various information using the front image. For example, the control unit 7 can display a reference front image and / or a follow-up front image on a display device, and, based on the positional misalignment information acquired by the positional misalignment information acquisition unit 86, can overlay a reference scan position image indicating the reference scan position and a follow-up scan position image indicating the follow-up scan position onto the front image.

[0327] In such display control, relative position information equivalent to or similar to positional misalignment information can be referenced instead of, or in addition to, positional misalignment information.

[0328] When displaying either the reference frontal image or the follow-up frontal image, the reference scan position image and the follow-up scan position image can be overlaid on the one frontal image. When displaying both the reference frontal image and the follow-up frontal image, the reference scan position image and the follow-up scan position image may be overlaid on each of the two frontal images, or they may be overlaid on only one of the frontal images.

[0329] The control unit 7 can display the reference scan position image and the follow-up scan position image in different ways. For example, the two scan position images can be displayed in different colors, with different line thicknesses, or with different densities. Identification information (string information, image information, etc.) can also be attached to each scan position image. The two scan position images may always be displayed in different ways, or they may be switched to different display ways in response to a predetermined trigger.

[0330] Figure 11 shows an example of the information displayed by the above display control. Figure 11 shows a frontal image 100 of the anterior segment (for example, a follow-up frontal image or a reference frontal image) overlaid with a reference scan position image 101 (shown by a dotted line) and a follow-up scan position image 102 (shown by a dashed line). The follow-up scan position image 102 shows the range to which the scan was applied in the follow-up imaging. The reference scan position image 101 shows the scan range if this follow-up imaging had been performed without error, that is, the range to which the scan was applied in past imaging and which was the target range for the current follow-up imaging. By displaying information in this way, it becomes possible to visually understand the error (accuracy) of the position of the follow-up imaging.

[0331] Such information is displayed, for example, as follows. First, the memory unit 10 stores first position information indicating the position in the reference frontal image of the images (image group) acquired by the anterior segment scan together with the reference frontal image, and second position information indicating the position in the follow-up frontal image of the images (image group) acquired by the anterior segment scan together with the follow-up frontal image. The first position information is information based on the first anterior segment scan and first anterior segment radiography performed in the past, and indicates the application position (range of application) of the first anterior segment scan in the frontal image (reference frontal image) acquired by the first anterior segment radiography. The second position information is information based on the follow-up radiography (second anterior segment scan) and second anterior segment radiography performed with reference to the reference frontal image, and indicates the application position (range of application) of the second anterior segment scan in the frontal image (follow-up frontal image) acquired by the second anterior segment radiography. The control unit 7 displays at least one of the first front image and the second front image on a display device, and, based on the positional misalignment information acquired by the positional misalignment information acquisition unit 86, overlays and displays the first position image corresponding to the first position information and the second position image corresponding to the second position information on the front image. For example, when both the first position image and the second position image are overlaid and displayed on the follow-up front image, the display position of the second position image on the follow-up front image is determined by the second position information, but the display position of the first position image on the follow-up front image is determined by correcting the position indicated by the first position information with the positional misalignment information.

[0332] The display controls that can be executed in this embodiment are not limited to those described above. For example, the control unit 7 can display any information representing the error in the scan position during follow-up imaging. As a specific example, the control unit 7 can display the evaluation value calculated by the evaluation value calculation unit 87 on the display device. The evaluation value may be displayed alone or together with the front image or the image showing the scan position. In addition, the judgment result obtained by the pass / fail judgment unit 88 can be displayed instead of, or together with, the evaluation value.

[0333] <Operation> Several examples of the operation of a slit lamp microscope according to an exemplary embodiment will be described below. The operation of slit lamp microscope 1 will be described as an example, but the operation of the slit lamp microscope in the embodiment shown in Figure 10 or other embodiments of slit lamp microscopes can be performed in the same manner.

[0334] Although not shown in the diagram, at any stage, the user (subject, examiner, assistant, etc.) can input subject information into the slit lamp microscope 1. The input subject information is stored in the storage unit 10 by the control unit 7. The subject information typically includes the subject's identification information (subject ID).

[0335] Furthermore, background information can be entered. Background information is any information about the subject, and examples include the subject's medical history, information filled out by the subject on a designated sheet, and information recorded in the subject's electronic medical record. Typically, background information includes gender, age, height, weight, disease name, candidate disease names, test results (visual acuity, refractive power, intraocular pressure, etc.), history of wearing corrective devices (glasses, contact lenses, etc.) and prescription, examination history, and treatment history. These are examples, and background information is not limited to these.

[0336] In addition, as preparation for imaging, the table on which the slit lamp microscope 1 is set up, the chair on which the subject sits, and the chin rest of the slit lamp microscope 1 are adjusted (all of which are not shown in the illustration). For example, the height of the table, chair, and chin rest are adjusted. The chin rest is equipped with a chin rest and a forehead rest to stably position the subject's face.

[0337] Once preparation is complete, the subject sits in the chair, rests their chin on the chin rest, and places their forehead against the forehead rest. Before or after these actions, the user performs an instruction operation to start imaging of the eye under test. This operation may be, for example, pressing an imaging start trigger button (not shown) or inputting an instruction voice. Alternatively, the control unit 7 may detect the completion of the preparation phase and automatically proceed to the imaging phase. A fixation target (not shown) may also be presented to the subject (the eye under test E or its sister eye).

[0338] <Example of operation 1> The first example of operation performed after the above preparations will be explained with reference to Figures 12 and 14. Figure 12 shows an example of operation for obtaining a set of good quality images by follow-up imaging. Figure 14 shows an example of operation for evaluating the quality (error) of follow-up imaging.

[0339] In this example, the operation example shown in Figure 12 is followed by the operation example shown in Figure 14. In other words, in this example, the quality of the follow-up imaging is evaluated after a set of good quality images have been acquired. Although not illustrated and described in detail, several other exemplary embodiments, conversely to this example, can evaluate the quality of the image set after evaluating the quality of the follow-up imaging. Furthermore, several other exemplary embodiments can perform the evaluation of the quality of the follow-up imaging and the evaluation of the quality of the image set in parallel. Those skilled in the art will be able to understand these other exemplary embodiments from the first operation example described below.

[0340] (S1: Alignment) In response to the start of imaging, the slit lamp microscope 1 first aligns the illumination system 2 and the imaging system 3 with respect to the eye E under examination. Unlike general alignment to align the optical axis of the optical system with the corneal apex or pupil center of the eye E under examination, the alignment in step S1 is performed to position the illumination system 2 and the imaging system 3 at the starting position of the scan performed in step S2. The manner of alignment in step S1 may be arbitrary and may be any of the examples described above. Before, during, and / or after the alignment is completed, operations such as adjusting the image sensor 5 and adjusting the focus may be performed.

[0341] (S2: Follow-up imaging (anterior segment scan)) The slit lamp microscope 1 scans the anterior segment of the eye E under examination by combining illumination by the illumination system 2, video recording by the imaging system 3, and movement of the illumination system 2 and imaging system 3 by the movement mechanism 6. This scan is applied to the three-dimensional region of the anterior segment. A single scan (scan from the start position to the end position of the scan) yields, for example, a group of images (multiple anterior segment images) F1 to FN shown in Figure 3.

[0342] The data processing unit 8 may perform predetermined processing on the image obtained by scanning. For example, it is possible to apply any signal processing or image processing, such as noise reduction, contrast adjustment, brightness adjustment, and color correction.

[0343] (S3: Evaluate the quality of the image set) The image group evaluation unit 81 evaluates the quality of the image group collected in the scan of step S2.

[0344] (S4: Is the quality good?) If the image quality is determined to be good in step S3 (S4: Yes), this operation proceeds to the evaluation of the quality (error) of the follow-up imaging (S5). On the other hand, if the image quality is determined to be poor in step S3 (S4: No), this operation proceeds to step S6.

[0345] (S5: Proceed to error evaluation) If the quality of the image group is determined to be good in step S3 (S4: Yes), the control unit 7 performs control to execute a process to evaluate the quality (error) of the follow-up imaging from which this image group was obtained.

[0346] For example, the control unit 7 sends information for error evaluation to the data processing unit 8 (for example, the positional misalignment information acquisition unit 86). The information for error evaluation includes, for example, the group of images (follow-up images) that were determined to be of good quality in step S4, and past images (reference images) that were referenced in the follow-up shooting. The information for error evaluation may further include the shooting conditions applied in the past shooting, the shooting conditions applied in the follow-up shooting, etc. The destination of the information for error evaluation is not limited to the positional misalignment information acquisition unit 86. For example, when generating positional misalignment information using a processed image, the control unit 7 can send the information for error evaluation to the processed image construction unit 85.

[0347] After step S5, this operation proceeds to step S31 in Figure 14.

[0348] (S6: Prompt reshoot) If the image quality is determined to be poor in step S3 (S4: No), the control unit 7 performs control to collect a new image set from the anterior segment of the eye E under examination. In this example, the control unit 7 performs control to display and / or output audio information to prompt the user to retake the images. The user performs an instruction operation to start retaking the images or an instruction operation to not retake the images.

[0349] When the user performs an instruction operation to start reshooting, the control unit 7 performs control to repeat the operation from step S1 (or step S2). Reshooting is repeated, for example, up to a predetermined number of times.

[0350] On the other hand, if the user performs an instruction operation to not perform reshoots, the control unit 7 may, for example, perform control to provide the group of images determined to be of poor quality to the error evaluation of follow-up shooting. Alternatively, the control unit 7 may perform control to delete, save, or record the group of images determined to be of poor quality.

[0351] (S31: Generate positional shift information) After step S5, the operation proceeds to step S31 in Figure 14. In step S31, the positional displacement information acquisition unit 86 analyzes the group of images (follow-up images) that were determined to be of good quality in step S4 and the past images (reference images) referenced in the follow-up shooting, and generates positional displacement information between the follow-up images and the reference images.

[0352] (S32: Calculate evaluation value) Next, the evaluation value calculation unit 87 calculates an evaluation value of the scan position error in follow-up imaging based on the positional displacement information acquired in step S31.

[0353] (S33: Determining the quality of follow-up imaging) Next, the quality determination unit 88 determines whether the follow-up imaging was performed well based on the evaluation value calculated in step S32.

[0354] (S34: Is it good?) If it is determined in step S33 that the follow-up imaging was performed successfully (S34: Yes), this operation proceeds to step S35.

[0355] On the other hand, if it is determined in step S33 that the follow-up imaging was not performed successfully (S34: No), this operation returns to step S6 in Figure 12. The processing in this case is the same as the processing related to step S6 described above.

[0356] (S35: Output image) If it is determined in step S33 that the follow-up imaging was performed successfully (S34: Yes), the control unit 7 performs control to output this image set. For example, the control unit 7 can transmit this image set to another device by controlling the communication unit 9. The output image set is a set of good quality images obtained from follow-up imaging with small positional errors compared to previous imaging. This image set is suitable not only for image interpretation but also for follow-up (comparison with previous images).

[0357] Examples of devices to which image sets are transmitted include information processing devices and storage devices. Information processing devices include, for example, servers on wide-area networks, servers on LANs, and computer terminals. Storage devices include storage devices located on wide-area networks and storage devices located on LANs.

[0358] The image set output in step S35 may include the aforementioned background information, positional displacement information, evaluation values, and pass / fail judgment results. Alternatively, this information may be supplementary information to the image set. In general, the data structure of the information output in step S35 is arbitrary.

[0359] Furthermore, the image set transmitted in step S35 typically includes a series of images of the anterior segment of the subject's right eye and a series of images of the anterior segment of the left eye. The series of images of the right eye and the series of images of the left eye are obtained by applying the operation of this example to the right eye and the left eye, respectively. The aforementioned subject eye information is attached to the series of images of the right eye and the series of images of the left eye, respectively, which allows for the distinction between the series of images of the right eye and the series of images of the left eye.

[0360] The subject's identification information is transmitted along with the image set. This identification information may be the subject ID entered into the slit lamp microscope 1, or it may be identification information generated based on the subject ID. For example, the subject ID (internal identification information) used for personal identification within the facility where the slit lamp microscope 1 is installed can be converted into external identification information used outside the facility. This improves information security regarding personal information such as image sets and background information.

[0361] The image data transmitted from the slit lamp microscope 1 in step S35 (or S6) is sent directly or indirectly to an information processing device. A typical example of this information processing device is the aforementioned image interpretation terminal used by a physician (or optometrist).

[0362] A physician can use a radiographing terminal to interpret a series of images included in a group of images (for example, the series of images F1 to FN shown in Figure 3). It is also possible to construct a three-dimensional image based on the series of images, display a rendered version of this three-dimensional image, and display information transmitted along with the image group. Furthermore, the radiographing terminal or other information processing device can analyze any of the images in the series, analyze the three-dimensional image, analyze the rendered image, and analyze the information transmitted along with the image group.

[0363] A physician can use a radiograph interpretation terminal to create a report (radiography report) containing information obtained from the radiograph interpretation. The radiograph interpretation report is provided, for example, to the facility where the slit lamp microscope 1 is installed, to a medical institution designated by the patient, to an information processing device used by a physician designated by the patient, or to an address registered by the patient (email address, address, etc.). Alternatively, the radiograph interpretation report may be sent to a designated database system for storage and management.

[0364] Another example of an information processing device to which the image group transmitted from the slit lamp microscope 1 in step S35 (or S6) is received is a picture interpretation device. The picture interpretation device includes a picture interpretation processor. The picture interpretation processor operates, for example, according to a picture interpretation program, and analyzes a series of images included in the image group to obtain findings. Furthermore, the picture interpretation processor creates a report based on the obtained findings.

[0365] The image interpretation processor may include an artificial intelligence engine that performs image interpretation using a trained model (inference model). This artificial intelligence engine typically includes a convolutional neural network (CNN) trained with training data containing a large number of images acquired by a slit lamp microscope and their interpretation information.

[0366] If the image interpretation device includes an artificial intelligence engine, and the slit lamp microscope 1 (data processing unit 8) also includes an artificial intelligence engine, these artificial intelligence engines can be adjusted to have equivalent capabilities. In other words, it is possible to adjust them so that there is no difference (or the difference is small) between the artificial intelligence engine of the image interpretation device and the artificial intelligence engine of the slit lamp microscope 1. To put it another way, the artificial intelligence engine provided in the image interpretation device may be at least partially the same as the aforementioned artificial intelligence engine provided in the slit lamp microscope 1.

[0367] For example, the same neural network model and the same parameters can be applied to both artificial intelligence engines. Furthermore, updates to the models and parameters in both engines can be synchronized.

[0368] By uniformly adjusting the artificial intelligence engine in this way, it becomes possible to prevent the inconvenience of inconsistencies or errors occurring between the output of the artificial intelligence engine of the slit lamp microscope 1 and the output of the artificial intelligence engine of the image interpretation device. Furthermore, as mentioned above, the quality evaluation performed before transmitting the image group from the slit lamp microscope 1 evaluates the image quality necessary for effective image interpretation and diagnosis. Therefore, by uniformly adjusting the artificial intelligence engine, it becomes possible to appropriately evaluate the quality of the image group before transmission. In other words, the image group that can be interpreted by the image interpretation device's artificial intelligence engine can be evaluated with high accuracy as an "interpretable image group" before transmission from the slit lamp microscope 1, and the image group that cannot be interpreted by the image interpretation device's artificial intelligence engine can be evaluated with high accuracy as an "uninterpretable image group" before transmission from the slit lamp microscope 1.

[0369] Furthermore, in cases where such unified adjustment of artificial intelligence engines is not performed, or when requesting image interpretation from other image interpretation devices (including artificial intelligence engines with different models and / or parameters), it is possible to attach the processing conditions (representing the model, parameters, etc.) of the artificial intelligence engine of the slit lamp microscope 1 to the image set, or to attach the processing conditions of the artificial intelligence engine of the image interpretation device to the image set.

[0370] According to the first example of operation, if a good quality set of images is obtained in the first scan, this set of images can be provided for subsequent processing (such as image interpretation). If a good quality set of images is not obtained in the first scan, the first example of operation can perform another scan to obtain a new set of images. More generally, the first example of operation can repeat the acquisition process until a good quality set of images is obtained. Therefore, whether or not a good quality set of images is obtained in the first scan, it is possible to provide a good quality set of images for subsequent processing. Consequently, processing such as image interpretation can be performed appropriately.

[0371] Furthermore, according to the first operational example, if the first follow-up imaging is performed successfully, the image set acquired in this follow-up imaging can be provided for subsequent processing (such as image interpretation). If the first follow-up imaging is not performed successfully, the first operational example can perform the follow-up imaging again to acquire a new image set. More generally, the first operational example can repeat imaging until a successful follow-up imaging is performed. Therefore, whether the first follow-up imaging is performed successfully or not, it is possible to provide a group of images (follow-up images) with small positional errors relative to the images obtained in previous imaging (reference images) for subsequent processing. Thus, it is possible to suitably perform image comparison in follow-up.

[0372] Thus, the first operational example allows for both quality evaluation of the image group and quality evaluation of follow-up imaging. Therefore, it facilitates automated image interpretation by an information processing device (image interpretation device) and interpretation by a physician (such as a radiologist), as well as facilitates image comparison during follow-up. This concludes the explanation of the first operational example.

[0373] <Second example of operation> A second example of the operation performed after the preparations described above will be explained with reference to Figures 13 and 14. Figure 13 shows an example of the operation to obtain a good quality image set by follow-up imaging. Figure 14 shows an example of the operation to evaluate the quality (error) of the follow-up imaging.

[0374] The first example of operation involves repeating anterior segment scanning (capture) until a good quality image set (a series of images collected in a single scan) is obtained. However, this example of operation involves repeating scanning (capture) until a good quality image set (for example, a series of images arranged within a predetermined scan range) is obtained.

[0375] In other words, while the data ultimately obtained in the first operational example is a series of images collected in a single scan, the data ultimately obtained in this operational example is, for example, a series of images corresponding to multiple cross-sections arranged within a predetermined scan range (a three-dimensional region of the anterior segment of the eye). That is, the data ultimately obtained in this operational example may be a series of images obtained in a single scan, similar to the first operational example, or, unlike the first operational example, a series of images selected from two or more image groups obtained in two or more scans.

[0376] In this example, the operation example shown in Figure 13 is followed by the operation example shown in Figure 14. In other words, in this example, the quality of the follow-up imaging is evaluated after a good quality image set has been obtained. Although not illustrated and described in detail, several other exemplary embodiments, conversely to this example, can evaluate the quality of the image set after evaluating the quality of the follow-up imaging. Several further exemplary embodiments can perform the evaluation of the quality of the follow-up imaging and the evaluation of the quality of the image set in parallel. Several further exemplary embodiments can perform the evaluation of the image group quality, the evaluation of the follow-up imaging quality, and the evaluation of the quality of the image set in that order. Those skilled in the art will be able to understand these other exemplary embodiments from the second operation example described below.

[0377] Steps S11 to S13 of this example may be performed in the same manner as steps S1 to S3 of the first example. Therefore, the explanation of steps S11 to S13 will be omitted. Unless otherwise specified, the matters described in the first example can be combined with this example.

[0378] (S14: Is the quality good?) If the image quality is determined to be good in step S13 (S14: Yes), this operation proceeds to the evaluation of the quality (error) of the follow-up imaging (S15). On the other hand, if the image quality is determined to be poor in step S13 (S14: No), this operation proceeds to step S16.

[0379] (S15: Proceed to error evaluation) If the quality of the image group is determined to be good in step S13 (S14: Yes), the control unit 7 performs control to execute a process to evaluate the quality (error) of the follow-up imaging obtained from this image group. This control may be performed in the same manner as in step S5 of the first operation example, for example. After step S15, this operation proceeds to step S31 in Figure 14.

[0380] (S16: Prompt reshoot) If the quality of the image group is determined to be poor in step S13 (S14: No), the control unit 7 will perform a control to prompt the user to retake the images. The user will perform an instruction operation to start retaking the images, or an instruction operation to not retake the images.

[0381] (S17: Alignment) When the user performs an instruction operation to start reshooting, the control unit 7 performs alignment in the same manner as in step S11.

[0382] (S18: Follow-up imaging (scan)) Once the alignment in step S17 is complete, the slit lamp microscope 1 scans the three-dimensional region of the anterior segment of the eye E under examination in the same manner as in step S12. This yields a new set of images.

[0383] (S19: Select image) The selection unit 821 selects one or more images that satisfy predetermined conditions from a set of images including the image group collected in the scan of step S12 and the image group collected in the scan of step S19. If "No" is determined two or more times in step S14, the selection unit 821 selects one or more images that satisfy predetermined conditions from a set of images including the image group collected in the scan of step S12 and two or more image groups collected in the scan of step S19 which has been performed two or more times.

[0384] In some exemplary embodiments, the operations in steps S18 and S19 may be either an operation in which scanning and image selection are repeated alternately, as described above, or an operation in which image selection is performed after all scans have been completed. Through such coordinated scanning and image selection operations, a series of images are selected from two or more image groups obtained from two or more scans.

[0385] (S20: Create image set) The image set creation unit 82 creates an image set containing the series of images selected in step S19.

[0386] (S21: Evaluate the quality of the image set) The image set evaluation unit 83 evaluates the quality of the image set created in step S20.

[0387] (S14: Is the quality good?) If the image set quality is determined to be good in step S21 (S14: Yes), this operation proceeds to the evaluation of the quality (error) of the follow-up imaging (S15). On the other hand, if the image set quality is determined to be poor in step S21 (S14: No), this operation proceeds to step S16, and steps S16 to S14 are executed again.

[0388] (S15: Proceed to error evaluation) If the quality of the image set is determined to be good in step S13 (S14: Yes), the control unit 7 performs control to execute a process to evaluate the quality (error) of the follow-up imaging from which this image set was obtained. This control may be performed in the same manner as in step S5 of the first operation example, for example. After step S15, this operation proceeds to step S31 in Figure 14.

[0389] If this image set consists of a single group of images acquired in a single anterior segment scan, that is, if "No" is never determined in step S14, in other words, if "Yes" is determined in the first step S14, then the operations in steps S31 to S35 may be performed in the same manner as in the first example of operation.

[0390] In contrast, if an image set is created from two or more image groups acquired from two or more anterior segment scans, that is, if "No" is determined at least once in step S14, in other words, if "No" is determined in the first step S14, the operations in steps S31 to S35 may be performed, for example, according to any of the embodiments illustrated below.

[0391] The first embodiment involves selecting an anterior segment scan from two or more anterior segment scans, and then performing steps S31 to S35 for the selected anterior segment scan in the same manner as in the first example of operation. The method of selecting the anterior segment scan may be arbitrary. For example, the first anterior segment scan and / or the last anterior segment scan can be selected.

[0392] The second embodiment involves performing steps S31 to S35 for each of two or more anterior segment scans in the same manner as in the first example, and evaluating the error of the follow-up imaging from the two or more results obtained. This error evaluation is performed, for example, by statistically processing the two or more results. One example involves selecting one of the two or more results (for example, the evaluation result with the largest error) and performing a final error evaluation based on the selected result. Another example involves applying a predetermined statistical processing to the two or more results and performing a final error evaluation based on the data obtained therefrom (for example, the mean).

[0393] (S31: Generate positional shift information) After step S15, the operation proceeds to step S31 in Figure 14. In step S31, the positional displacement information acquisition unit 86 analyzes the image set and / or one or more image groups (follow-up images) that were determined to be of good quality in step S14, and past images (reference images) referenced in the follow-up shooting, to generate positional displacement information between the follow-up images and the reference images.

[0394] (S32: Calculate evaluation value) Next, the evaluation value calculation unit 87 calculates an evaluation value of the scan position error in follow-up imaging based on the positional displacement information acquired in step S31.

[0395] (S33: Determining the quality of follow-up imaging) Next, the quality determination unit 88 determines whether the follow-up imaging was performed well based on the evaluation value calculated in step S32.

[0396] (S34: Is it good?) If it is determined in step S33 that the follow-up imaging was performed successfully (S34: Yes), this operation proceeds to step S35.

[0397] On the other hand, if it is determined in step S33 that the follow-up imaging was not performed successfully (S34: No), this operation returns to step S16 in Figure 13. The processing in this case is the same as the processing related to step S16 described above.

[0398] (S35: Output image) If it is determined in step S33 that the follow-up imaging was successful (S34: Yes), the control unit 7 performs the necessary actions to output this image set. In this example, the control unit 7 controls the communication unit 9 to transmit the image set to another device.

[0399] The destination for sending the image set may be the same as or different from the destination for sending the image group in the first example of operation. The output image set may include background information, positional displacement information, evaluation values, pass / fail judgment results, etc. Alternatively, this information may be supplementary information to the image set. Similar to the image group, the transmitted image set may include a series of images of the anterior segment of the subject's right eye and a series of images of the anterior segment of the left eye, and furthermore, subject eye information may be attached to the series of images of the right eye and the series of images of the left eye, respectively. In addition, subject identification information may be transmitted along with the image set.

[0400] The image set transmitted from the slit lamp microscope 1 in step S35 is sent directly or indirectly to an information processing device. This information processing device may be, for example, a picture interpretation terminal and / or picture interpretation device.

[0401] According to the second example of operation, if a good quality set of images is obtained in the first scan, this set of images can be provided for subsequent processing (such as image interpretation). If a good quality set of images is not obtained in the first scan, the second example of operation can acquire a new set of images and select a series of images from two or more sets of images obtained up to that point to create a good quality image set. More generally, the second example of operation can repeat the acquisition until a good quality image set is obtained. Therefore, if a good quality set of images is obtained in the first scan, that set of images can be provided for subsequent processing, and if a good quality set of images is not obtained in the first scan, it is possible to create a good quality image set from two or more sets of images obtained in two or more scans and provide it for subsequent processing. Thus, processing such as image interpretation can be performed suitably.

[0402] Furthermore, according to the second operational example, if the first follow-up imaging is successful, the image set acquired in this follow-up imaging can be provided for subsequent processing (such as image interpretation). If the first follow-up imaging is not successful, the second operational example can perform the follow-up imaging again to acquire a new image set, and create an image set from two or more image sets, including this new image set. More generally, the second operational example can repeat imaging until an image set based on successful follow-up imaging is obtained. Therefore, whether the first follow-up imaging is successful or not, it is possible to provide an image set or group of images (follow-up images) with small positional errors relative to the images obtained in previous imaging (reference images) for subsequent processing. Thus, it is possible to suitably perform image comparison in follow-up.

[0403] Thus, the second operational example can perform quality evaluation of image groups, quality evaluation of image sets, and quality evaluation of follow-up imaging. Therefore, it can suitably perform automatic image interpretation by an information processing device (image interpretation device) or image interpretation by a physician (such as a radiologist), as well as suitably perform image comparison during follow-up. This concludes the explanation of the second operational example.

[0404] The operations that the slit lamp microscope 1 of this embodiment can perform are not limited to the two examples described above. For example, the slit lamp microscope 1 can perform operations based on at least one or more of the matters described in this embodiment, variations thereof, and any known techniques.

[0405] For example, the slit lamp microscope 1 may be configured to transmit directly or indirectly to an information processing device the image set acquired in a single scan (single follow-up scan) of the anterior segment of the eye E under examination, without performing re-shooting as in the two examples described above.

[0406] Furthermore, the slit lamp microscope 1 may be configured to apply an analysis process to a series of images (image group, image set) collected from the three-dimensional region of the anterior segment of the eye E under examination to obtain analysis data (e.g., anterior segment parameter values), and to transmit this analysis data directly or indirectly to the information processing device. Alternatively, the slit lamp microscope 1 may be configured to apply an analysis process to a series of images (image group, image set) collected from the three-dimensional region of the anterior segment of the eye E under examination to obtain analysis data (e.g., anterior segment parameter values), and to transmit this analysis data and the series of images directly or indirectly to the information processing device.

[0407] <Effects> Some features, functions, and effects of the slit lamp microscope in the exemplary embodiment described above will be explained.

[0408] The slit lamp microscope 1 includes an image acquisition unit (illumination system 2, imaging system 3, movement mechanism 6, etc.), a storage unit 10, and a positional displacement information acquisition unit 86. The image acquisition unit scans the anterior segment of the eye E under examination with slit light to acquire an image. The storage unit 10 stores a first image of the anterior segment of the eye E under examination, and also stores a second image acquired in a follow-up imaging performed by the image acquisition unit by referring to this first image. After the follow-up imaging is performed, the positional displacement information acquisition unit 86 analyzes the first and second images to acquire positional displacement information between the first and second images.

[0409] With this type of slit-lamp microscope 1, follow-up imaging can be performed using a modality that scans the anterior segment of the eye E under examination with slit light, and furthermore, positional displacement information regarding the error of this follow-up imaging can be obtained. Therefore, it is possible to know the accuracy and precision of the follow-up imaging, that is, to obtain information on the error of the imaging position (imaging range) in the follow-up, and it is possible to provide the user with information indicating whether the follow-up imaging was performed properly, and even information indicating the degree of accuracy and precision with which the follow-up imaging was performed, thereby contributing to the improvement of diagnostic accuracy. Thus, the slit-lamp microscope 1 in this exemplary embodiment can provide a useful evaluation regarding the error of follow-up imaging.

[0410] A slit lamp microscope having the optical system shown in Figure 10 is capable of capturing a frontal image of the anterior segment of the eye E under examination (frontal image acquisition unit). Frontal imaging of the anterior segment is performed, for example, by a video recording unit 40. In this case, the frontal image acquisition unit includes the video recording unit 40. In another example, the frontal image of the anterior segment is constructed by combining two or more anterior segment images acquired by two or more anterior segment cameras disclosed by the present applicant in Japanese Patent Application Publication No. 2013-248376, etc. In this case, the frontal image acquisition unit includes two or more anterior segment cameras and a processor that performs image synthesis (for example, a data processing unit 8).

[0411] The first image referenced in the follow-up imaging of the slit lamp microscope in this example may include a first frontal image obtained by imaging the anterior segment E. The first frontal image may be obtained by the slit lamp microscope in this example, or it may be obtained (and constructed) by another device. Furthermore, the second image obtained in the follow-up imaging performed with reference to the first frontal image may include a second frontal image obtained by the frontal image acquisition unit in this follow-up imaging. The positional displacement information acquisition unit 86 in this example is capable of generating positional displacement information based on at least the first frontal image and the second frontal image.

[0412] A slit lamp microscope capable of acquiring a frontal image in this manner can display the scan position on the frontal image based on positional displacement information. To this end, the storage unit 10 further stores first position information indicating the position of the image acquired by the image acquisition unit together with the first frontal image in the first frontal image, and second position information indicating the position of the image acquired by the image acquisition unit together with the second frontal image in the second frontal image. The first position information is, for example, the application position (application range) of the anterior segment scan performed when the first frontal image referenced in follow-up imaging was acquired, expressed in the coordinate system of the first frontal image. The second position information is, for example, the application position (application range) of the anterior segment scan in follow-up imaging performed with reference to the first frontal image, expressed in the coordinate system of the second frontal image obtained by frontal imaging performed together with this anterior segment scan. Furthermore, the slit lamp microscope in this example can display the first frontal image and / or the second frontal image on a display device. In addition, the slit lamp microscope in this example can overlay a first position image corresponding to the first position information and a second position image corresponding to the second position information onto the displayed first front image and / or second front image, based on the positional displacement information acquired by the positional displacement information acquisition unit 86. These display processes are performed by the display control unit. In the slit lamp microscope of this exemplary embodiment, the control unit 7 functions as the display control unit.

[0413] A slit lamp microscope capable of acquiring a frontal image, or a slit lamp microscope not capable of acquiring a frontal image, may be configured as follows: The first image referenced in follow-up imaging may include a first scan image acquired by scanning the anterior segment of the eye E with slit light. The first scan image may be acquired by the slit lamp microscope of this example or by another device. The second image acquired in follow-up imaging performed with reference to the first scan image may include a second scan image acquired by the image acquisition unit in this follow-up imaging. The positional displacement information acquisition unit 86 of this example can acquire positional displacement information based on at least the first scan image and the second scan image. According to this example, since positional displacement information can be generated from the scan image obtained in the anterior segment scan, there is no need to provide a configuration for acquiring a frontal image (frontal image acquisition unit), and the configuration can be simplified.

[0414] Furthermore, the process of determining the positional displacement from the front image and the process of determining the positional displacement from the scanned image can be combined. In some exemplary embodiments, the slit lamp microscope may be configured to determine positional displacement information in the XY direction from the front image (and scanned image), and positional displacement information in the Z direction from the scanned image.

[0415] An exemplary slit lamp microscope 1 may include an evaluation value calculation unit 87. The evaluation value calculation unit 87 calculates an evaluation value of the scan position error in follow-up imaging based on the positional misalignment information acquired by the positional misalignment information acquisition unit 86. This makes it possible to quantitatively grasp the degree of error in follow-up imaging.

[0416] An exemplary slit lamp microscope 1 may include a quality determination unit 88. The quality determination unit 88 determines whether the follow-up imaging was performed well based on the evaluation value calculated by the evaluation value calculation unit 87. This makes it possible to quantitatively and automatically evaluate the degree of error in the follow-up imaging.

[0417] An exemplary slit lamp microscope 1 may include a communication unit 9 (first communication unit). Furthermore, the slit lamp microscope 1 can transmit the images acquired in the follow-up imaging to a predetermined external device, provided that the quality determination unit 88 determines that the follow-up imaging was performed successfully. This makes it possible to provide images obtained from successful follow-up imaging to be used for image interpretation and other purposes. The conditions for transmitting the images acquired from the follow-up imaging to the external device may consist only of the conditions based on the results of the quality determination unit 88 described herein, or they may include other conditions as well.

[0418] In some exemplary embodiments, the image acquisition unit includes a scanning unit that collects a set of images by scanning the anterior segment of the eye E under examination with a slit beam. The scanning unit includes an illumination system (2), an imaging system (3), and a moving mechanism (6). The illumination system irradiates the anterior segment of the eye E under examination with a slit beam. The imaging system photographs the anterior segment from a different direction than the illumination system. The moving mechanism moves the illumination system and the imaging system. In parallel with the movement of the illumination system and the imaging system by the moving mechanism, the imaging system repeatedly takes images. This repeated shooting is, for example, video recording at a predetermined shooting rate.

[0419] With this configuration, firstly, it is possible to acquire a set of images representing the anterior segment of the eye E under examination without contact, thus eliminating the need to apply topical anesthetics or corneal protective agents (ophthalmic gels) to the eye E under examination, thereby reducing the burden on the patient. Furthermore, unlike conventional slit-lamp microscopes that provide images of a single cross-section, the slit-lamp microscope in this exemplary embodiment can scan a three-dimensional region of the anterior segment of the eye E under examination to obtain a set of images. This makes it possible to observe in detail a desired part of the anterior segment during diagnosis (such as image interpretation) performed after imaging.

[0420] Furthermore, as mentioned above, obtaining good images using conventional slit-lamp microscopes requires delicate and complicated operations. However, with remote operation technologies such as those disclosed in Patent Documents 3 and 4, such difficult operations had to be performed remotely, making it extremely difficult to obtain images of sufficient quality for diagnosis (interpretation). In addition, while slit-lamp microscopes are powerful tools for screening and follow-up, considering the difficulty of remote operation, it can even be said that using slit-lamp microscopes for telemedicine is practically impossible with conventional technology.

[0421] In contrast, the exemplary slit-lamp microscope is configured to first scan the eye E under examination and collect a set of images, thus eliminating the need for remote operation by a physician as in conventional methods. In other words, conventionally, a physician performs the examination while operating the microscope remotely, but when using the slit-lamp microscope 1, the physician only needs to interpret the pre-acquired image set. Therefore, the physician is freed from the time and effort of taking images and can concentrate on interpreting the images. Thus, this embodiment contributes to expanding the range of high-quality slit-lamp microscopy examinations that can be provided.

[0422] In an exemplary embodiment, the moving mechanism 6 may move the illumination system 2 and the imaging system 3 so as to move the slit light, which is irradiated onto the anterior segment of the eye E under examination by the illumination system 2, in the width direction of the slit light.

[0423] An exemplary slit-lamp microscope 1 may function as a Scheinproof camera to acquire, for example, an image (two-dimensional cross-sectional image) in focus from the anterior surface of the cornea to the posterior surface of the lens. To this end, the imaging system 3 may include an optical system 4 that guides the light (reflected light) from the anterior segment of the eye E irradiated by the slit light, and an image sensor 5 that receives the light guided by the optical system 4 on its imaging surface. Furthermore, the slit-lamp microscope 1 may be configured such that the object surface along the optical axis of the illumination system 2, the optical system 4, and the image sensor 5 (imaging surface) satisfy the Scheinproof condition. With such a configuration, it is possible to focus and image the entire three-dimensional region of the anterior segment, and to acquire images of good quality.

[0424] In some exemplary embodiments, the second image acquired in the follow-up imaging may include at least one image from the image set collected by the scanning unit in this follow-up imaging. Each image in this image set is typically a two-dimensional cross-sectional image.

[0425] In some exemplary embodiments, the image acquisition unit may include a processed image construction unit (85) that constructs a processed image from a group of images collected by the scanning unit during follow-up imaging. The processed image may be, for example, a three-dimensional image constructed from the group of images, or a rendering image constructed from the three-dimensional image. The second image obtained by follow-up imaging may include such a processed image.

[0426] Thus, the second image obtained from follow-up imaging may include various types of images, such as images included in the image group (2D cross-sectional images), 3D images, and rendered images. The types of images included in the second image may be determined arbitrarily. Furthermore, processed images may be constructed using external devices (image interpretation devices, image interpretation terminals, other information processing devices, etc.).

[0427] An exemplary slit lamp microscope 1 may include an image processing unit 80 that processes the image group collected by the scanning unit. This makes it possible to perform desired processing on the image group collected by the scanning unit.

[0428] In an exemplary embodiment, the image group processing unit 80 may include an image group evaluation unit 81 (first evaluation unit). The image group evaluation unit 81 is configured to evaluate the quality of the image group collected by the scanning unit. With this configuration, the quality of the image group collected by scanning the anterior segment of the eye E under examination can be evaluated by the slit lamp microscope itself, so the quality of the image group can be confirmed before providing it to a physician or image interpretation device.

[0429] For example, as explained below, the control content can be switched according to the evaluation result of the image group quality by the image group evaluation unit 81. This allows for the execution of appropriate processing if a good quality image group is obtained, and for the execution of appropriate processing if a good quality image group is not obtained.

[0430] In this embodiment, the slit lamp microscope 1 can transmit the image group to a predetermined external device via the communication unit 9 (second communication unit) if the image group evaluation unit 81 evaluates that the quality of the image group collected by the scanning unit is good. With this configuration, when a good quality image group is acquired, this image group can be provided, for example, to a physician in a remote location or a remote image interpretation device. The conditions for transmitting the image group acquired by follow-up imaging to an external device may be limited to the image group quality conditions described herein, or they may include other conditions as well.

[0431] The slit lamp microscope 1 in this embodiment may include a control unit 7 (first control unit) as follows. When the image group evaluation unit 81 evaluates that the quality of the image group is not good, the control unit 7 executes control to cause the scanning unit to perform a new scan of the anterior segment of the eye E under examination. The control to cause the scanning unit to perform a new scan may be, for example, control to prompt the user to retake the image, or control to perform a retake automatically (control to automatically transition to retaking the image). With this configuration, if a good quality image group cannot be obtained in one scan, it is possible to smoothly transition to retaking (rescanning).

[0432] In an exemplary embodiment, the image group processing unit 80 may include an image set creation unit 82. The image set creation unit 82 is configured to create an image set by selecting a series of images corresponding to a predetermined scan range from two or more image groups, which include an image group already acquired by scanning the anterior segment of the eye E under examination and a new image group collected by a new scan of the anterior segment of the eye E under examination. With this configuration, if two or more scans are applied to the eye under examination, an image set can be created from a series of images selected from the two or more image groups obtained thereby, thus improving the likelihood of obtaining an image set of good quality. For example, even if blinking or eye movement occurs during a scan and a good image cannot be obtained, it is possible to compensate for this with images obtained from other scans.

[0433] In an exemplary embodiment, the image group processing unit 80 may include an image set evaluation unit 83 (second evaluation unit) that evaluates the quality of the image set created by the image set creation unit 82. For example, the image set evaluation unit 83 can evaluate a series of images included in the image set for any of the following: sequence order, missing images, or positional misalignment. This evaluation is performed by analyzing a series of images in the image set. For example, the quality of a series of images is evaluated based on landmarks in a series of frames (image regions corresponding to tissues such as the cornea, iris, pupil, and iridocorneal angle). By performing such an evaluation of the image set, it is possible to prepare an image set of good quality that enables effective interpretation by physicians and imaging devices.

[0434] In an exemplary embodiment, the slit lamp microscope 1 can transmit an image set to a predetermined external device via the communication unit 9, provided that the image set evaluation unit 83 has evaluated the image set as being of good quality. With this configuration, when a good quality image set is obtained, this image set can be provided, for example, to a physician in a remote location or a remote image interpretation device. The conditions for transmitting the image set acquired by follow-up imaging to an external device may be limited to the image set quality conditions described herein, or they may include other conditions as well.

[0435] An exemplary slit lamp microscope 1 may include a control unit 7 (second control unit) as follows: When the image set evaluation unit 83 evaluates that the quality of the image set is not good, the control unit 7 executes control to cause the scanning unit to perform a new scan of the anterior segment of the eye E under examination. The control to cause the scanning unit to perform a new scan may be, for example, control to prompt the user to retake the image, or control to perform a retake automatically (control to automatically transition to retaking the image). With this configuration, if a good quality image set cannot be obtained after two or more scans up to this point, it is possible to smoothly transition to retaking (rescanning).

[0436] <Other matters> The embodiments described above are merely illustrative examples of the present invention. Therefore, any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the present invention can be appropriately applied to the above embodiments.

[0437] In the embodiments described above, the image group and / or image set are evaluated after each scan. In other embodiments, the slit lamp microscope may be configured to first perform multiple scans in succession to collect multiple image groups, and then create an image set from these image groups. That is, the slit lamp microscope in this embodiment may be configured as follows: the scanning unit applies two or more scans to the anterior segment of the eye under examination to collect two or more image groups; the image group processing unit (image set creation unit) selects a series of images corresponding to the scan range from the two or more collected image groups to create an image set. According to this embodiment, a good image set can be obtained as long as no problems (blinking, eye movement, etc.) occur in all of the multiple scans, thus increasing the likelihood of obtaining a good image set. It is possible to combine any embodiment relating to the quality evaluation of follow-up imaging with such embodiments.

[0438] The use of a slit lamp microscope according to any of the exemplary embodiments is arbitrary, and it may be equipped with functions for such purposes. Some exemplary slit lamp microscopes may be equipped with statistical analysis functions in addition to the follow-up functions described above. These functions are used to quantitatively and objectively evaluate the condition of the eye under examination. As described above, the slit lamp microscope according to the exemplary embodiments makes it possible to perform detailed and reliable non-contact anterior segment observation, thus enabling effective and suitable follow-up and statistical analysis. Note that devices other than slit lamp microscopes (information processing devices, image interpretation terminals, image interpretation devices, ophthalmic devices, etc.) may be equipped with similar functions (the same applies hereinafter).

[0439] Such a slit lamp microscope has a function (parameter calculation function) that analyzes images acquired based on a scan of the anterior segment of the eye under examination and calculates the values ​​of predetermined parameters. The images to be analyzed may be any images, for example, a group of images collected by a scan, at least one image included in the group of images, a three-dimensional image constructed from at least a portion of the group of images, a rendered image of the three-dimensional image, a set of images obtained from two or more group of images, at least one image included in the set of images, a three-dimensional image constructed from at least a portion of the set of images, and a rendered image of the three-dimensional image. The parameters to be calculated may be any parameters, for example, they may include any of the anterior segment parameters mentioned above. Parameter calculation includes processing according to the target parameters and typically includes segmentation, feature point detection, distance measurement, ratio calculation, angle calculation, etc.

[0440] Statistical analysis functions include, for example, comparison with normal eye data (normative data comparison function) and comparison with diseased eye data (disease eye data comparison function).

[0441] A slit-lamp microscope with a normative data comparison function either has a normal eye database (normative database) obtained by statistically processing parameter values ​​from a large number of normal eyes, or is configured to refer to a normative database stored in another device. A typical normative database defines the normal range of parameter values. The slit-lamp microscope determines whether the parameter value obtained from the eye under examination falls within the normal range. If the parameter value obtained from the eye under examination falls within the normal range, the eye under examination is judged to be normal (at least with respect to this parameter). If the parameter value obtained from the eye under examination is determined not to fall within the normal range, the eye under examination is judged to be abnormal (at least with respect to this parameter).

[0442] A slit-lamp microscope with a diseased eye data comparison function either has a database for a specific disease (diseased eye database) or is configured to reference a diseased eye database stored in another device. The diseased eye database is obtained by statistically processing parameter values ​​obtained from a large number of eyes (disease eyes, affected eyes) diagnosed with a specific disease. A typical diseased eye database defines the abnormal range of parameter values. The slit-lamp microscope determines whether the parameter value obtained from the eye under examination falls within the abnormal range. If the parameter value obtained from the eye under examination does not fall within the abnormal range, the eye under examination is judged to be normal (at least with respect to this parameter). If the parameter value obtained from the eye under examination is determined not to fall within the abnormal range, the eye under examination is judged to be abnormal (at least with respect to this parameter).

[0443] A slit-lamp microscope with follow-up capabilities typically has the ability to compare or present time-series data from the same patient in a comparable manner. For example, such a slit-lamp microscope may have the ability to determine the time-series changes in multiple parameter values ​​acquired at multiple times for the same patient, to determine the trend of these parameter values, to compare the time-series changes in these parameter values ​​with standard changes (standard changes in a normal or diseased eye), and to estimate future values ​​based on these parameter values ​​(and standard changes). The slit-lamp microscope also has the ability to present the results obtained from any of these functions. Any method can be used to present the results, such as graphs, histograms, maps, or color codes.

[0444] This disclosure provides a method for controlling a slit lamp microscope according to any embodiment. The slit lamp microscope includes a processor, a memory device, and an image acquisition unit that scans the anterior segment of the eye under examination with slit light to acquire an image. The control method includes the steps of causing the processor to perform control for storing a first image of the anterior segment of the eye under examination in the memory device, and causing the processor to perform control for storing a second image acquired in a follow-up imaging performed by the image acquisition unit with reference to the first image in the memory device. Furthermore, the control method includes the step of controlling the processor after follow-up imaging has been performed to cause the processor to perform analysis of the first and second images to acquire positional displacement information between the first and second images.

[0445] This disclosure provides a program for causing a computer to execute such a control method. Furthermore, this disclosure provides a computer-readable non-temporary recording medium on which such a program is recorded. This non-temporary recording medium may be in any form, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory.

[0446] This disclosure provides an imaging method and a processing method according to any embodiment. The imaging method includes the steps of acquiring a first image of the anterior segment of the eye under examination and storing the acquired first image. Furthermore, the imaging method includes the step of performing a follow-up imaging with reference to the first image. This follow-up imaging is performed by scanning the anterior segment of the eye under examination with a slit light. This yields a second image. Furthermore, the imaging method includes the step of storing the second image acquired in the follow-up imaging. In addition, after the follow-up imaging has been performed, the imaging method includes the step of analyzing the first and second images to acquire positional displacement information between the first and second images.

[0447] This processing method includes the steps of storing a first image of the anterior segment of the eye under examination and storing a second image obtained by follow-up imaging referencing the first image. This follow-up imaging is performed by scanning the anterior segment of the eye under examination with a slit light. Furthermore, after the follow-up imaging is performed, this processing method includes the step of analyzing the first and second images to obtain positional displacement information between the first and second images.

[0448] This disclosure provides a program that causes a computer to perform such a photographic and / or processing method. Furthermore, this disclosure provides a computer-readable non-temporary recording medium on which such a program is recorded.

[0449] Any of the matters described in the above exemplary embodiments can be combined with any of the exemplary control methods for a slit lamp microscope, exemplary imaging methods, exemplary processing methods, exemplary programs, and exemplary recording media. [Explanation of Symbols]

[0450] 1. Slit lamp microscope 2. Lighting System 3. Photography 4 Optical system 5 Image sensor 6 Moving mechanism 7 Control Unit 8. Data Processing Unit 80 Image group processing unit 81 Image Group Evaluation Unit 82 Image Set Creation Section 83 Image Set Evaluation Section 85 Image Processing and Construction Section 86 Position shift information acquisition unit 87 Evaluation Value Calculation Unit 88. Quality Determination Unit 9 Communications Department 10 Storage section

Claims

1. an image acquisition unit that acquires an image by scanning an anterior segment of the subject's eye with a slit light; a storage unit configured to store a first image of the anterior segment and a second image acquired in a follow-up imaging performed by the image acquisition unit with reference to the first image; a positional deviation information acquisition unit that analyzes the first image and the second image after the follow-up imaging is performed and acquires positional deviation information between the first image and the second image; Including, slit lamp microscope.

2. further comprising a front image acquisition unit that acquires a front image by photographing the anterior segment of the eye, the first image includes a first front image acquired by photographing the anterior segment of the eye, the second image includes a second front image acquired by the front image acquisition unit in the follow-up imaging performed with reference to the first front image, the positional deviation information acquisition unit acquires the positional deviation information based on at least the first front image and the second front image.

2. The slit lamp microscope of claim 1.

3. the storage unit further stores first position information indicating a position in the first front image of an image acquired by the image acquisition unit together with the first front image, and second position information indicating a position in the second front image of an image acquired by the image acquisition unit together with the second front image, a display control unit that causes a display unit to display at least one of the first front image and the second front image, and that causes a first position image corresponding to the first position information and a second position image corresponding to the second position information to be superimposed on the at least one front image based on the positional deviation information.

3. The slit lamp microscope of claim 2.

4. the first image includes a first scanned image acquired by scanning the anterior segment with a slit light; the second image includes a second scan image acquired by the image acquisition unit in the follow-up imaging performed with reference to the first scan image; the positional deviation information acquisition unit acquires the positional deviation information based on at least the first scanned image and the second scanned image. The slit lamp microscope according to any one of claims 1 to 3.

5. further including an evaluation value calculation unit that calculates an evaluation value of an error in the scan position in the follow-up imaging based on the positional deviation information. The slit lamp microscope according to any one of claims 1 to 4.

6. further including a quality determination unit that determines whether the follow-up imaging has been performed satisfactorily based on the evaluation value calculated by the evaluation value calculation unit.

6. The slit lamp microscope of claim 5.

7. further including a first communication unit; and on condition that the quality determination unit determines that the follow-up photographing has been performed satisfactorily, the first communication unit transmits the image acquired by the image acquisition unit in the follow-up photographing to a predetermined external device.

7. The slit lamp microscope of claim 6.

8. the image acquisition unit includes a scanning unit that scans the anterior segment with the slit light to collect a group of images; The scanning unit an illumination system that irradiates the anterior segment with the slit light; an imaging system that images the anterior segment from a direction different from that of the illumination system; a movement mechanism for moving the illumination system and the imaging system; Including, the imaging system repeatedly performs imaging in parallel with the movement of the illumination system and the imaging system by the movement mechanism; The slit lamp microscope according to any one of claims 1 to 7.

9. The imaging system includes: an optical system that guides light from the anterior segment irradiated with the slit light; an imaging element that receives the light guided by the optical system on an imaging surface; Including, an object plane along the optical axis of the illumination system, the optical system, and the imaging plane satisfy the Scheimpflug condition; 9. The slit lamp microscope of claim 8.

10. the moving mechanism moves the illumination system and the imaging system so that the slit light irradiated on the anterior eye segment by the illumination system moves in a width direction of the slit light.

10. The slit lamp microscope of claim 8 or 9.

11. the second image includes at least one image included in an image group acquired by the scanning unit in the follow-up imaging; The slit lamp microscope according to any one of claims 8 to 10.

12. The image acquisition unit further includes a processed image construction unit that constructs a processed image from the image group. The slit lamp microscope according to any one of claims 8 to 11.

13. the processed image construction unit constructs a three-dimensional image as the processed image.

13. The slit lamp microscope of claim 12.

14. the processed image construction unit constructs a rendering image of the three-dimensional image as the processed image.

14. The slit lamp microscope of claim 13.

15. The second image includes the processed image. The slit lamp microscope according to any one of claims 12 to 14.

16. The slit lamp microscope further includes an image group processing unit that processes the image group collected by the scanning unit. The slit lamp microscope according to any one of claims 8 to 15.

17. the image group processing unit includes a first evaluation unit that evaluates the quality of the image group; 17. The slit lamp microscope of claim 16.

18. further including a second communication unit; and on condition that the first evaluation unit evaluates that the quality of the image group is good, the second communication unit transmits the image group to a predetermined external device.

18. The slit lamp microscope of claim 17.

19. a first control unit that, when the first evaluation unit evaluates that the quality of the image group is not good, controls the scanning unit to perform a new scan of the anterior segment, 19. A slit lamp microscope according to claim 17 or 18.

20. the image group processing unit further includes an image set creation unit that selects a series of images corresponding to a scan range from two or more image groups including the image group and a new image group acquired by the new scan to create an image set; 20. The slit lamp microscope of claim 19.

21. the scanning unit applies two or more scans to the anterior segment to collect two or more sets of images; The image group processing unit includes an image set creation unit that selects a series of images corresponding to a scan range from the two or more image groups and creates an image set.

17. The slit lamp microscope of claim 16.

22. the image group processing unit further includes a second evaluation unit that evaluates the quality of the image set created by the image set creation unit.

22. A slit lamp microscope according to claim 20 or 21.

23. further including a third communication unit; and on condition that the second evaluation unit evaluates that the quality of the image set is good, the third communication unit transmits the image set to a predetermined external device.

23. The slit lamp microscope of claim 22.

24. a second control unit that controls the scanning unit to perform a new scan of the anterior segment when the second evaluation unit evaluates that the quality of the image set is not good, 24. A slit lamp microscope according to claim 22 or 23.

Images