Ophthalmic device, method for controlling an ophthalmic device, program, and recording medium

Abstract

To improve the image quality of an ophthalmologic apparatus in an imaging system using optical scan.SOLUTION: An ophthalmologic apparatus according to one embodiment comprises: an illumination system; an imaging system; a movement mechanism; and a control unit. The illumination system projects illumination light to a subject eye. The imaging system images the subject eye. The imaging system includes an imaging element. The illumination system and the imaging system are configured to satisfy a condition of shine proof. The movement mechanism moves the illumination system and the imaging system. The control unit performs control of the illumination system, imaging system, and movement mechanism to cause the imaging system to collect a series of images. The control unit controls at least one of the illumination system and the imaging system such that the projection time of the illumination light to the subject eye becomes shorter than the exposure time of the imaging element and at least a portion of the projection period of the illumination light to the subject eye and at least a portion of the exposure period of the imaging element overlap each other in the control for causing the imaging system to collect a series of images of the subject eye.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an ophthalmic apparatus, a method for controlling an ophthalmic apparatus, a program, and a recording medium. [Background technology]

[0002] Diagnostic imaging plays an important role in the field of ophthalmology. Various ophthalmic devices are used for diagnostic imaging. These devices include slit lamp microscopes, fundus cameras, scanning laser ophthalmoscopes (SLO), and optical coherence tomography (OCT). In addition, various examination and measurement devices, such as refractometers, keratometers, tonometers, specular microscopes, wavefront analyzers, and microperimeters, are also equipped with the function to photograph the anterior segment and fundus.

[0003] Among these various ophthalmic devices, one of the most widely and frequently used is the slit lamp microscope, also known as the stethoscope for ophthalmologists. A slit lamp microscope is an ophthalmic device that illuminates the subject's eye with a slit of light and observes and photographs the illuminated cross-section from the side using a microscope (see, for example, Patent Documents 1 and 2). Also known is a slit lamp microscope that can scan a three-dimensional area of ​​the subject's eye at high speed by using an optical system configured to satisfy the Scheimpflug condition (see, for example, Patent Document 3). In addition to slit lamp microscopes, other imaging methods that scan an object with a slit of light include rolling shutter cameras. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-159073 [Patent Document 2] Japanese Patent Application Laid-Open No. 2016-179004 [Patent Document 3] Japanese Patent Application Publication No. 2019-213733 Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present invention is to improve the image quality of an ophthalmic apparatus that uses an imaging method that uses optical scanning. [Means for solving the problem]

[0006] An ophthalmic apparatus according to an exemplary embodiment includes an illumination system, an imaging system, a movement mechanism, and a control unit. The illumination system projects illumination light onto the subject's eye. The imaging system captures an image of the subject's eye. The imaging system includes an image sensor. The illumination system and the imaging system are configured to satisfy the Scheimpflug condition. The movement mechanism moves the illumination system and the imaging system. The control unit controls the illumination system, the imaging system, and the movement mechanism to cause the imaging system to collect a series of images. In controlling the imaging system to collect a series of images of the subject's eye, the control unit controls at least one of the illumination system and the imaging system so that the projection time of the illumination light onto the subject's eye is shorter than the exposure time of the image sensor, and so that at least a portion of the projection period of the illumination light onto the subject's eye overlaps with at least a portion of the exposure period of the image sensor. [Effects of the Invention]

[0007] According to the exemplary embodiment, it is possible to improve the image quality of an ophthalmic apparatus that uses an imaging method that uses optical scanning. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for explaining the background of an embodiment. [Figure 2] FIG. 1 is a diagram for explaining an overview of an embodiment. [Figure 3] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 4A] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 4B] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 4C]1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 5] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 6] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 7] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 8] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 9] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 10] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 11] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 12] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 13] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 14] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 15] FIG. 10 is a flowchart illustrating a process performed by an ophthalmic apparatus according to an exemplary embodiment. [Figure 16] FIG. 10 is a flowchart illustrating a process performed by an ophthalmic apparatus according to an exemplary embodiment. [Figure 17A] 1 is a schematic diagram for explaining a process executed by an ophthalmologic apparatus according to an exemplary embodiment. [Figure 17B] 1 is a schematic diagram for explaining a process executed by an ophthalmologic apparatus according to an exemplary embodiment. [Figure 18] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. [Figure 19] 1 is a schematic diagram illustrating a configuration of an ophthalmic apparatus according to an exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Some exemplary aspects of the embodiments will now be described in detail with reference to the drawings.

[0010] Any known technology can be combined with any aspect of the present disclosure. For example, any matter disclosed in the documents cited in this specification can be combined with any aspect of the present disclosure. Furthermore, any known technology in the technical field related to the present disclosure can be combined with any aspect of the present disclosure.

[0011] The entire disclosure of Patent Document 3 (JP 2019-213733 A) is incorporated herein by reference. In addition, any technical matter disclosed by the applicant of the present application regarding the technology related to the present disclosure (matters disclosed in patent applications, papers, etc.) can be combined with any aspect of the present disclosure.

[0012] Any two or more of the various aspects of the present disclosure may be at least partially combined.

[0013] At least a portion of the functionality of the elements described in this disclosure is implemented using circuitry or processing circuitry. The circuitry or processing circuitry may be a general-purpose processor, a special-purpose processor, an integrated circuit, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), a field programmable gate array (FPGA)), or a combination of these devices configured and / or programmed to perform at least a portion of the disclosed functionality. The term "circuitry," "unit," "means," or the like refers to hardware that performs at least a portion of the disclosed functions or that is programmed to perform at least a portion of the disclosed functions. The hardware may be the hardware disclosed herein or may be known hardware that is programmed and / or configured to perform at least a portion of the described functions. In the case of a processor, where the hardware can be considered a type of circuitry, the term "circuitry," "unit," "means," or the like refers to a combination of hardware and software, where the software is used to configure the hardware and / or the processor.

[0014] <Background of the embodiment> The embodiment aims to improve ophthalmic imaging by collecting images while moving an optical system that satisfies the Scheimpflug condition. In conventional technology, image collection is performed in the operational mode shown in FIG. 1. The operational mode shown in FIG. 1 illustrates coordinated control (synchronous control) of light emission from a light source (output of illumination light, projection of illumination light onto the subject's eye), exposure (photography) by a camera, and scan position (position of the moved optical system). More specifically, the operational mode shown in FIG. 1 combines continuous emission of illumination light, repeated photography (exposure) by a camera, and continuous movement of the optical system from a scan start position to a scan end position. The repeated exposure of the camera is performed by alternating between exposure and charge transfer (and exposure standby).

[0015] In conventional ophthalmic imaging techniques, blurred images can be obtained due to movement of the scanning position during camera exposure or eye movement of the subject's eye. Image blurring not only degrades image quality but also adversely affects the quality of image analysis such as corneal detection and cell detection.

[0016] One possible solution to this problem would be to shorten the camera's exposure time, but this would place a heavy burden on the subject because illumination light would be continuously projected onto the subject's eye even when not being exposed. It is also possible to synchronize the movement control of the scan position with the exposure control of the camera to move the scan position in a stepped manner, i.e., to move the optical system intermittently, but repeated sudden starts and stops of the optical system would cause vibrations, adversely affecting the quality of the image.

[0017] <Outline of the embodiment> An embodiment of the present disclosure has been created based on at least this background, and is configured to perform control so that the projection time of illumination light onto the subject's eye is shorter than the exposure time of a camera (image sensor). This control includes control of an optical system (illumination system) that projects illumination light onto the subject's eye and / or control of an optical system (image capturing system) that captures an image of the subject's eye using an image sensor.

[0018] The illumination system may be controlled by any method, for example, electrical control such as control of a light source (on / off) or control of an electronic shutter, mechanical control such as control of a mechanical shutter or control of a rotary shutter, or a combination of electrical and mechanical control. These shutters are provided in the illumination system and are configured to switch between passing and blocking illumination light output from the light source (i.e., to switch between projecting and not projecting illumination light onto the subject's eye).

[0019] The control of the illumination system is not limited to control for switching between a state in which illumination light is projected onto the subject's eye (projection state) and a state in which it is not projected onto the subject's eye (non-projection state), but may also be control for modulating the intensity (light amount) of illumination light projected onto the subject's eye. Switching between the projection state and the non-projection state corresponds to switching the intensity of illumination light projected onto the subject's eye between a positive value and zero. On the other hand, intensity modulation corresponds to switching the intensity of illumination light projected onto the subject's eye between a first value and a second value that are different from each other. Here, both the first value and the second value are non-negative values, and one or both of the first value and the second value are positive values. Therefore, switching between the projection state and the non-projection state corresponds to an example of intensity modulation.

[0020] The control of the shooting system may be of any type, for example, electrical control such as control of an image sensor or control of an electronic shutter, mechanical control such as control of a mechanical shutter or control of a rotary shutter, or a combination of electrical control and mechanical control.

[0021] The embodiment relates to a technology for generating a series of digital images (called Scheimpflug images) by photographing the test eye multiple times while moving an illumination system and a photographing system that satisfy the Scheimpflug conditions, and in order to collect this series of Scheimpflug images, at least one of the illumination system and the photographing system is configured to be controlled so that the projection time of illumination light onto the test eye is shorter than the exposure time of the photographing system (image sensor).

[0022] When both the projection time of the illumination light and the exposure time of the image sensor are variable, the embodiment may be configured to make the projection time shorter than a fixed or preset exposure time by controlling only the illumination system, or may be configured to make the exposure time longer than a fixed or preset projection time by controlling only the imaging system, or may be configured to make the projection time shorter than the exposure time (in other words, make the exposure time longer than the projection time) by controlling both the illumination system and the imaging system.

[0023] In the case where the projection time of the illumination light is variable and the exposure time of the image sensor is fixed or preset, the embodiment is configured to make the projection time shorter than the exposure time by controlling the illumination system. Conversely, in the case where the projection time of the illumination light is fixed or preset and the exposure time of the image sensor is variable, the embodiment is configured to make the exposure time longer than the projection time by controlling the imaging system.

[0024] The embodiment is configured to control the length (projection time) of the period (projection period) during which illumination light is projected onto the subject's eye to be shorter than the length (exposure time) of the period (exposure period) during which the imaging element can receive light, and also to control the length so that at least a portion of the projection period overlaps with at least a portion of the exposure period.

[0025] An example of an operation mode based on such control is shown in Figure 2. The operation mode in Figure 2 shows coordinated control (synchronous control) of light emission from the light source, exposure of the camera, and scan position, similar to the conventional operation mode shown in Figure 1, but unlike the operation mode shown in Figure 1 in which illumination light is emitted continuously, illumination light is output intermittently (pulse emission). The sequence of illumination light emission (projection) shown in Figure 2 (multiple emissions arranged in chronological order) and the sequence of camera exposure (multiple exposures arranged in chronological order) are synchronized with each other.

[0026] The period of each light emission in the illumination light sequence corresponds to a projection period, and its length corresponds to the projection time. Similarly, the period of each exposure in the exposure sequence corresponds to an exposure period, and its length corresponds to the exposure time. The projection time may be constant or non-constant. The exposure time may be constant or non-constant.

[0027] 2, for each exposure period in the exposure sequence, a portion of that exposure period coincides with one projection period. That is, for each exposure period in the exposure sequence, the length of the projection period (projection time) is shorter than the length of that exposure period (exposure time), and a portion of this exposure period overlaps with the entire projection period.

[0028] By adopting the operational mode shown in FIG. 2, exposure (light reception by the image sensor and charge accumulation) is substantially performed only during a projection period shorter than each exposure period in the exposure sequence. This reduces image blurring caused by movement of the scan position or eye movement during exposure, compared to conventional ophthalmic imaging techniques. This makes it possible to provide images of higher quality than conventional ophthalmic imaging techniques. It also makes it possible to perform image analysis, such as corneal detection and cell detection, with higher quality than conventional ophthalmic image analysis techniques. Furthermore, since the time during which illumination light is projected onto the subject's eye can be shortened, there is no significant strain on the subject. Furthermore, because the optical system does not repeatedly start and stop suddenly, vibrations caused by this do not occur during scanning, which does not adversely affect the quality of the image.

[0029] Several exemplary aspects of the embodiment outlined above will be described. The exemplary aspects described below include an ophthalmic apparatus, a method for controlling the ophthalmic apparatus, a program, and a recording medium, but the aspects of the embodiment are not limited to these.

[0030] <Ophthalmological equipment> 1 provides some exemplary aspects of an ophthalmic device according to an embodiment.

[0031] The configuration of an ophthalmic apparatus according to one aspect of the embodiment is shown in Fig. 3. The ophthalmic apparatus 1000 of this example includes an image acquisition unit 1010 and a control unit 1020.

[0032] The image acquisition unit 1010 acquires Scheimpflug images of the subject's eye. The image acquisition unit 1010 includes an optical system that satisfies the Scheimpflug condition, which projects illumination light onto the subject's eye and captures images with an image sensor. The image acquisition unit 1010 acquires a series of Scheimpflug images while changing the illumination light projection position and the image capture position. That is, the image acquisition unit 1010 acquires a series of Scheimpflug images while moving the scan position (illumination light projection position and image capture position). In other words, the image acquisition unit 1010 acquires a series of Scheimpflug images by scanning the subject's eye while maintaining the optical system in a state that satisfies the Scheimpflug condition. Several configuration examples of such an image acquisition unit 1010 are shown in FIGS. 4A to 4C.

[0033] The image acquisition unit 1010A shown in FIG. 4A will be described. The image acquisition unit 1010A includes an illumination system 1011 and an imaging system 1012. The illumination system 1011 is configured to project illumination light onto the subject's eye. The illumination light may be slit light. The slit light is light whose beam cross-sectional shape is defined, for example, by a mechanism for forming a slit (narrow slit) of a predetermined shape or by other mechanisms. The imaging system 1012 is configured to capture an image of the subject's eye, and includes an image sensor 1013 and an optical system (not shown) that guides light from the subject's eye to the image sensor 1013.

[0034] The illumination system 1011 and the imaging system 1012 are configured to satisfy the Scheimpflug condition and function as a Scheimpflug camera. More specifically, the illumination system 1011 and the imaging system 1012 are configured so that a plane passing through the optical axis of the illumination system 1011 (a plane including the object plane), the principal plane of the imaging system 1012, and the imaging plane of the image sensor 1013 intersect on the same straight line. By using the illumination system 1011 and the imaging system 1012 that satisfy the Scheimpflug condition, it is possible to perform imaging with the imaging system 1012 focused on all positions within the object plane (all positions in the direction along the optical axis of the illumination system 1011).

[0035] 4A is configured to collect a series of Scheimpflug images by scanning a three-dimensional region of the subject's eye with illumination light (e.g., a slit of light). The image collection unit 1010A in this example is configured to collect the series of Scheimpflug images by repeatedly capturing images while moving the projection position of the slit of light relative to the three-dimensional region of the subject's eye.

[0036] In some exemplary embodiments, the image acquisition unit 1010A may be configured to scan a three-dimensional region of the subject's eye by translating the slit light in a direction perpendicular to the longitudinal direction of the slit light, which differs from conventional anterior segment imaging devices that scan the anterior segment by rotating the slit light.

[0037] The longitudinal direction of the slit light is the longitudinal direction of the beam cross section of the slit light at the projection position on the subject's eye, in other words, the longitudinal direction of the image of the slit light formed on the subject's eye, and may be approximately aligned with the direction along the subject's body axis (body axis direction). The size of the slit light in the longitudinal direction may be arbitrary. In some exemplary embodiments, the size of the slit light in the longitudinal direction may be equal to or greater than the corneal diameter in the body axis direction of the subject, and the distance of translation of the slit light may be equal to or greater than the corneal diameter in a direction perpendicular to the body axis direction of the subject. Note that the conditions (length, width, etc.) of the slit light may be changeable depending on the imaging mode applied.

[0038] The series of Scheimpflug images collected by the image collection unit 1010A in this example is a group of images (group of frames) collected continuously over time (in a chronological order), but since they are a group of images collected sequentially from multiple different positions in the three-dimensional area of ​​the test eye, they are a group of images that are spatially distributed, unlike general moving images.

[0039] In the image acquisition unit 1010A of this example, the illumination system 1011 projects slit light onto a three-dimensional region of the subject's eye, and the imaging system 1012 images the three-dimensional region of the subject's eye onto which the slit light from the illumination system 1011 is projected. Furthermore, the image acquisition unit 1010A of this example collects a series of Scheimpflug images representing the three-dimensional region of the subject's eye by scanning the subject's eye while maintaining a state in which the illumination system 1011 and the imaging system 1012 satisfy the Scheimpflug condition.

[0040] In order to move the scan position relative to the subject's eye in the scan performed by the image acquisition unit 1010, the image acquisition unit 1010A of this example may include, for example, a mechanism (moving mechanism) for moving the illumination system 1011 and the imaging system 1012, or other mechanisms. Mechanisms other than the moving mechanism include, for example, a mechanism for moving the illumination position by deflecting illumination light (slit light) (illumination scanner, movable illumination mirror), and a mechanism for moving the imaging position by deflecting light from the subject's eye toward the imaging element (imaging scanner, movable imaging mirror).

[0041] The control unit 1020 of the ophthalmic apparatus 1000 to which the image acquisition unit 1010A of this example is applied controls the image acquisition unit 1010A so that the projection time of the slit light onto the subject's eye is shorter than the exposure time of the image sensor 1013, and so that at least a part of the projection period of the slit light onto the subject's eye overlaps at least a part of the exposure period of the image sensor 1013. The control of the image acquisition unit 1010A includes either or both of the control of the illumination system 1011 and the control of the imaging system 1012.

[0042] Furthermore, the control unit 1020 of the ophthalmic apparatus 1000 to which the image acquisition unit 1010A of this example is applied may be configured to execute control of the imaging system 1012 to repeatedly expose the image sensor 1013 in controlling the image acquisition unit 1010A (imaging system 1012) to acquire a series of Scheimpflug images. Each exposure in this repeated exposure corresponds to one Scheimpflug image.

[0043] The operational mode of Fig. 2 described above illustrates an example of repeated exposure ("camera exposure"). Such repeated exposure results in the collection of a series of Scheimpflug images. While the operational mode of Fig. 2 uses multiple pulsed light as illumination light, the illumination light is not limited to this.

[0044] For example, by using continuous light as illumination light, providing a shutter in the imaging system 1012, and rapidly switching the shutter open and closed by the control unit 1020, repeated exposure can be achieved. The advantages of this example include low cost and easy control. Furthermore, when imaging intraocular floaters, a light source with a relatively high light output is used, but it is difficult to perform pulse control (pulse driving) of a high-intensity light source with high precision. Therefore, this example is considered advantageous. However, since continuous light is used as illumination light, there is a disadvantage in that it places a heavy burden on the subject. Those who intend to apply this example to a specific patient should consider comparing the advantages and disadvantages of this example and taking into account the necessity of the examination, etc., before deciding whether to apply this example.

[0045] The control unit 1020 may be configured to control the illumination system 1011 to modulate the intensity of the illumination light in parallel with the control of the imaging system 1012 to repeatedly expose the image sensor 1013 .

[0046] In some examples, the control unit 1020 controls the illumination system 1011 to repeatedly turn the illumination light on and off (projection / non-projection) (i.e., so that the illumination light is intermittently projected onto the test eye), as in the operating mode of Figure 2.

[0047] In some other examples, the control unit 1020 controls the illumination system 1011 so as to switch the intensity of the illumination light between two different values, both of which are positive values. In still other examples, the control unit 1020 may control the illumination system 1011 so as to switch the intensity of the illumination light between three or more different values.

[0048] The control unit 1020 may be configured to execute synchronous control of the illumination system 1011 and the imaging system 1012 in controlling the image acquisition unit 1010 to acquire a series of Scheimpflug images. That is, the control unit 1020 synchronously controls the illumination system 1011 and the imaging system 1012, thereby causing the image acquisition unit 1010 to acquire a series of Scheimpflug images. This synchronous control may synchronize the control of the imaging system 1012 with the control of the illumination system 1011, or may synchronize the control of the illumination system 1011 with the control of the imaging system 1012, or may synchronize the control of the illumination system 1011 and the control of the imaging system 1012 for a specific operation. This specific operation may be, for example, the operation of a movement mechanism that moves the illumination system 1011 and the imaging system 1012.

[0049] Such synchronous control enables highly accurate control for collecting a series of Scheimpflug images. That is, such synchronous control enables highly accurate control for making the projection time of illumination light onto the subject's eye shorter than the exposure time of the image sensor 1013, while also enabling highly accurate control for overlapping at least a portion of the projection period of illumination light onto the subject's eye with at least a portion of the exposure period of the image sensor 1013. For example, such synchronous control enables highly accurate scanning such as the operation mode shown in FIG. 2.

[0050] When performing synchronous control between the illumination system 1011 and the imaging system 1012, the control unit 1020 may perform synchronous control between the illumination system 1011 and the imaging system 1012 so that the projection time of the illumination light is shorter than the exposure time of the imaging element 1013, and so that the total time of the projection time and non-projection time of the illumination light is equal to the total time of the exposure time and non-exposure time of the imaging element 1013.

[0051] Here, the illumination light projection time is the length of the period (projection period) during which the illumination light is projected onto the subject's eye, and may be, for example, the length of the period during which a pulse-controlled illumination light source is on (the length of one on period), or the length of the period during which a shutter that intermittently passes continuous light from the illumination light source is open (the length of one period during which the shutter is open). Furthermore, the illumination light non-projection time is the length of the period during which the illumination light is not projected onto the subject's eye (non-projection period), and may be, for example, the length of the period during which a pulse-controlled illumination light source is off (the length of one off period), or the length of the period during which a shutter that intermittently passes continuous light from the illumination light source is closed (the length of one period during which the shutter is closed).

[0052] The exposure time of the image sensor 1013 is the length of a period (exposure period) during which the image sensor 1013 can receive light, and the non-exposure time of the image sensor 1013 is the length of a period (non-exposure period) during which the image sensor 1013 cannot receive light. During the non-exposure period, the image sensor 1013 performs operations such as transferring charges and waiting for exposure.

[0053] In the operation mode of Fig. 2, one upper side (top side) in the rectangular pulse train representing the operation mode of light emission of the light source represents a projection period, and its length represents the projection time. Furthermore, one lower side (bottom side) in the rectangular pulse train representing the operation mode of light emission of the light source represents a non-projection period, and its length represents the non-projection time. Furthermore, one upper side (top side) in the rectangular pulse train representing the operation mode of exposure of the camera represents an exposure period, and its length represents the exposure time. Furthermore, one lower side (bottom side) in the rectangular pulse train representing the operation mode of exposure of the camera represents a non-exposure period, and its length represents the non-exposure time.

[0054] By performing synchronization control between the illumination system 1011 and the imaging system 1012 so that the projection time is shorter than the exposure time and so that the total time of the projection time and non-projection time is equal to the total time of the exposure time and non-exposure time, it becomes possible to perform an operation mode such as the example shown in FIG. 2 (i.e., a scan in which the projection time is shorter than the exposure time and at least a part of the projection period overlaps with at least a part of the exposure period) with high precision.

[0055] The control unit 1020 may be configured to execute control of the illumination system 1011 to change the projection time of illumination light onto the subject's eye. This projection time change control includes, for example, light source control that changes the frequency of a control pulse signal applied to a pulse-controlled illumination light source, or shutter control that changes the opening and closing frequency of a shutter that intermittently passes continuous light from the illumination light source. By employing projection time change control, it becomes possible to set the projection time of illumination light according to various conditions related to the subject's eye, the type of examination, the camera (image sensor 1013), etc.

[0056] The control unit 1020 may be configured to execute projection time change control based on the speed (rate) of the scan executed by the image collecting unit 1010. That is, the control unit 1020 may be configured to execute projection time change control based on the moving speed of the scan position relative to the subject's eye in the scan executed by the image collecting unit 1010.

[0057] In some exemplary aspects, the control unit 1020 may be configured to change the projection time of the illumination light by controlling the illumination system 1011 based on the speed of the illumination system 1011 and the imaging system 1012 moved by the movement mechanism described above. In other words, the control unit 1020 in some exemplary aspects may be configured to execute projection time change control depending on the content of control over the movement mechanism.

[0058] In addition, in some exemplary embodiments, the projection time change control may be configured to be performed depending on the content of control over mechanisms other than the moving mechanism (such as the lighting scanner, movable lighting mirror, photographing scanner, and movable photographing mirror mentioned above).

[0059] By adopting the control of changing the projection time based on the scan speed, it becomes possible to set the projection time of the illumination light that is suited to the scan speed and perform Scheimpflug image collection.

[0060] The control unit 1020 may be configured to execute projection time change control according to an applied imaging mode. For example, the control unit 1020 may be configured to determine the movement speed of the scan position relative to the subject's eye (for example, the speed of the illumination system 1011 and the imaging system 1012 moved by a movement mechanism) based on the applied imaging mode.

[0061] The imaging mode is a preset operation content for each type of examination (photographing, scanning). The number of imaging modes prepared is arbitrary (one or more), but in this example, two or more imaging modes are prepared. The user or the control unit 1020 selects an imaging mode to be applied to the subject's eye from the two or more prepared imaging modes. The number of imaging modes selected may be arbitrary (one or more). When two or more imaging modes are selected, for example, two or more scans corresponding to the two or more selected imaging modes are sequentially applied to the subject's eye.

[0062] The ophthalmic apparatus 1000 of this example stores imaging mode information in which a plurality of imaging modes are recorded, for example, in a storage device in the control unit 1020. Alternatively, the ophthalmic apparatus 1000 of this example can refer to imaging mode information stored in an external device.

[0063] An example of the imaging mode information is shown in Fig. 5. In this example, the imaging mode information 1200 includes imaging modes such as an anterior eye shape imaging mode, an inflammatory cell imaging mode, an intraocular lens (IOL) imaging mode, and an intraocular contact lens (ICL) imaging mode. Note that the types of imaging modes are not limited to these.

[0064] Furthermore, the imaging mode information 1200 of this example includes illumination control pulse height, illumination control pulse width, and scan range as types of imaging operation, i.e., types of parameters related to scanning. The illumination control pulse height is the height of each pulse in a control pulse signal applied to a pulse-controlled illumination light source. The illumination control pulse width is the width of each pulse in a control pulse signal applied to a pulse-controlled illumination light source. The scan range is the distance the illumination light moves during scanning. The imaging mode information 1200 of this example is applied when a pulse-controlled illumination light source (an illumination light source that emits pulsed light) is used, but it will be apparent to those skilled in the art that similar parameters can be provided in other cases as well.

[0065] The anterior segment topography mode is an imaging mode used to observe and analyze the shapes of anterior segment tissues such as the cornea, iridocorneal angle, iris, and lens. In the anterior segment topography mode, the scan range is set relatively large (20 mm) to collect images over a wide area of ​​the anterior segment. In addition, taking into account the large light reflection from the cornea and the fast scan speed, the illumination control pulse height is set relatively large (0.1 mW) and the illumination control pulse width is set relatively small (10 ms).

[0066] The inflammatory cell imaging mode is used to observe and analyze inflammatory cells (anterior chamber cells) present in the anterior chamber. In inflammatory cell imaging mode, the illumination control pulse height is set relatively high (1 milliwatt) and the illumination control pulse width is set sufficiently small (5 milliseconds), taking into account that light reflection from inflammatory cells is small and that inflammatory cells float and move within the anterior chamber. The scan range is also set relatively small (2 millimeters).

[0067] The IOL imaging mode is used to observe and analyze intraocular lenses implanted in the subject's eye. In IOL imaging mode, the illumination control pulse height is set relatively low (0.1 milliwatts) and the illumination control pulse width is set medium (20 milliseconds) to accurately determine the position of the intraocular lens relative to the pupil and cornea, and to account for the relatively small amount of light reflection from the intraocular lens. In addition, in IOL imaging mode, the intraocular lens, which is located behind the iris, is imaged through the pupil, so the scan range is set relatively large (15 mm) to adequately cover the pupil.

[0068] The ICL imaging mode is used to observe and analyze an intraocular contact lens implanted in the subject's eye. In ICL imaging mode, the hole formed in the center of the intraocular contact lens must be imaged, a narrow scan interval is required to detect its position with high accuracy, and the intraocular contact lens itself does not move. Taking these factors into consideration, the illumination control pulse width is set large (30 milliseconds), and the illumination control pulse height is set relatively small (0.1 milliwatts). The scan range is set to a medium value (10 mm).

[0069] By referring to such shooting mode information 1200, the control unit 1020 can identify parameter values ​​(illumination control pulse height value, illumination control pulse width value, scan range value) corresponding to the selected shooting mode.

[0070] The control unit 1020 may be configured to determine the scan speed depending on the imaging mode. For example, the control unit 1020 may be configured to determine the scan speed based on an imaging mode selected from two or more preset imaging modes. In some exemplary embodiments, the control unit 1020 may be configured to determine the scan speed based on parameter values ​​identified with reference to the imaging mode information 1200.

[0071] An example of the process for determining the scan speed will be described. As a premise, there is a limit to the time a person can refrain from blinking (the time they can keep their eyes open). This limit time is defined as "T0." The limit time T0 may be a default value or may be a value set for each subject (eye to be examined). Furthermore, the time taken for scanning (scan time), that is, the time from the start of scanning to the end of scanning, is defined as "ST." The scan time ST is set to the limit time T0 or less (ST≦T0). The imaging mode information 1200 sets a scan range for each imaging mode. The scan range corresponding to any imaging mode is defined as "SD." The scan speed corresponding to this imaging mode is defined as "SS." In this case, the scan speed SS can be determined by the following formula: SS=SD / ST.

[0072] Based on these assumptions and calculations, the scan speed corresponding to each shooting mode can be determined. The shooting mode information may include the value of the scan speed determined in this manner. The shooting mode information 1210 shown in Fig. 6 is obtained by adding the scan speed to the shooting mode information 1200 in Fig. 5. The scan speed corresponding to each shooting mode in the shooting mode information 1210 is calculated based on the assumption that the scan time ST is 2 seconds.

[0073] The assumptions and calculation methods for determining the value of the scan speed are not limited to the examples described above. Furthermore, the types of parameters whose values ​​are determined by the control unit 1020 are not limited to the scan speed.

[0074] In some exemplary embodiments, the number of Scheimpflug images (image number) to be collected in a scan can be a precondition. In other words, predetermined parameters can be set so that a predetermined number of Scheimpflug images are collected. The parameters set based on the number of images can be, for example, one or more of an illumination control pulse width, a pulse interval (the interval between two adjacent pulses), a scan range, a scan speed, and other parameters.

[0075] The control unit 1020 may be configured to execute control of the imaging system 1012 to change the exposure time of the image sensor 1013. This exposure time change control includes, for example, image sensor control that changes the frequency of a control pulse signal applied to the image sensor 1013, or shutter control that changes the opening and closing frequency of a shutter that intermittently passes light guided to the image sensor 1013. By employing exposure time change control, it becomes possible to set the exposure time of the image sensor 1013 according to various conditions related to the subject's eye, the type of examination, the illumination light, etc.

[0076] The control unit 1020 may be configured to execute exposure time change control based on the speed of the scan performed by the image acquisition unit 1010. That is, the control unit 1020 may be configured to execute exposure time change control based on the movement speed of the scan position relative to the subject's eye in the scan performed by the image acquisition unit 1010.

[0077] In some exemplary aspects, the control unit 1020 may be configured to control the imaging system 1012 based on the speed of the illumination system 1011 and the imaging system 1012 moved by the moving mechanism described above, to change the exposure time of the image sensor 1013. In other words, the control unit 1020 in some exemplary aspects may be configured to execute exposure time change control depending on the content of control over the moving mechanism.

[0078] In addition, in some exemplary embodiments, the exposure time change control may be configured to be performed depending on the content of control over mechanisms other than the moving mechanism (such as the aforementioned lighting scanner, movable lighting mirror, photographing scanner, movable photographing mirror, etc.).

[0079] By adopting exposure time change control based on the scan speed, it becomes possible to set the exposure time of the image sensor 1013 that is suited to the scan speed and perform Scheimpflug image acquisition.

[0080] The control unit 1020 may be configured to execute exposure time change control according to an applied imaging mode. For example, the control unit 1020 may be configured to determine the movement speed of the scan position relative to the subject's eye (for example, the speed of the illumination system 1011 and the imaging system 1012 moved by a movement mechanism) based on the applied imaging mode.

[0081] As with the imaging modes related to the projection time change control, the imaging modes related to the exposure time change control are preset with operation details for each type of examination (imaging, scanning), and any number of imaging modes greater than or equal to one are prepared. When two or more imaging modes are prepared, the user or the control unit 1020 selects an imaging mode to be applied to the subject's eye from among these imaging modes. The number of imaging modes selected may be any number greater than or equal to one. When two or more imaging modes are selected, for example, two or more scans corresponding to the two or more selected imaging modes are sequentially applied to the subject's eye.

[0082] As in the case of projection time change control, the ophthalmic device 1000 capable of executing exposure time change control stores shooting mode information in which multiple shooting modes are recorded, for example, in a storage device within the control unit 1020, or can refer to shooting mode information stored in an external device.

[0083] As in the case of projection time change control, the control unit 1020 of the ophthalmic device 1000 capable of executing exposure time change control may be configured to identify parameter values ​​(illumination control pulse height value, illumination control pulse width value, scan range value, scan speed value) corresponding to the selected shooting mode by referring to the shooting mode information 1200 in Figure 5 and / or the shooting mode information 1210 in Figure 6.

[0084] As in the case of the projection time change control, the control unit 1020 of the ophthalmologic apparatus 1000 capable of executing the exposure time change control may be configured to calculate the scan speed based on the shooting mode. The method of calculating the scan speed may be the same as in the case of the projection time change control.

[0085] As in the case of projection time change control, the control unit 1020 of the ophthalmologic apparatus 1000 capable of executing exposure time change control may be configured to execute exposure time change control with any parameter such as the number of images as a prerequisite.

[0086] The projection time change control and the exposure time change control may be executed separately, or may be executed at least partially in cooperation (at least partially linked). As an example of the latter, it is possible to execute the exposure time change control using any information (final generated information, intermediate generated information) obtained in the projection time change control, and conversely, it is also possible to execute the projection time change control using any information (final generated information, intermediate generated information) obtained in the exposure time change control. It is also possible to execute the projection time change control and the exposure time change control at least partially in parallel.

[0087] In some exemplary embodiments, the imaging system 1012 of the image acquisition unit 1010A of Figure 4A may include two or more imaging systems, an example of which is shown in Figure 4B. Unless otherwise stated below, anything related to the image acquisition unit 1010A of Figure 4A can be combined with the image acquisition unit 1010B of this example.

[0088] The imaging system 1012A of the image collecting unit 1010B includes a first imaging system 1014 and a second imaging system 1015. The first imaging system 1014 and the second imaging system 1015 are arranged so as to capture images of the subject's eye from different directions.

[0089] The image collecting unit 1010B of this example may be configured to capture images of a three-dimensional region of the subject's eye from two different directions using the first imaging system 1014 and the second imaging system 1015 in a scan (slit scan) using a slit light for collecting a series of Scheimpflug images. For example, when the image collecting unit 1010B is configured so that the longitudinal direction of the beam cross section of the slit light at the incident position on the subject's eye is the up-down direction (Y direction) and the moving direction of the slit light is the horizontal direction (left-right direction, X direction), the first imaging system 1014 and the second imaging system 1015 may be arranged so that one captures an image of the subject's eye from a left oblique direction and the other captures an image of the subject's eye from a right oblique direction.

[0090] A series of Scheimpflug images collected by the first imaging system 1014 is called a first Scheimpflug image group, and a series of Scheimpflug images collected by the second imaging system 1015 is called a second Scheimpflug image group. The series of Scheimpflug images collected by such an image collecting unit 1010B includes a first Scheimpflug image group and a second Scheimpflug image group.

[0091] Note that even when one Scheimpflug image (first Scheimpflug image) is acquired by the first imaging system 1014 and one Scheimpflug image (second Scheimpflug image) is acquired by the second imaging system 1015 without performing slit scanning, for convenience of terminology, the one Scheimpflug image acquired by the first imaging system 1014 may be referred to as a first Scheimpflug image group, and the one Scheimpflug image acquired by the second imaging system 1015 may be referred to as a second Scheimpflug image group. Thus, in the present disclosure, the term "group" may be used not only when multiple elements are included, but also when only one element is included.

[0092] When the image collecting unit 1010B of this example performs slit scanning, the first imaging system 1014 and the second imaging system 1015 capture images of the subject's eye in parallel. That is, the image collecting unit 1010B performs imaging by the first imaging system 1014 and the second imaging system 1015 in parallel while moving the projection position of the slit light with respect to the three-dimensional region of the subject's eye.

[0093] Furthermore, the image collecting unit 1010B of this example may be configured to perform the photographing (collection of Scheimpflug images) by the first photographing system 1014 and the photographing (collection of Scheimpflug images) by the second photographing system 1015 in synchronization with each other. By referring to this synchronization relationship, the first Scheimpflug image group and the second Scheimpflug image group can be easily associated with each other without using image processing or the like. This association is performed, for example, so as to associate Scheimpflug images that are acquired at a small difference from each other.

[0094] In such an embodiment, the ophthalmic device 1000 can reconstruct a series of Scheimpflug images corresponding to the slit scan from the first Scheimpflug image group and the second Scheimpflug image group by referring to the mutual synchronization relationship between the image capture by the first image capture system 1014 and the image capture by the second image capture system 1015.

[0095] Fig. 4C shows an example of the configuration of an ophthalmic apparatus 1000 to which the image acquisition unit 1010B shown in Fig. 4B is applied. In the image acquisition unit 1010B of this example, the optical axis of the first imaging system 1014 and the optical axis of the second imaging system 1015 are arranged to be inclined in opposite directions relative to the optical axis of the illumination system 1011.

[0096] A series of Scheimpflug images collected by the imaging system 1012A of this example are sent directly or indirectly to the data processing device 1100. The data processing device 1100 is configured to be able to perform data processing such as image processing, and may be a component of the ophthalmic apparatus 1000 of this example, or may be provided in a computer or system capable of data communication with the ophthalmic apparatus 1000 of this example.

[0097] The data processing device 1100 includes an image selection unit 1110. The image selection unit 1110 is configured to select an image from among a first group of Scheimpflug images acquired by the first imaging system 1014 and a second group of Scheimpflug images acquired by the second imaging system 1015. For example, the image selection unit 1110 may be configured to select one of the first Scheimpflug image acquired by the first imaging system 1014 and the second Scheimpflug image acquired by the second imaging system 1015.

[0098] The image selection unit 1110 is configured to select, from the first Scheimpflug image group and the second Scheimpflug image group, a new series of Scheimpflug images corresponding to the slit scans from which the first Scheimpflug image group and the second Scheimpflug image group were collected, based on a correspondence relationship between the first Scheimpflug image group and the second Scheimpflug image group, respectively, collected based on synchronization between the image capturing by the first imaging system 1014 and the image capturing by the second imaging system 1015. In other words, the image selection unit 1110 is configured to reconstruct a series of Scheimpflug images from the first Scheimpflug image group and the second Scheimpflug image group collected by the first imaging system 1014 and the second imaging system 1015, respectively.

[0099] Any method may be used for the image selection process executed by the image selection unit 1110. For example, the method for the image selection process may be determined and / or selected based on predetermined conditions or predetermined parameters, such as the configuration and / or arrangement of the first imaging system 1014 and the second imaging system 1015, or the purpose and / or use of image selection.

[0100] The image acquisition unit 1010B synchronously executes imaging by the first imaging system 1014 and imaging by the second imaging system 1015. As described above, the optical axis of the first imaging system 1014 and the optical axis of the second imaging system 1015 are arranged to be inclined in opposite directions relative to the optical axis of the illumination system 1011. For example, the optical axis of the first imaging system 1014 is arranged to be inclined leftward relative to the optical axis of the illumination system 1011, and the optical axis of the second imaging system 1015 is arranged to be inclined rightward relative to the optical axis of the illumination system 1011. The first imaging system 1014 and the second imaging system 1015 arranged in this manner may be referred to as the left imaging system and the right imaging system, respectively.

[0101] The tilt angle of the optical axis of the first imaging system 1014 relative to the optical axis of the illumination system 1011 and the tilt angle of the optical axis of the second imaging system 1015 relative to the optical axis of the illumination system 1011 may be equal to or different from each other. Furthermore, these tilt angles may be fixed or variable.

[0102] In this example, for example, the illumination system 1011 is constructed and arranged to project slit light, the longitudinal direction of the cross section of which is oriented in the Y direction, onto the subject's eye from the front direction, and the image acquisition unit 1010B applies a slit scan to a three-dimensional region of the anterior segment of the subject's eye by moving the illumination system 1011, the first imaging system 1014, and the second imaging system 1015 together in the X direction.

[0103] The image selection unit 1110 of this example selects a new set of Scheimpflug images corresponding to the slit scan from which the first and second Scheimpflug image groups were acquired by selecting a plurality of Scheimpflug images that do not contain artifacts from the first and second Scheimpflug image groups based on the correspondence between the first and second Scheimpflug image groups acquired by the first imaging system 1014 and the second Scheimpflug image group acquired by the second imaging system 1015. This artifact may be any type of artifact. When performing an anterior segment scan as in this example, this artifact may be an artifact caused by corneal reflection (referred to as a corneal reflection artifact). Below, several examples of the processing performed by the image selection unit 1110 will be described.

[0104] The projection position (Scheimpflug image, frame) of the slit light at which the corneal reflection artifact occurs differs between the left and right imaging systems. For example, as in this example, when slit scanning is performed by projecting slit light whose cross-section longitudinal direction is oriented in the Y direction onto the subject's eye from the front direction while integrally moving the illumination system 1011, the first imaging system 1014, and the second imaging system 1015 in the X direction, the corneal reflection light of the slit light is likely to enter the left imaging system when the slit light is projected to a position to the left of the corneal apex, and is likely to enter the right imaging system when the slit light is projected to a position to the right of the corneal apex.

[0105] Taking these circumstances into consideration, the image selection unit 1110 in some exemplary embodiments first identifies a Scheimpflug image (first corneal apex image) corresponding to the corneal apex from the first Scheimpflug image group collected by the first imaging system 1014 as the left imaging system, and also identifies a Scheimpflug image (second corneal apex image) corresponding to the corneal apex from the second Scheimpflug image group collected by the second imaging system 1015 as the right imaging system.

[0106] In some exemplary embodiments, the process of identifying the corneal apex image may include a process of detecting an image corresponding to the corneal surface from each Scheimpflug image included in the first Scheimpflug image group, a process of identifying a pixel closest to the ophthalmic apparatus 1000 of this example based on the Z coordinates of pixels in these detected images, and a process of setting the Scheimpflug image including the identified pixel as the first corneal apex image. Setting the second corneal apex image may be performed in the same manner.

[0107] Next, the image selector 1110 selects a Scheimpflug image group located to the right of the first corneal apex image from the first Scheimpflug image group, and selects a Scheimpflug image group located to the left of the second corneal apex image from the second Scheimpflug image group, thereby forming a series of Scheimpflug images consisting of the two selected Scheimpflug image groups (and the first corneal apex image and / or the second corneal apex image). This results in a series of Scheimpflug images that cover the three-dimensional region of the anterior segment to which the slit scan has been applied and that are (highly likely to) not contain corneal reflection artifacts.

[0108] Another example of the process for identifying a corneal apex image will be described. In some exemplary embodiments, the image selection unit 1110 determines whether a corneal reflection artifact is included in either of two images acquired substantially simultaneously by the first imaging system 1014 (e.g., the left imaging system) and the second imaging system 1015 (e.g., the right imaging system). This corneal reflection artifact determination process includes a predetermined image analysis, such as threshold processing on brightness information assigned to pixels. Note that the process for determining whether the two images were acquired substantially simultaneously can be performed based on the synchronization between the imaging by the first imaging system 1014 and the imaging by the second imaging system 1015.

[0109] The threshold processing used in the artifact determination process is performed, for example, to identify pixels assigned a brightness value exceeding a preset threshold. Typically, the threshold may be set higher than the brightness value of the slit light image (the slit light projection area) in the image. As a result, the image selection unit 1110 is configured to determine an image brighter than the slit light image as an artifact without determining the slit light image as an artifact. Considering that an image brighter than the slit light image in a Scheimpflug image is likely to be an image caused by specular reflection from the cornea, it can be considered that an artifact detected by the image selection unit 1110 configured in this manner is likely to be a corneal reflection artifact.

[0110] For artifact determination, the image selector 1110 may perform any image analysis other than thresholding, such as pattern recognition, segmentation, edge detection, etc. In general, any information processing technique, such as image analysis, image processing, machine learning, artificial intelligence, cognitive computing, etc., can be applied to artifact determination.

[0111] When it is determined as a result of the artifact determination that one of the two images acquired substantially simultaneously by the first imaging system 1014 and the second imaging system 1015 contains an artifact, the image selection unit 1110 selects the other image. In other words, the image selection unit 1110 selects the other image, which is not the image determined to contain the artifact, from the two images acquired substantially simultaneously by the first imaging system 1014 and the second imaging system 1015.

[0112] Assuming that both images are determined to contain artifacts, the image selection unit 1110 may be configured to, for example, perform a process of evaluating the magnitude of the adverse effect that the artifacts have on observation or diagnosis, and a process of selecting the image with the smaller adverse effect. This evaluation process may be performed based on, for example, one or more conditions of the size, intensity, shape, and position of the artifact. Typically, artifacts with large size, high intensity, and artifacts located in or near a region of interest such as a slit light image are evaluated as having a large adverse effect.

[0113] In addition, in cases where artifacts are included in both images, the artifact removal disclosed in Patent Document 3 (JP 2019-213733 A) may be applied.

[0114] By providing the image selection unit 1110 as described above, it is possible to provide an image of a three-dimensional region of the subject's eye that does not contain artifacts that interfere with observation, analysis, or diagnosis. Furthermore, it is possible to provide an image of a three-dimensional region of the subject's eye that does not contain artifacts to subsequent processing. For example, it is possible to construct a three-dimensional image or a rendering image of the subject's eye based on a group of images that do not contain artifacts.

[0115] Even when images are obtained by photographing substantially the same position, the dimensions of the depicted predetermined region may differ between the image obtained by the left photographing system and the image obtained by the right photographing system. For example, the thickness of the depicted cornea or the dimensions of inflammatory cells may differ between the left and right images obtained by photographing substantially the same position with the left and right photographing systems, respectively. Even in such cases, the image selection unit 1110 can be used to match the dimensions of the object.

[0116] The configuration of an ophthalmic apparatus according to one aspect of the embodiment is shown in Fig. 7. The ophthalmic apparatus 1500 of this example includes an evaluation processing unit 1030 in addition to an image acquisition unit 1010 and a control unit 1020. The image acquisition unit 1010 of this example may be configured at least in part in the same manner as the image acquisition unit 1010 of the ophthalmic apparatus 1000 of Fig. 3. Furthermore, the control unit 1020 of this example may be configured at least in part in the same manner as the control unit 1020 of the ophthalmic apparatus 1000 of Fig. 3.

[0117] The evaluation processing unit 1030 generates evaluation information of the subject's eye based on the series of Scheimpflug images collected by the image collection unit 1010. The evaluation information includes information generated by evaluating data obtained in a medical examination (clinical examination) of the subject's eye. This medical examination may be any ophthalmic examination that utilizes images of the eye.

[0118] In some exemplary embodiments, the medical examination may be for floaters present in the subject's eye, such as floaters present in the anterior chamber, such as inflammatory cells (anterior chamber cells), proteins (anterior chamber flare), and floaters present in the vitreous, such as vitreous fibers and detached retinal cells.

[0119] Since intraocular floaters are subject to movement (movement), images of intraocular floaters obtained by conventional imaging are often blurred, making it difficult to evaluate them with high quality (accuracy, precision, reproducibility, etc.) In contrast, imaging according to this embodiment can reduce image blur as described above, making it possible to improve the quality of examinations related to intraocular floaters.

[0120] Furthermore, imaging according to this embodiment can also reduce the adverse effects of eye movement, body movement, pulsation, and the like on images. Therefore, it is possible to improve the quality of evaluations of objects other than intraocular floaters. For example, this embodiment can be applied to evaluations of any part of the eyeball, or evaluations of artificial objects (intraocular lenses, intraocular contact lenses, minimally invasive glaucoma surgery (MIGS) devices, etc.).

[0121] The evaluation processing unit 1030 may be configured to execute a process of processing the series of Scheimpflug images collected by the image collection unit 1010 to generate processed image data, and a process of generating evaluation information based on the generated processed image data. For example, the evaluation processing unit 1030 may be configured to execute a process of constructing a three-dimensional image (an example of processed image data) from multiple Scheimpflug images included in the series of Scheimpflug images, and a process of generating evaluation information based on the three-dimensional image. The evaluation processing unit 1030 may also be configured to execute a process of constructing a three-dimensional image from multiple Scheimpflug images included in the series of Scheimpflug images, a process of generating a rendering image (an example of processed image data) from the three-dimensional image, and a process of generating evaluation information based on the rendering image. The evaluation processing unit 1030 may also perform any digital image processing such as correction, editing, and enhancement to process the Scheimpflug images.

[0122] The evaluation information generated in this example may include any information (inflammatory state information) related to the inflammatory state of the subject's eye. For example, the inflammatory state information may include one or more of information on inflammatory cells present in the anterior chamber (anterior chamber cells), information on proteins present in the anterior chamber (anterior chamber flare), information on crystalline lens opacity, information on the onset and progression of disease, and information on disease activity. The inflammatory state information may also include comprehensive information based on two or more pieces of information.

[0123] Information about inflammatory cells includes information on arbitrary parameters such as the density (concentration), number, position, and distribution of inflammatory cells, as well as evaluation information based on information on specified parameters. Evaluation information about inflammatory cells is sometimes called cell evaluation information. Information about anterior chamber flare includes information on arbitrary parameters such as the density, number, position, and distribution of flare, as well as evaluation information based on information on specified parameters. Information about lens opacity includes information on arbitrary parameters such as the density, number, position, and distribution of opacity, as well as evaluation information based on information on specified parameters. Information about the onset and progression of disease includes information on arbitrary parameters such as the presence or absence of onset, the state of onset, the duration of disease, and the state of progression, as well as evaluation information based on information on specified parameters. Information about disease activity includes information on arbitrary parameters such as the state of disease activity, as well as evaluation information based on information on specified parameters.

[0124] In some exemplary embodiments, the evaluation processing unit 1030 may be configured to generate evaluation information based on information generated by the ophthalmic device 1500 as well as information input to the ophthalmic device 1500 from outside (e.g., information acquired by another ophthalmic device, information input by a doctor).

[0125] The information referenced to generate the inflammatory status information exemplified above may be any information and may include, for example, the classification criteria for uveitis diseases proposed by the Standardization of Uveitis Nomenclature (SUN) Working Group ("Standardization of uveitis nomenclature for reporting clinical data. Results of the First International Workshop," American Journal of Ophthalmology, Volume 140, Issue 3, September 2005, Pages 509-516).

[0126] It should be noted that the inflammation state information is not limited to the above example, and the information referenced to generate the inflammation state information is not limited to the above example either.

[0127] In this example, a series of Scheimpflug images are acquired by controlling the image acquisition unit 1010 so that the projection time of illumination light onto the subject's eye is shorter than the exposure time of the image sensor, and so that at least a portion of the projection period of illumination light onto the subject's eye overlaps at least a portion of the exposure period of the image sensor. The region of the subject's eye to which this slit scan is applied includes at least a portion or the entire region to which slit scan is applied to acquire data used to generate inflammation state information. For example, when generating inflammation state information including information on inflammatory cells and / or information on anterior chamber flare, slit scan is applied to a region including at least a portion of the anterior chamber. One example is the inflammatory cell imaging mode described above. When generating inflammation state information including information on lens opacity, slit scan is applied to a region including at least a portion of the lens. The same applies when acquiring two or more Scheimpflug images using a method other than slit scan, or when acquiring only one Scheimpflug image.

[0128] Generally, the region (site) of the subject's eye to which the slit scan is applied includes at least a portion of the anterior segment (e.g., tissues such as the cornea, iris, anterior chamber, angle, ciliary body, zonules of Zinn, lens, nerves, and blood vessels; lesions; treatment scars; and artificial structures such as intraocular lenses, intraocular contact lenses, and MIGS devices) and / or at least a portion of the posterior segment (e.g., tissues such as the vitreous body, retina, choroid, sclera, optic disc, lamina cribrosa, macula, nerves, and blood vessels; lesions; treatment scars; and artificial structures such as retinal prostheses). In some exemplary embodiments, the slit scan may be applied to at least a portion of tissues near the eye, such as the eyelids and meibomian glands. In some exemplary embodiments, the slit scan may be applied to a three-dimensional region including any two or all of at least a portion of the anterior segment, at least a portion of the posterior segment, and at least a portion of tissues near the eye.

[0129] An embodiment capable of generating inflammation state information including cell evaluation information may be configured to perform at least one of the following three processes: (1) segmentation (referred to as first segmentation or anterior chamber segmentation) for identifying an image region corresponding to the anterior chamber of the subject's eye (referred to as an anterior chamber region); (2) segmentation (referred to as second segmentation or cell segmentation) for identifying an image region corresponding to inflammatory cells (referred to as a cell region); (3) a process for generating cell evaluation information (referred to as a cell evaluation information generation process). Some exemplary aspects of such an embodiment are described below.

[0130] In a first exemplary embodiment, the evaluation processing unit 1030 is configured to perform at least one of the following processes: a first segmentation for identifying an anterior chamber region from the Scheimpflug image; a second segmentation for identifying a cellular region from the anterior chamber region identified by the first segmentation; and a cell evaluation information generation process for generating cell evaluation information from the cellular region identified by the second segmentation. The first segmentation of this embodiment may be performed using a neural network trained by machine learning, but is not limited to this. The second segmentation of this embodiment may be performed using a neural network trained by machine learning, but is not limited to this. The cell evaluation information generation process of this embodiment may be performed using a neural network trained by machine learning, but is not limited to this. Details of this embodiment will be described later.

[0131] In a second exemplary embodiment, the evaluation processing unit 1030 is configured to perform at least one of a second segmentation for identifying a cellular region from a Scheimpflug image without performing a first segmentation for identifying an anterior chamber region, and a cell evaluation information generation process for generating cell evaluation information from the cellular region identified by the second segmentation. The second segmentation in this embodiment may be performed using a neural network trained by machine learning, but is not limited to this. The cell evaluation information generation process in this embodiment may be performed using a neural network trained by machine learning, but is not limited to this. Details of this embodiment will be described later.

[0132] In a third exemplary embodiment, the evaluation processing unit 1030 is configured to perform a cell evaluation information generation process for generating cell evaluation information from a Scheimpflug image without performing a first segmentation for identifying an anterior chamber region and a second segmentation for identifying a cellular region. The cell evaluation information generation process of this embodiment may be performed using a neural network trained by machine learning, but is not limited to this. Details of this embodiment will be described later.

[0133] In a fourth exemplary embodiment, the evaluation processing unit 1030 is configured to perform at least one of a first segmentation for identifying an anterior chamber region from a Scheimpflug image and a cell evaluation information generation process for generating cell evaluation information from the anterior chamber region identified by the first segmentation, without performing a second segmentation for identifying a cellular region. The first segmentation in this embodiment may be performed using a neural network trained by machine learning, but is not limited to this. The cell evaluation information generation process in this embodiment may be performed using a neural network trained by machine learning, but is not limited to this. Details of this embodiment will be described later.

[0134] In a fifth exemplary embodiment, the evaluation processing unit 1030 is executed without using a neural network trained by machine learning. This embodiment is configured to execute a first segmentation for analyzing a Scheimpflug image to identify an anterior chamber region, a second segmentation for analyzing the anterior chamber region identified by the first segmentation to identify a cellular region, and a cell evaluation information generation process for generating cell evaluation information based on the cellular region identified by the second segmentation. Details of this embodiment will be described later.

[0135] In a sixth exemplary aspect, the evaluation processing unit 1030 is configured to execute at least one or more of at least a part of the first segmentation, at least a part of the second segmentation, and at least a part of the cell evaluation information generation process using a machine learning-based configuration, and to execute processes other than those executed using the machine learning-based configuration using a non-machine learning-based configuration. This aspect is realized, for example, by partially combining any of the above-described first to fourth exemplary aspects with the fifth exemplary aspect, and therefore a detailed description thereof will be omitted.

[0136] As described above, the first segmentation may be a machine learning-based process or a non-machine learning-based process, or a combination of machine learning and non-machine learning-based processes. The second segmentation may be a machine learning-based process or a non-machine learning-based process, or a combination of machine learning and non-machine learning-based processes. Furthermore, the cell evaluation information generation process may be a machine learning-based process or a non-machine learning-based process, or a combination of machine learning and non-machine learning-based processes.

[0137] The types of data input into the first segmentation, the second segmentation, and the cell evaluation information generation process may all be arbitrary. Below, several examples of possible combinations of multiple processes including the first segmentation, the second segmentation, and the cell evaluation information generation process will be described.

[0138] 8 shows an example of the configuration of the evaluation processing unit 1030. The evaluation processing unit 1030A of this example includes a first segmentation unit 1031, a second segmentation unit 1032, and a cell evaluation information generation processing unit 1033.

[0139] The first segmentation unit 1031 includes a processor that performs first segmentation to identify the anterior chamber region, and is configured to identify the anterior chamber region from the Scheimpflug image acquired by the image acquisition unit 1010.

[0140] When the first segmentation is performed using machine learning, the first segmentation unit 1031 is configured to perform the first segmentation using a pre-constructed inference model (first inference model). The first inference model includes a neural network (first neural network) constructed by machine learning using training data including at least eye images (e.g., Scheimpflug images of the eye, eye images acquired by other modalities, or Scheimpflug images of the eye and eye images acquired by other modalities).

[0141] The data input to the first neural network is a Scheimpflug image, and the data output from the first neural network is an anterior chamber region. That is, the first segmentation unit 1031 is configured to receive a Scheimpflug image (e.g., one or more Scheimpflug images, one or more processed image data, or one or more Scheimpflug images and one or more processed image data) acquired by the image acquisition unit 1010, input the Scheimpflug image to a first neural network of a first inference model, and acquire output data from the first neural network (the anterior chamber region in the input Scheimpflug image).

[0142] The device for constructing the first inference model (inference model construction device) may be provided in the ophthalmic device 1500, or may be provided in a peripheral device (such as a computer) of the ophthalmic device 1500, or may be another computer. The inference model construction device includes a learning processing unit and a neural network.

[0143] The neural network provided in the inference model building device typically includes a convolutional neural network (CNN), which has a known structure and may include an input layer, a convolutional layer, a pooling layer, a fully connected layer, an output layer, etc.

[0144] An image is input to the input layer. After the input layer, multiple pairs of convolutional layers and pooling layers are arranged. The number of pairs of convolutional layers and pooling layers can be arbitrary.

[0145] The convolution layer performs convolution operations to extract features (such as contours) from an image. A convolution operation is a multiplication and accumulation operation of a filter function (weighting coefficients, filter kernel) of the same dimension as the input image on the input image. The convolution layer applies the convolution operation to multiple parts of the input image. More specifically, the convolution layer multiplies the value of each pixel in the partial image to which the filter function has been applied by the value (weight) of the filter function corresponding to that pixel to calculate the product, and then calculates the sum of the products across multiple pixels in this partial image. The resulting sum-of-products value is assigned to the corresponding pixel in the output image. By performing the multiplication and accumulation operation while shifting the location (partial image) to which the filter function is applied, the convolution operation result for the entire input image is obtained. This convolution operation generates multiple images in which various features have been extracted using multiple weighting coefficients. In other words, multiple filtered images, such as smoothed images and edge images, are obtained. The multiple images generated by the convolution layer are called feature maps.

[0146] The pooling layer compresses (e.g., thins out data) the feature map generated by the immediately preceding convolutional layer. More specifically, the pooling layer calculates statistical values ​​of predetermined neighboring pixels of a pixel of interest in the feature map at predetermined pixel intervals, and outputs an image with dimensions smaller than the input feature map. The statistical values ​​applied to the pooling operation are, for example, maximum values ​​(max pooling) or average values ​​(average pooling). The pixel interval applied to the pooling operation is called the stride.

[0147] A convolutional neural network can extract many features from an input image by processing it using multiple pairs of convolutional layers and pooling layers.

[0148] A fully connected layer is provided after the last pair of convolutional and pooling layers. The number of fully connected layers can be arbitrary. In the fully connected layer, features compressed by a combination of convolution and pooling are used to perform processes such as image classification, image segmentation, and regression. An output layer is provided after the last fully connected layer to provide output results.

[0149] In some exemplary embodiments, the convolutional neural network may not include a fully connected layer (e.g., a fully convolutional network (FCN)) and may include a support vector machine, a recurrent neural network (RNN), or the like. Furthermore, the machine learning applied to the neural network by the inference model construction device may include transfer learning. That is, the neural network may include a neural network that has already been trained using other training data (training images) and whose parameters have been adjusted. Furthermore, the inference model construction device may be configured to be able to apply fine tuning to the trained neural network. Furthermore, the neural network to which the inference model construction device applies machine learning may be constructed using a known open-source neural network architecture.

[0150] The inference model construction device applies machine learning using training data to a neural network. When the neural network includes a convolutional neural network, the parameters of the neural network adjusted by the inference model construction device include, for example, filter coefficients of a convolutional layer and connection weights and offsets of a fully connected layer.

[0151] As described above, the training data may include one or more Scheimpflug images of one or more eyes. Because the Scheimpflug images of the eyes are the same type of images as the images input to the first neural network, the quality (accuracy, precision, etc.) of the output of the first neural network can be improved compared to when machine learning is performed using training data that includes only other types of images.

[0152] The types of images included in the training data are not limited to Scheimpflug images, and the training data may include, for example, images acquired by other ophthalmic modalities (fundus cameras, OCT devices, SLO, surgical microscopes, etc.), images acquired by imaging diagnostic modalities of any medical department (ultrasound diagnostic devices, X-ray diagnostic devices, X-ray CT devices, magnetic resonance imaging (MRI) devices, etc.), images generated by processing actual eye images (processed image data), pseudo images, etc. Furthermore, the number of images, etc. included in the training data may be increased using techniques such as data expansion and data augmentation.

[0153] The training method (machine learning method) for constructing the first neural network may be any method, such as supervised learning, unsupervised learning, or reinforcement learning, or a combination of two or more of these.

[0154] In some exemplary embodiments, supervised learning is performed using training data generated by annotation that labels input images. For example, this annotation involves identifying and labeling the anterior chamber region in each image included in the training data. The anterior chamber region is identified by, for example, at least one of a doctor, a computer, and another inference model. The inference model construction device can construct a first neural network by applying supervised learning using such training data to the neural network.

[0155] The first inference model including the first neural network constructed in this manner is a trained model that takes a Scheimpflug image (e.g., a Scheimpflug image acquired by the image collecting unit 1010, or its processed image data) as input and outputs the anterior chamber region in the input Scheimpflug image (e.g., information indicating the range or position of the anterior chamber region).

[0156] To avoid concentrating processing on specific units of the first neural network, the inference model construction device may randomly select and disable some units of the neural network and perform learning using the remaining units (dropout).

[0157] The techniques used to build the inference model are not limited to the examples shown here. For example, any technique such as a support vector machine, a Bayesian classifier, boosting, k-means, kernel density estimation, principal component analysis, independent component analysis, self-organizing map, random forest, or generative adversarial network (GAN) can be used to build the inference model.

[0158] The first segmentation unit 1031 of this example uses such a first inference model (first neural network) to perform processing to identify the anterior chamber region from the Scheimpflug image of the subject's eye.

[0159] The second segmentation unit 1032 includes a processor that performs second segmentation to identify a cellular region, and is configured to identify the cellular region from the anterior chamber region identified by the first segmentation unit 1031.

[0160] When the second segmentation of this example is performed using machine learning, the second segmentation unit 1032 is configured to perform the second segmentation using a pre-constructed inference model (second inference model). The second inference model includes a neural network (second neural network) constructed by machine learning using training data including at least eye images (e.g., Scheimpflug images of the eye, eye images acquired by other modalities, or Scheimpflug images of the eye and eye images acquired by other modalities).

[0161] The eye images included in the training data in this example include images corresponding to at least a portion of the anterior chamber of the eye (referred to as an anterior chamber image). The eye images included in the training data in this example may include the results of manual or automatic segmentation of an anterior segment image (e.g., a Scheimpflug image or an image acquired by another modality), such as an anterior chamber image extracted from the anterior segment image, or information indicating the extent or location of the anterior chamber image within the anterior segment image.

[0162] The data input to the second neural network is the anterior chamber region identified from the Scheimpflug image by the first segmentation unit 1031 (or the identified and extracted anterior chamber region (the same applies hereinafter)), and the data output from the second neural network is a cellular region. That is, the second segmentation unit 1032 is configured to receive the anterior chamber region identified by the first segmentation unit 1031, input this anterior chamber region to the second neural network of the second inference model, and obtain output data (cellular regions in the input anterior chamber region) from the second neural network.

[0163] The construction of the second inference model (second neural network) may be performed in the same manner as the construction of the first inference model (first neural network). For example, the construction of the second inference model (second neural network) is performed by the inference model construction device described above. Unless otherwise specified, the inference model construction device (learning processing unit and neural network) in this example may be the same as the inference model construction device used to construct the first inference model (first neural network).

[0164] The training data used to construct the second neural network may include one or more Scheimpflug images (e.g., anterior segment images with the anterior chamber region identified, anterior chamber images) acquired for one or more eyes. The types of images included in the training data are not limited to Scheimpflug images, and may include, for example, images acquired by other ophthalmological modalities, images acquired by an imaging diagnostic modality of any medical department, images generated by processing actual eye images, pseudo images, etc.

[0165] The training method (machine learning method) for constructing the second neural network may be any method, such as supervised learning, unsupervised learning, or reinforcement learning, or a combination of two or more of these.

[0166] In some exemplary embodiments, supervised learning is performed using training data generated by annotation that labels input images. For example, this annotation involves identifying and labeling cellular regions in each image included in the training data. The identification of cellular regions is performed by, for example, at least one of a doctor, a computer, and another inference model. The inference model construction device can construct a second neural network by applying supervised learning using such training data to the neural network.

[0167] The second inference model including the second neural network constructed in this manner is a trained model that takes the anterior chamber region identified by the first segmentation unit 1031 as input and the cellular region (e.g., information indicating the range or position of the cellular region) within the input anterior chamber region as output.

[0168] The second segmentation unit 1032 of the present example uses such a second inference model (second neural network) to perform processing to identify a cellular region from the anterior chamber region in the Scheimpflug image of the subject's eye.

[0169] The cell evaluation information generation processing unit 1033 includes a processor that executes cell evaluation information generation processing for generating cell evaluation information, and generates cell evaluation information from the cellular region identified by the second segmentation unit 1032.

[0170] When the cell evaluation information generation process of this example is performed using machine learning, the cell evaluation information generation processing unit 1033 is configured to perform the cell evaluation information generation process using a pre-constructed inference model (third inference model). The third inference model includes a neural network (third neural network) constructed by machine learning using training data including at least eye images (e.g., Scheimpflug images of the eye, eye images acquired by other modalities, or Scheimpflug images of the eye and eye images acquired by other modalities).

[0171] The ocular images included in the training data of this example include at least an anterior chamber image in which an image of inflammatory cells is depicted, and may further include an anterior chamber image in which an image of inflammatory cells is not depicted. The ocular images included in the training data of this example may include the results of manual or automatic segmentation of an anterior segment image (e.g., a Scheimpflug image or an image acquired by another modality), and may be, for example, a cell image extracted from the anterior chamber image in the anterior segment image, or information indicating the range or location of a cellular region in the anterior chamber image.

[0172] The data input to the third neural network is the output from the second segmentation unit 1032 or data generated based thereon (e.g., data indicating the range, position, distribution, etc. of the cellular region, or the anterior chamber region accompanied by the cellular region identification result), and the data output from the third neural network is cell evaluation information. That is, the cell evaluation information generation processing unit 1033 is configured to receive the cellular region identification result from the second segmentation unit 1032 or data generated based thereon, input the cellular region identification result or the data generated based thereon to the third neural network of the third inference model, and acquire output data (cell evaluation information) from the third neural network. As described above, the cell evaluation information is evaluation information regarding predetermined parameters related to inflammatory cells (e.g., the density, number, position, distribution, etc. of inflammatory cells).

[0173] The construction of the third inference model (third neural network) may be performed in the same manner as the construction of the first inference model (first neural network). For example, the construction of the third inference model (third neural network) is performed by the inference model construction device described above. Unless otherwise specified, the inference model construction device (learning processing unit and neural network) in this example may be the same as the inference model construction device used to construct the first inference model (first neural network).

[0174] The training data used to construct the third neural network may include one or more Scheimpflug images acquired for one or more eyes (e.g., an anterior segment image including an anterior chamber region in which a cellular region is identified, an anterior chamber image in which a cellular region is identified). The types of images included in the training data are not limited to Scheimpflug images, and the training data may include, for example, images acquired by other ophthalmology modalities, images acquired by an imaging diagnostic modality of any medical department, images generated by processing actual eye images, pseudo images, etc.

[0175] The training method (machine learning method) for constructing the third neural network may be any method, such as supervised learning, unsupervised learning, or reinforcement learning, or a combination of two or more of these.

[0176] In some exemplary embodiments, supervised learning is performed using training data generated by annotation that labels input images. For example, in this annotation, each image (in which a cellular region is identified) included in the training data is labeled with cell evaluation information generated from the image. The generation of cell evaluation information from the image is performed by, for example, at least one of a doctor, a computer, and another inference model. The inference model construction device can construct a third neural network by applying supervised learning using such training data to the neural network.

[0177] The third inference model including the third neural network constructed in this manner is a trained model that takes as input the results of cell area identification by the second segmentation unit 1032 or data generated based on the results, and outputs cell evaluation information based on the input results of cell area identification or data generated based on the results.

[0178] The cell evaluation information generation processing unit 1033 in this example uses such a third inference model (third neural network) to perform a process of generating cell evaluation information from a cell region in the anterior chamber region in a Scheimpflug image of the test eye.

[0179] 9 shows an example of the configuration of the evaluation processing unit 1030. The evaluation processing unit 1030B of this example includes a first segmentation unit 1041, a conversion processing unit 1042, a second segmentation unit 1043, and a cell evaluation information generation processing unit 1044.

[0180] The first segmentation unit 1041 has the same configuration and function as the first segmentation unit 1031 of the evaluation processing unit 1030A, and is configured to perform first segmentation to identify the anterior chamber region from the Scheimpflug image acquired by the image collection unit 1010.

[0181] The conversion processing unit 1042 converts the anterior chamber region identified by the first segmentation unit 1041 into data with a structure corresponding to the second segmentation performed by the second segmentation unit 1043. The second segmentation unit 1043 of the present example is configured to perform the second segmentation using a neural network (second neural network) constructed by machine learning, like the second segmentation unit 1032 of the evaluation processing unit 1030A. The conversion processing unit 1042 is configured to perform conversion processing for converting the anterior chamber region identified from the Scheimpflug image by the first segmentation unit 1041 into image data with a structure corresponding to the input layer of the second neural network of the second segmentation unit 1043.

[0182] For example, the input layer of the second neural network (convolutional neural network) of the second segmentation unit 1043 may be configured to accept data of a predetermined structure (shape, format). This predetermined data structure may be, for example, a predetermined image size (e.g., the number of vertical and horizontal pixels) or a predetermined image shape (e.g., square or rectangular). However, the image size and shape of the anterior chamber region identified by the first segmentation unit 1041 vary depending on the specifications of the ophthalmic apparatus, the conditions and settings at the time of imaging, and individual differences in the size and shape of the subject's eye. The conversion processing unit 1042 converts the structure of the anterior chamber region identified by the first segmentation unit 1041 (e.g., image size and / or image shape) into a structure that can be accepted by the input layer of the second neural network of the second segmentation unit 1043.

[0183] The image size conversion may be performed using any known image size conversion technique, and may include, for example, a process of dividing the anterior chamber region identified by the first segmentation unit 1041 into multiple partial images having image sizes according to the input layer, or a process of resizing the anterior chamber region identified by the first segmentation unit 1041 into a single image having an image size according to the input layer. The image shape conversion may be performed using any known image deformation technique. The same applies to conversion processes of other data structures.

[0184] In this disclosure, several examples are described in detail for applying conversion processing corresponding to the structure of a neural network to the anterior chamber region of a Scheimpflug image, but the manner of conversion processing and the configuration therefor are not limited to these.

[0185] For example, if the image input to the neural network is a Scheimpflug image, a configuration can be adopted in which a similar conversion process is applied to the input Scheimpflug image.Also, if the image input to the neural network is any processed image data of a Scheimpflug image, a configuration can be adopted in which a similar conversion process is applied to the input processed image data.

[0186] Furthermore, the arrangement of the element that performs the conversion process (an element including a processor that performs the conversion process, referred to as a conversion processing unit) may also be arbitrary. For example, the conversion processing unit may be arranged in a stage before the target neural network in the flow of a series of processes that are performed based on the acquired Scheimpflug image (e.g., a stage before an inference model that includes this neural network, or a stage inside this inference model but before this neural network), or may be arranged inside the target neural network. When arranged inside the target neural network, the conversion processing unit is arranged in a stage before the input layer that receives input that directly corresponds to the output of this neural network.

[0187] The second segmentation unit 1043 has the same configuration and function as the second segmentation unit 1032 of the evaluation processing unit 1030A, and is configured to perform second segmentation to identify a cellular region from the anterior chamber region whose data structure has been converted by the conversion processing unit 1042. The second neural network of the second segmentation unit 1043 is configured to receive input of the image data (the anterior chamber region whose data structure has been converted) generated by the conversion processing unit 1042, and to output a cellular region. The machine learning for constructing the second neural network of this example may be performed in the same manner as the machine learning for constructing the second neural network of the evaluation processing unit 1030A.

[0188] The cell evaluation information generation processing unit 1044 has the same configuration and function as the cell evaluation information generation processing unit 1033 of the evaluation processing unit 1030A, and is configured to execute cell evaluation information generation processing to generate cell evaluation information from the cell region identified by the second segmentation unit 1043.

[0189] In this way, the evaluation processing unit 1030B of this example may be configured such that the conversion processing unit 1042 is disposed between the first segmentation unit 1031 and the second segmentation unit 1032 of the evaluation processing unit 1030A. However, the configuration of the evaluation processing unit 1030B is not limited to this.

[0190] 10 shows an example of the configuration of the evaluation processing unit 1030. The evaluation processing unit 1030C of this example includes a second segmentation unit 1051 and a cell evaluation information generation processing unit 1052.

[0191] The second segmentation unit 1051 includes a processor that performs second segmentation to identify cellular regions, and is configured to identify cellular regions from the Scheimpflug image acquired by the image acquisition unit 1010.

[0192] When the second segmentation of this example is performed using machine learning, the second segmentation unit 1051 is configured to perform the second segmentation using a pre-constructed inference model (fourth inference model). The fourth inference model includes a neural network (fourth neural network) constructed by machine learning using training data including at least eye images (e.g., Scheimpflug images of the eye, eye images acquired by other modalities, or Scheimpflug images of the eye and eye images acquired by other modalities).

[0193] In some exemplary embodiments, the fourth neural network may include at least a portion of the first neural network and at least a portion of the second neural network in the evaluation processing unit 1030A. For example, the fourth neural network may be a neural network in which the first neural network and the second neural network are arranged in series. The fourth neural network configured in this manner has a function of identifying an anterior chamber region from a Scheimpflug image and a function of identifying a cellular region from the anterior chamber region.

[0194] In some exemplary embodiments, the fourth neural network may be trained to directly identify cellular regions from a Scheimpflug image without identifying the anterior chamber region. The fourth neural network is not limited to these embodiments and may include any machine-learned neural network for identifying cellular regions from a Scheimpflug image.

[0195] The data input to the fourth neural network is a Scheimpflug image, and the data output from the fourth neural network is a cell region. That is, the second segmentation unit 1051 is configured to receive a Scheimpflug image, input the Scheimpflug image to the fourth neural network of the fourth inference model, and obtain output data from the fourth neural network (cell regions in the input Scheimpflug image).

[0196] The construction of the fourth inference model (fourth neural network) may be performed in the same manner as the construction of the first inference model (first neural network) of the evaluation processing unit 1030A. For example, the construction of the fourth inference model (fourth neural network) is performed by the inference model construction device described above. Unless otherwise specified, the inference model construction device (learning processing unit and neural network) of this example may be the same as the inference model construction device used in constructing the first inference model (first neural network) of the evaluation processing unit 1030A.

[0197] The training data used to construct the fourth neural network may include one or more Scheimpflug images acquired for one or more eyes. The types of images included in the training data are not limited to Scheimpflug images. For example, the training data may include images acquired by other ophthalmology modalities, images acquired by an imaging diagnostic modality of any medical department, images generated by processing actual eye images, pseudo-images, etc.

[0198] Any images included in the training data may be annotated with information to aid in the processing performed by the fourth neural network, for example, by labeling the anterior chamber region in the image through pre-annotation.

[0199] The training method (machine learning method) for constructing the fourth neural network may be any method, such as supervised learning, unsupervised learning, or reinforcement learning, or a combination of two or more of these.

[0200] In some exemplary embodiments, supervised learning is performed using training data generated by annotation that labels input images. For example, this annotation involves identifying and labeling cellular regions in each image included in the training data. The identification of cellular regions is performed by, for example, at least one of a doctor, a computer, and another inference model. The inference model construction device can construct a fourth neural network by applying supervised learning using such training data to the neural network.

[0201] The fourth inference model including the fourth neural network constructed in this manner is a trained model that takes as input a Scheimpflug image (or its processed image data, etc.) acquired by the image collection unit 1010 and outputs a cellular area (e.g., information indicating the range or position of the cellular area) in the input Scheimpflug image.

[0202] The second segmentation unit 1051 of the present example uses such a fourth inference model (fourth neural network) to perform processing to identify a cellular region from a Scheimpflug image of the subject's eye.

[0203] The cell evaluation information generation processing unit 1052 includes a processor that executes cell evaluation information generation processing for generating cell evaluation information, and generates cell evaluation information from the cellular region identified by the second segmentation unit 1051.

[0204] When the cell evaluation information generation process of this example is performed using machine learning, the cell evaluation information generation processing unit 1052 is configured to perform the cell evaluation information generation process using a pre-constructed inference model (fifth inference model). The fifth inference model includes a neural network (fifth neural network) constructed by machine learning using training data including at least eye images (e.g., Scheimpflug images of the eye, eye images acquired by other modalities, or Scheimpflug images of the eye and eye images acquired by other modalities).

[0205] The data input to the fifth neural network is the output from the second segmentation unit 1051 or data generated based thereon (for example, data indicating the range, position, distribution, etc. of the cellular region, or the anterior chamber region accompanied by the identification result of the cellular region), and the data output from the fifth neural network is cell evaluation information. That is, the cell evaluation information generation processing unit 1052 is configured to receive the identification result of the cellular region by the second segmentation unit 1051 or data generated based thereon, input the identification result of the cellular region or the data generated based thereon to the fifth neural network of the fifth inference model, and acquire output data (cell evaluation information) from the fifth neural network.

[0206] The machine learning technique for constructing the fifth inference model (fifth neural network) may be the same as the machine learning technique for constructing the third neural network of the cell evaluation information generation processing unit 1033 of the evaluation processing unit 1030A. Furthermore, the training data used in the machine learning for constructing the fifth inference model (fifth neural network) may be the same as the training data used in the machine learning for constructing the third neural network of the evaluation processing unit 1030A.

[0207] The fifth inference model including the fifth neural network is a trained model that takes as input the results of cell area identification by the second segmentation unit 1051 or data generated based on that, and outputs cell evaluation information based on the input results of cell area identification or data generated based on that.

[0208] The cell evaluation information generation processing unit 1052 of this example executes a process of generating cell evaluation information from a cellular region in a Scheimpflug image of the subject's eye by using such a fifth inference model (fifth neural network).

[0209] 11 shows an example of the configuration of the evaluation processing unit 1030. The evaluation processing unit 1030D of this example includes a cell evaluation information generation processing unit 1061.

[0210] The cell evaluation information generation processing unit 1061 includes a processor that executes cell evaluation information generation processing for generating cell evaluation information, and generates cell evaluation information from the Scheimpflug image acquired by the image collection unit 1010.

[0211] When the cell evaluation information generation process of this example is performed using machine learning, the cell evaluation information generation processing unit 1061 is configured to perform the cell evaluation information generation process using a pre-constructed inference model (sixth inference model). The sixth inference model includes a neural network (sixth neural network) constructed by machine learning using training data including at least eye images (e.g., Scheimpflug images of the eye, eye images acquired by other modalities, or Scheimpflug images of the eye and eye images acquired by other modalities).

[0212] In some exemplary embodiments, the sixth neural network may include at least a portion of the first neural network, at least a portion of the second neural network, and at least a portion of the third neural network in the evaluation processing unit 1030A. For example, the sixth neural network may be a neural network in which the first neural network, the second neural network, and the third neural network are arranged in series. The sixth neural network configured in this manner has the functions of identifying an anterior chamber region from a Scheimpflug image, identifying a cellular region from the anterior chamber region, and generating cellular evaluation information from the cellular region.

[0213] In some exemplary embodiments, the sixth neural network may be trained to generate cell evaluation information directly from a Scheimpflug image without identifying an anterior chamber region and / or a cellular region. The sixth neural network is not limited to these embodiments and may include any machine learning neural network for identifying cell evaluation information from a Scheimpflug image.

[0214] The data input to the sixth neural network is the output from the image collecting unit 1010 or data generated based on it, and the data output from the sixth neural network is cell evaluation information. That is, the cell evaluation information generation processing unit 1061 is configured to receive a Scheimpflug image (and / or data generated based on this Scheimpflug image) acquired by the image collecting unit 1010, input this Scheimpflug image or data generated based on it to the sixth neural network of the sixth inference model, and acquire output data (cell evaluation information) from the sixth neural network.

[0215] The machine learning technique for constructing the sixth inference model (sixth neural network) may be the same as the machine learning technique for constructing the third neural network of the cell evaluation information generation processing unit 1033 of the evaluation processing unit 1030A. Furthermore, the training data used in the machine learning for constructing the sixth inference model (sixth neural network) may be the same as the training data used in the machine learning for constructing the third neural network of the evaluation processing unit 1030A.

[0216] The sixth inference model including the sixth neural network is a trained model that takes as input a Scheimpflug image (and / or data generated based on this Scheimpflug image) acquired by the image collection unit 1010, and outputs cell evaluation information based on the input Scheimpflug image (and / or data generated based on this Scheimpflug image).

[0217] The cell evaluation information generation processing unit 1061 in this example uses such a sixth inference model (sixth neural network) to perform a process of generating cell evaluation information from a Scheimpflug image of the test eye (and / or data generated based on this Scheimpflug image).

[0218] 12 shows an example of the configuration of the evaluation processing unit 1030. The evaluation processing unit 1030E of this example includes a first segmentation unit 1071 and a cell evaluation information generation processing unit 1072.

[0219] The first segmentation unit 1071 includes a processor that performs first segmentation to identify the anterior chamber region, and is configured to identify the anterior chamber region from the Scheimpflug image acquired by the image acquisition unit 1010.

[0220] When the first segmentation of this example is performed using machine learning, the first segmentation unit 1071 is configured to perform the first segmentation using a pre-constructed inference model (seventh inference model). The seventh inference model includes a neural network (seventh neural network) constructed by machine learning using training data including at least eye images (e.g., Scheimpflug images of the eye, eye images acquired by other modalities, or Scheimpflug images of the eye and eye images acquired by other modalities).

[0221] The machine learning technique for constructing the seventh inference model (seventh neural network) may be similar to the machine learning technique for constructing the first neural network of the first segmentation unit 1031 of the evaluation processing unit 1030A. Furthermore, the training data used in the machine learning for constructing the seventh inference model (seventh neural network) may be similar to the training data used in the machine learning for constructing the first neural network of the evaluation processing unit 1030A. In some exemplary embodiments, the seventh neural network may be the same as or similar to the first neural network, and the seventh inference model may be the same as or similar to the first inference model.

[0222] The seventh inference model including the seventh neural network is a trained model that takes as input a Scheimpflug image (or its processed image data, etc.) acquired by the image collection unit 1010 and outputs the anterior chamber region in the input Scheimpflug image (e.g., information indicating the range or position of the anterior chamber region).

[0223] The first segmentation unit 1071 of this example uses such a seventh inference model (seventh neural network) to perform processing to identify the anterior chamber region from the Scheimpflug image of the subject's eye.

[0224] The cell evaluation information generation processing unit 1072 includes a processor that executes cell evaluation information generation processing for generating cell evaluation information, and generates cell evaluation information from the anterior chamber region identified by the first segmentation unit 1071.

[0225] When performing the cell evaluation information generation process of this example using machine learning, the cell evaluation information generation processing unit 1072 is configured to perform the cell evaluation information generation process using a pre-constructed inference model (eighth inference model). The eighth inference model includes a neural network (eighth neural network) constructed by machine learning using training data including at least eye images (e.g., Scheimpflug images of the eye, eye images acquired by other modalities, or Scheimpflug images of the eye and eye images acquired by other modalities).

[0226] In some exemplary embodiments, the eighth neural network may include at least a portion of the second neural network and at least a portion of the third neural network of the evaluation processing unit 1030A. For example, the eighth neural network may be a neural network in which the second neural network and the third neural network are arranged in series. The eighth neural network configured in this manner has a function of identifying a cellular region from the anterior chamber region and a function of generating cellular evaluation information from the cellular region.

[0227] In some exemplary embodiments, the eighth neural network may be trained to generate cell-evaluation information directly from the anterior chamber region without identifying the cellular region. The eighth neural network is not limited to these embodiments and may include any machine-learned neural network for identifying cell-evaluation information from the anterior chamber region.

[0228] The data input to the eighth neural network is the output from the first segmentation unit 1071 or data generated based thereon, and the data output from the eighth neural network is cell evaluation information. That is, the cell evaluation information generation processing unit 1072 is configured to receive the anterior chamber region identified from the Scheimpflug image by the first segmentation unit 1071 (and / or data generated based on this anterior chamber region), input this anterior chamber region or the data generated based on it to the eighth neural network of the eighth inference model, and acquire output data (cell evaluation information) from the eighth neural network.

[0229] The machine learning technique for constructing the eighth inference model (eighth neural network) may be the same as the machine learning technique for constructing the third neural network of the cell evaluation information generation processing unit 1033 of the evaluation processing unit 1030A. Furthermore, the training data used in the machine learning for constructing the eighth inference model (eighth neural network) may be the same as the training data used in the machine learning for constructing the third neural network of the evaluation processing unit 1030A.

[0230] The eighth inference model including the eighth neural network is a trained model that receives as input the anterior chamber region identified by the first segmentation unit 1071 (and / or data generated based on this anterior chamber region) and outputs cell evaluation information based on the input anterior chamber region (and / or data generated based on this anterior chamber region).

[0231] The cell evaluation information generation processing unit 1072 in this example uses such an eighth inference model (eighth neural network) to perform a process of generating cell evaluation information from the anterior chamber region in the Scheimpflug image of the test eye (and / or data generated based on this anterior chamber region).

[0232] 13 shows an example of the configuration of the evaluation processing unit 1030. The evaluation processing unit 1030F of this example includes a first segmentation unit 1081, a conversion processing unit 1082, and a cell evaluation information generation processing unit 1083.

[0233] The first segmentation unit 1081 has the same configuration and function as the first segmentation unit 1031 of the evaluation processing unit 1030A, and is configured to perform first segmentation to identify the anterior chamber region from the Scheimpflug image acquired by the image collection unit 1010.

[0234] The conversion processing unit 1082 converts the anterior chamber region identified by the first segmentation unit 1081 into data with a structure corresponding to the cell evaluation information generation processing executed by the cell evaluation information generation processing unit 1083. The cell evaluation information generation processing unit 1083 of this example is configured to execute the cell evaluation information generation processing using a neural network (eighth neural network) constructed by machine learning, like the cell evaluation information generation processing unit 1072 of the evaluation processing unit 1030E. The conversion processing unit 1082 is configured to execute conversion processing for converting the anterior chamber region identified from the Scheimpflug image by the first segmentation unit 1081 into image data with a structure corresponding to the input layer of the eighth neural network of the cell evaluation information generation processing unit 1083.

[0235] For example, the input layer of the eighth neural network (convolutional neural network) of the cell evaluation information generation processing unit 1083 may be configured to accept data of a predetermined structure (shape, format). This predetermined data structure may be, for example, a predetermined image size (e.g., the number of vertical and horizontal pixels) or a predetermined image shape (e.g., square or rectangular). However, the image size and shape of the anterior chamber region identified by the first segmentation unit 1081 vary depending on the specifications of the ophthalmic apparatus, the conditions and settings at the time of imaging, and individual differences in the size and shape of the subject's eye. The conversion processing unit 1082 converts the structure of the anterior chamber region identified by the first segmentation unit 1081 (e.g., image size and / or image shape) into a structure that can be accepted by the input layer of the eighth neural network of the cell evaluation information generation processing unit 1083.

[0236] The image size conversion may be performed using any known image size conversion technique, and may include, for example, a process of dividing the anterior chamber region identified by the first segmentation unit 1081 into multiple partial images having image sizes according to the input layer, or a process of resizing the anterior chamber region identified by the first segmentation unit 1081 into a single image having an image size according to the input layer. The image shape conversion may be performed using any known image deformation technique. The same applies to conversion processes of other data structures.

[0237] The cell evaluation information generation processing unit 1083 has the same configuration and function as the cell evaluation information generation processing unit 1072 of the evaluation processing unit 1030E, and is configured to execute cell evaluation information generation processing for generating cell evaluation information from data obtained by processing the anterior chamber region identified by the first segmentation unit 1081 using the conversion processing unit 1082.

[0238] In this way, the evaluation processing unit 1030F of this example may be configured such that the conversion processing unit 1082 is disposed between the first segmentation unit 1071 and the cell evaluation information generation processing unit 1072 of the evaluation processing unit 1030E. However, the configuration of the evaluation processing unit 1030F is not limited to this.

[0239] So far, several examples of the evaluation processing unit 1030 including an inference model (neural network) constructed using machine learning have been mainly described. However, the evaluation processing unit 1030 is not limited to such a machine learning-based configuration. The evaluation processing unit 1030 according to the present disclosure may be implemented solely by a machine learning-based configuration, may be implemented by a combination of a machine learning-based configuration and a non-machine learning-based configuration, or may be implemented solely by a non-machine learning-based configuration.

[0240] Below, we will explain some examples of the evaluation processing unit 1030 that has only a non-machine learning based configuration. A person skilled in the art would be able to understand the aspects of the evaluation processing unit 1030 that combines a machine learning based configuration and a non-machine learning based configuration based on the various examples of machine learning based configurations described above and some examples of the evaluation processing unit 1030 that has only a non-machine learning based configuration described below.

[0241] 14 shows an example of the configuration of the evaluation processing unit 1030. The evaluation processing unit 1030G of this example includes a first analysis processing unit 1091, a second analysis processing unit 1092, and a third analysis processing unit 1093.

[0242] The first analysis processing unit 1091 includes a processor that performs first segmentation to identify the anterior chamber region, and is configured to apply a predetermined analysis process (referred to as the first analysis process) to the Scheimpflug image (and / or its processed image data) acquired by the image acquisition unit 1010 to identify the anterior chamber region in this Scheimpflug image.

[0243] The first analysis process may include any known segmentation for identifying the anterior chamber region in the Scheimpflug image. For example, the segmentation for identifying the anterior chamber region includes a segmentation for identifying an image region corresponding to the cornea (particularly, the posterior surface of the cornea) and a segmentation for identifying an image region corresponding to the lens (particularly, the anterior surface of the lens). The image region corresponding to the cornea is referred to as the corneal region, the image region corresponding to the posterior surface of the cornea is referred to as the posterior corneal region, the image region corresponding to the lens is referred to as the lens region, and the image region corresponding to the anterior surface of the lens is referred to as the anterior lens region.

[0244] The segmentation of the posterior corneal surface region may include any known segmentation method. In the segmentation of the posterior corneal surface region, artifacts in the Scheimpflug image and saturation of pixel values ​​can be problematic. To solve these problems, for example, the configuration shown in FIG. 4C can be employed. That is, by combining the imaging technique using the first imaging system 1014 and the second imaging system 1015 with the selection process of the Scheimpflug image using the image selection unit 1110, it becomes possible to select a Scheimpflug image free of artifacts and saturation and identify the posterior corneal surface region from the selected Scheimpflug image.

[0245] The segmentation of the anterior surface of the lens may include any known segmentation method. In the segmentation of the anterior surface of the lens, a problem arises, for example, in that the representation of the Scheimpflug image (the appearance of the Scheimpflug image) changes depending on the state of the pupil of the subject's eye (e.g., mydriatic state, non-mydriatic state, small pupil eye, etc.). For example, when the subject's eye is non-mydriatic or has a small pupil, the imaged area of ​​the lens is smaller than when the subject's eye is dilated. To solve this problem, a process for uniforming the representation of the Scheimpflug image can be applied, such as a process for estimating the position and shape of the unimaged portion of the anterior surface of the lens (the portion covered by the pupil) based on the anterior surface of the lens depicted in the Scheimpflug image. The process for uniforming the representation of the Scheimpflug image may be performed based on machine learning or non-machine learning. Furthermore, the process for estimating the position and shape of the anterior surface of the lens may include, for example, any known extrapolation process.

[0246] When the image acquisition unit 1010 acquires a series of Scheimpflug images by slit scanning, some images may contain problems and some may not, and the degree of problems may vary among the images. For example, some images may contain artifacts or saturation and some may not, or some images may contain artifacts of various states (e.g., position, size, shape, etc.). These phenomena may adversely affect the quality (e.g., stability, robustness, reproducibility, accuracy, precision, etc.) of the processing performed by the evaluation processing unit 1030G. In some exemplary embodiments, measures can be taken to prevent these phenomena from occurring or to reduce the adverse effects caused by these phenomena. An example of the former measure is to combine an imaging technique using the first imaging system 1014 and the second imaging system 1015 with Scheimpflug image selection processing using the image selection unit 1110. An example of the latter measure is to perform image correction, noise removal, noise reduction, image parameter adjustment, etc.

[0247] The second analysis processing unit 1092 includes a processor that performs second segmentation to identify a cellular region, and is configured to identify a cellular region by applying the second analysis processing to the anterior chamber region identified from the Scheimpflug image by the first analysis processing unit 1091.

[0248] In some exemplary embodiments, the second analysis processing unit 1092 may be configured to identify a cellular region based on the value of each pixel in the anterior chamber region (e.g., at least one of a brightness value, an R value, a G value, and a B value). In some exemplary embodiments, the second analysis processing unit 1092 may be configured to apply segmentation to the anterior chamber region to identify a cellular region. This segmentation is performed, for example, according to a program created based on the standard morphology (e.g., size, shape, etc.) of inflammatory cells (cellular regions). In some exemplary embodiments, the second analysis processing unit 1092 may be configured to identify a cellular region by at least a partial combination of these two techniques.

[0249] Countermeasures that can be taken when the image collecting unit 1010 collects a series of Scheimpflug images by slit scanning may be the same as those taken by the first analysis processing unit 1091. Furthermore, taking into consideration that a cellular region is generally a minute image region, countermeasures may be taken to distinguish between a cellular region and minute artifacts. For example, by performing processing to remove artifacts (such as ghosts), it is possible to prevent erroneous detection of artifacts in the detection of a cellular region.

[0250] The third analysis processing unit 1093 includes a processor that executes a cell evaluation information generation process for generating cell evaluation information, and is configured to generate cell evaluation information by applying the third analysis process to a cell region identified from the anterior chamber region of the Scheimpflug image by the second analysis processing unit 1092.

[0251] As mentioned above, the cell evaluation information may be any evaluation information regarding inflammatory cells, and may include, for example, information representing the state of inflammatory cells (e.g., any parameters such as density, number, position, distribution, etc.), or may include evaluation information generated based on information on specified parameters regarding the state of inflammatory cells.

[0252] In some exemplary embodiments, the third analysis processing unit 1093 can determine the density, number, location, distribution, etc. of one or more cellular regions identified by the second analysis processing unit 1092.

[0253] The process of determining the density of inflammatory cells includes, for example, a process of setting an image region of a predetermined size (e.g., an image region of 1 mm square) and a process of counting the number of cellular regions detected by the second analysis processing unit 1092 in the set image region. Here, the size of the image region (e.g., a dimension in real space, such as "1 mm") is defined, for example, based on the specifications of the optical system of the ophthalmic apparatus 1500 (e.g., design data of the optical system and / or actual measurement data of the optical system), and is typically defined as a correspondence relationship between pixels and dimensions in real space (e.g., dot pitch). The cell evaluation information may include information on the density of inflammatory cells determined in this manner, or may include evaluation information obtained from this density information. This evaluation information may include, for example, an evaluation result using the classification criteria for uveitis diseases proposed by the SUN Working Group. This classification standard defines grades according to the number of inflammatory cells present in one field of view (a field of view measuring 1 mm square) (i.e., the density (concentration) of inflammatory cells), with grade "0" being defined as less than 1 cell, grade "0.5+" as 1 to 5 cells, grade "1+" as 6 to 15 cells, grade "2+" as 16 to 25 cells, grade "3+" as 26 to 50 cells, and grade "4+" as 50 or more cells. The grade divisions in this classification standard may be more detailed or coarse. Furthermore, cell evaluation information may be generated based on other classification standards.

[0254] In some exemplary embodiments, the evaluation processing unit 1030G (third analysis processing unit 1093) may be configured to execute the following processes: identify a partial region of the anterior chamber region (e.g., a 1-millimeter square image region) identified from the Scheimpflug image by the first analysis processing unit 1091; determine the number of cellular regions belonging to this partial region; and calculate the density of inflammatory cells based on the number and the dimensions of the partial region. Here, the evaluation processing unit 1030G may be configured to select cellular regions located within this partial region from the cellular regions detected from the entire anterior chamber region by the second analysis processing unit 1092, and calculate the density based on the selected cellular regions by the third analysis processing unit 1093. Alternatively, the evaluation processing unit 1030G may be configured to analyze the partial region to identify cellular regions by the second analysis processing unit 1092, and calculate the density based on the cellular regions identified from the partial region by the third analysis processing unit 1093.

[0255] The process of determining the number of inflammatory cells includes, for example, a process of counting the number of cellular regions detected by the second analysis processing unit 1092. The cell evaluation information may include information on the number of inflammatory cells determined in this manner, or may include evaluation information obtained from this number information. For example, the cell evaluation information can determine the average density of inflammatory cells in the entire anterior chamber region by dividing the number of cellular regions detected in the entire anterior chamber region by the dimensions of the anterior chamber region (e.g., area, volume, etc.). Furthermore, the cell evaluation information may include an evaluation result (e.g., grade) based on the number of cellular regions detected in the entire anterior chamber region, or may include the number of cellular regions in a partial region of the anterior chamber region and / or an evaluation result based thereon.

[0256] The process of determining the position of the inflammatory cells may include, for example, a process of identifying the position of the cellular region detected by the second analysis processing unit 1092. The position of the cellular region may be expressed, for example, as coordinates in a defined coordinate system of the Scheimpflug image, or as a relative position (e.g., distance, direction, etc.) to a predetermined image region (reference region) depicted in the Scheimpflug image. This reference region may be, for example, the corneal region, the posterior corneal region, the lens region, the anterior lens region, or an image region corresponding to the axis of the eye (e.g., a straight line connecting the apex position of the cornea and the apex position of the anterior lens). The cell evaluation information may include information on the position of the inflammatory cells determined in this manner, or may include evaluation information obtained from this position information. For example, the cell evaluation information may include information representing the distribution of inflammatory cells (distribution of multiple cell regions), or may include an evaluation result (e.g., grade) based on the positions of one or more inflammatory cells, or may include an evaluation result (e.g., grade) based on the positions (distribution) of multiple inflammatory cells.

[0257] According to this aspect, since evaluation information can be generated based on a Scheimpflug image in which blurring of images of intraocular floaters (such as inflammatory cells) is reduced, it is possible to perform evaluation with higher quality than in the past in any of the examples described above. Note that it will be understood by those skilled in the art that evaluation with higher quality than in the past can also be performed when other evaluation information generation processes are adopted.

[0258] Several examples of the operation of the ophthalmic apparatus (1000, 1500) according to the embodiments will be described. Any matter related to the present disclosure, any matter related to the documents cited in the present disclosure, any matter related to the technical field to which the embodiments according to the present disclosure belong, any matter related to the technical field related to the embodiments according to the present disclosure, etc. can be combined with the following operation examples.

[0259] An example of the operation of the ophthalmic apparatus 1500 is shown in Fig. 15. As will be understood from the following description, steps S1 and S2 in Fig. 15 can also be performed by the ophthalmic apparatus 1000.

[0260] It is assumed that several operations (preparatory operations) have been performed by the ophthalmic apparatus 1500 before capturing (scanning) the subject's eye. The preparatory operations include adjusting the table on which the ophthalmic apparatus 1500 is placed, adjusting the chair used by the subject, adjusting the face rest (chin rest, forehead rest, etc.) of the ophthalmic apparatus 1500, and aligning the ophthalmic apparatus 1500 with the subject's eye.

[0261] In this operation example, first, conditions for scanning the subject's eye (scan conditions) are set (S1). At least a part of the condition setting may be performed manually, or at least a part of the condition setting may be performed automatically. Exemplary method and exemplary operation of the condition setting will be described later.

[0262] Examples of scanning conditions include conditions regarding the illumination system 1011, conditions regarding the imaging system 1012, conditions regarding the moving mechanism (or elements having similar functions to the moving mechanism: illumination scanner, movable illumination mirror, imaging scanner, movable imaging mirror, etc.), and conditions regarding the coordination (synchronization) of any two or three of the illumination system 1011, imaging system 1012, and moving mechanism.

[0263] Examples of conditions related to the illumination system 1011 include the intensity of the illumination light (light quantity: illumination control pulse height in Figures 5 and 6), the pulse width of the illumination light (projection time: illumination control pulse width in Figures 5 and 6), the dimensions of the slit light, and the direction of the slit light.

[0264] Examples of conditions related to the imaging system 1012 include the exposure time of the image sensor 1013 and the gain (sensitivity) of the image sensor 1013.

[0265] Examples of conditions related to the movement mechanism include the movement distance of the illumination light (scan range in FIGS. 5 and 6) and the movement speed of the illumination light (scan speed in FIG. 6).

[0266] Examples of conditions regarding the coordination (synchronization) of any two or three of the illumination system 1011, the imaging system 1012, and the movement mechanism include the coordination (synchronization) between the projection time of the illumination light and the exposure time of the image sensor 1013, and the coordination (synchronization) between the projection period of the illumination light and the exposure period of the image sensor 1013. In this operation example, the conditions are set so that the projection time of the illumination light is shorter than the exposure time of the image sensor 1013, and so that at least a portion of the projection period of the illumination light overlaps with at least a portion of the exposure period of the image sensor 1013. For example, the conditions are set so that a portion of the exposure period of the image sensor 1013 corresponds to the projection period of the illumination light, and so that the total time of the projection time and non-projection time of the illumination light is equal to the total time of the exposure time and non-exposure time of the image sensor 1013. As a result, as shown in FIG. 2, the illumination light projection sequence (multiple projections of illumination light arranged in chronological order) and the camera exposure sequence (multiple exposures of the image sensor 1013 arranged in chronological order) are synchronized with each other.

[0267] Upon receiving the instruction to start imaging, the control unit 1020 of the ophthalmologic apparatus 1500 controls the image acquisition unit 1010 based on the scan conditions set in step S1 to scan the subject's eye. Through this scan, a series of Scheimpflug images of the subject's eye are acquired (S2).

[0268] Next, the evaluation processing unit 1030 of the ophthalmologic apparatus 1500 generates evaluation information of the subject's eye based on the series of Scheimpflug images acquired in step S2 (S3).

[0269] As shown in FIG. 4, if the imaging system 1012 of the ophthalmic device 1500 includes a first imaging system 1014 and a second imaging system 1015, and the ophthalmic device 1500 includes or is capable of using an image selection unit 1110, the ophthalmic device 1500 may collect a first Scheimpflug image group and a second Scheimpflug image group using the first imaging system 1014 and the second imaging system 1015, respectively, in step S2, generate a series of Scheimpflug images from the first Scheimpflug image group and the second Scheimpflug image group using the image selection unit 1110, and generate evaluation information from the series of Scheimpflug images using the evaluation processing unit 1030 in step S3.

[0270] The ophthalmic apparatus 1500 can display the Scheimpflug image acquired in step S2 and / or the evaluation information generated in step S3 on a display device. The display device may be an element of the ophthalmic apparatus 1500, or may be an external device connected to the ophthalmic apparatus 1500. Control for displaying information on the display device is executed by a display control processor (not shown). This display control processor may be included in the control unit 1020, or may be an element of the ophthalmic apparatus 1500 separate from the control unit 1020, or may be provided in a device separate from the ophthalmic apparatus 1500.

[0271] Below, several examples of information display that can be performed by the ophthalmic apparatus 1500 will be described. The manner of information display is not limited to these examples. At least two of these examples can be at least partially combined.

[0272] In a first example of information display, the ophthalmic apparatus 1500 displays the Scheimpflug image acquired in step S2 and / or the evaluation information generated in step S3 as is on the display device. The ophthalmic apparatus 1500 may display other information (additional information) together with the Scheimpflug image and / or the evaluation information. The additional information may be any type of information that is useful for treating the subject's eye when referred to together with the Scheimpflug image and / or the evaluation information.

[0273] In a second example of information display, the ophthalmologic apparatus 1500 generates an image simulating an image of the eye (slit lamp image) acquired by a conventional slit lamp microscope from the Scheimpflug image and displays the generated simulated image on the display device. This makes it possible to provide an image simulating a slit lamp image that has traditionally been used to observe the subject's eye and is familiar to many physicians.

[0274] The process of generating a pseudo-image from a slit lamp image may be formed by a machine learning based process and / or a non-machine learning based process.

[0275] The machine learning-based processing is performed using a neural network constructed by machine learning using training data including a plurality of pairs of Scheimpflug and slit lamp images, for example, a convolutional neural network configured to receive a Scheimpflug image as an input and output a pseudo-image.

[0276] The non-machine learning based processing may include, for example, processing for transforming the appearance of an image, such as processing for generating artificial blur, color conversion, image quality conversion, etc. Exemplary aspects of the non-machine learning based processing include processing for constructing a three-dimensional image (e.g., an 8-bit grayscale volume) from a series of Scheimpflug images collected by slit scanning, processing for reducing the maximum value of the pixel value range (pixel value gradation) (e.g., processing for reducing 256 gradations to 10 gradations), processing for setting a region of interest (e.g., a rectangular parallelepiped region of predetermined dimensions) in the gradation-converted three-dimensional image, and processing for constructing a front image of the set region of interest (e.g., maximum intensity projection (MIP)).

[0277] The pseudo-image may be an image that represents the same area as the Scheimpflug image with a wide focus, or an image that represents a subarea of ​​the area represented by the Scheimpflug image, such as a field of view (i.e., a field of view measuring 1 mm square), which is the evaluation range in the classification criteria for uveitis diseases proposed by the SUN Working Group.

[0278] For example, the ophthalmologic apparatus 1500 can highlight a portion of interest in the pseudo image. Examples of the portion of interest include an image region corresponding to inflammatory cells, an image region corresponding to anterior chamber flare, an image region corresponding to opacity of the lens, etc.

[0279] In a third example of information display, the ophthalmologic apparatus 1500 displays, on the display device, information that visualizes the evaluation information generated in step S3. This visualized information may be, for example, a map that represents the distribution of predetermined objects or a map that represents the distribution of values ​​of predetermined parameters.

[0280] An example of such a map is a map (inflammatory state map) showing the inflammatory state of the subject's eye. Examples of such inflammatory state maps include an inflammatory cell map showing the position (distribution) of inflammatory cells in the anterior chamber, and an inflammatory cell density map (inflammatory cell number map) showing the distribution of the density (or number) of inflammatory cells in the anterior chamber. The process of creating these maps related to inflammatory cells includes, for example, a process of identifying image regions (cellular regions) corresponding to inflammatory cells from each of a series of Scheimpflug images obtained using slit scanning (second segmentation), a process of determining the position of each identified cellular region (for example, two-dimensional coordinates in the definition coordinate system of the Scheimpflug image, or three-dimensional coordinates in the definition coordinate system of a series of three-dimensional images based on the Scheimpflug images), and a process of creating a map based on the determined position of each cellular region.

[0281] The ophthalmic apparatus 1500 can display an inflammation state map together with the Scheimpflug image and / or the inflammation state information. For example, the ophthalmic apparatus 1500 can display a front image based on a series of Scheimpflug images collected by slit scanning and an inflammation state map generated based on the same series of Scheimpflug images. As a specific example, the ophthalmic apparatus 1500 can display the inflammation state map superimposed on the front image, or the front image and the inflammation state map side by side.

[0282] The process of generating information to be displayed on the display device is executed by an information generation processor (not shown). This display control processor may be included in the evaluation processing unit 1030, or may be an element of the ophthalmic device 1500 separate from the evaluation processing unit 1030, or may be provided in a device separate from the ophthalmic device 1500. This completes the description of the operation example in FIG. 15.

[0283] An example of the operation of the ophthalmic apparatus 1500 is shown in Fig. 16. This operation example provides a specific example of step S1 (condition setting) and step S2 (scanning) of the operation example of Fig. 15 described above. As with the operation example of Fig. 15, this operation example can also be performed by the ophthalmic apparatus 1000. Also, as with the operation example of Fig. 15, it is assumed that some preparatory operations have already been completed. It is also assumed that the ophthalmic apparatus 1500 can refer to imaging mode information (for example, the imaging mode information 1200 of Fig. 5 or the imaging mode information 1210 of Fig. 6).

[0284] In this operation example, first, an imaging mode is designated (S11). The imaging mode is designated manually or automatically. Manual designation is performed using a user interface (not shown). Automatic designation is performed based on predetermined reference information such as a diagnosis name (disease name), examination type, and examination record. This reference information is obtained, for example, from medical records related to the subject (examined eye).

[0285] Next, the control unit 1020 of the ophthalmic device 1500 sets the pulse height (illumination control pulse height) and pulse width (illumination control pulse width) of the control pulse signal to be applied to the pulse-driven illumination light source based on the imaging mode specified in step S11 and the imaging mode information (S12).

[0286] Furthermore, the control unit 1020 of the ophthalmic device 1500 sets the movement distance (scan range) and movement speed (scan speed) of the optical system (illumination system 1011, imaging system 1012) moved by the movement mechanism for scanning based on the imaging mode specified in step S11 and the imaging mode information (S13).

[0287] In addition, the control unit 1020 of the ophthalmic device 1500 sets the exposure time of the image sensor 1013 based on one or more of the shooting mode specified in step S11, the conditions related to the illumination system 1011 set in step S12, and the conditions related to the movement mechanism set in step S13 (S14).

[0288] The order of steps S12 and S13 is not limited to this example, and the set conditions are also not limited to this example. For example, conditions regarding the coordination (synchronization) of any two or three of the illumination system 1011, the imaging system 1012, and the movement mechanism may be set. Note that the condition setting in this example is performed to satisfy the following two conditions: (1) the projection time of illumination light onto the subject's eye is shorter than the exposure time of the image sensor 1013; and (2) at least a portion of the projection period of illumination light onto the subject's eye overlaps with at least a portion of the exposure period of the image sensor. The condition setting in this example may also be performed to satisfy the following optional condition: (3) the total time of the projection and non-projection of illumination light onto the subject's eye is equal to the total time of the exposure and non-exposure periods of the image sensor 1013.

[0289] After the condition setting is completed, the optical system (illumination system 1011, imaging system 1012) of the ophthalmic apparatus 1500 is aligned with the eye to be examined (S15). This alignment may be automatic or manual. For details of alignment, see Patent Document 3 (JP 2019-213733 A).

[0290] Next, the optical system of the ophthalmic apparatus 1500 is placed at a predetermined scan start position (S16). This process may be performed automatically or manually.

[0291] If the alignment in step S15 is performed to guide the optical system to the scan start position, step S16 does not need to be performed, but the alignment result in step S15 may be confirmed or adjusted in step S16. This confirmation or adjustment may be performed automatically or manually. For details of the alignment to guide the optical system to the scan start position, see Patent Document 3 (JP 2019-213733 A).

[0292] After the optical system is placed at the scan start position, the image collecting unit 1010 starts scanning the subject's eye (slit scan) (S17). For example, the scan is started in response to a command from the user, or in response to detection that the optical system has been placed at the scan start position.

[0293] When scanning starts, the control unit 1020 first causes the image sensor 1013 to start exposure (see FIG. 2) (S18) based on the scanning conditions set in steps S12 to S14. This step corresponds to the start of one exposure period.

[0294] Next, the control unit 1020 causes the illumination light source to emit pulses of illumination light (see FIG. 2) (S19) based on the scan conditions set in steps S12 to S14. This step corresponds to one pulse of light emission.

[0295] Next, the control unit 1020 causes the image sensor 1013 to end exposure (see FIG. 2) (S20) based on the scanning conditions set in steps S12 to S14. This step corresponds to the end of one exposure period started in step S18.

[0296] In this manner, in this operation example (steps S18 to S20), as shown in FIG. 17A, one entire projection period corresponds to a part of one exposure period (overlapped in time, parallel in time).

[0297] Generally, the scanning according to the embodiment is performed under the condition that the projection time of the illumination light onto the subject's eye is shorter than the exposure time of the image sensor 1013, and that at least a part of the projection time of the illumination light onto the subject's eye overlaps with at least a part of the exposure time of the image sensor 1013. In the example shown in Fig. 17B, a part of one projection time corresponds to a part of one exposure period.

[0298] Furthermore, since it is also necessary to satisfy the condition that the length of one projection period (projection time) is shorter than the length of one exposure period (exposure time), among the conditions that at least a part of the projection period and at least a part of the exposure period overlap, the case where a part of the projection period overlaps with the entire exposure period and the case where the entire projection period overlaps with the entire exposure period are logically excluded.

[0299] Therefore, the condition in the embodiment that "the projection time of the illumination light is shorter than the exposure time of the imaging element, and at least a portion of the projection period of the illumination light onto the test eye overlaps with at least a portion of the exposure period of the imaging element" can be rephrased as, for example, "the projection time of the illumination light is shorter than the exposure time of the imaging element, and at least a portion of the projection period of the illumination light onto the test eye overlaps with at least a portion of the exposure period of the imaging element," "the projection time of the illumination light is shorter than the exposure time of the imaging element, and at least a portion of the projection period of the illumination light onto the test eye overlaps with at least a portion of the exposure period of the imaging element, or the (entire) projection period of the illumination light onto the test eye overlaps with at least a portion of the exposure period of the imaging element," "at least a portion of the projection period of the illumination light onto the test eye overlaps with at least a portion of the exposure period of the imaging element," "part of the projection period of the illumination light onto the test eye overlaps with at least a portion of the exposure period of the imaging element, or the (entire) projection period of the illumination light onto the test eye overlaps with at least a portion of the exposure period of the imaging element," etc.

[0300] Steps S18 to S20 are repeated until a predetermined condition for ending the scan is satisfied (S21: No). The condition for ending the scan may be, for example, that a predetermined number of Scheimpflug images have been acquired, that steps S18 to S20 have been repeated a predetermined number of times, that the execution time of steps S18 to S20 has reached a predetermined time, or that the user has performed a predetermined instruction operation (scan end operation).

[0301] When the scan end condition is satisfied (S21: Yes), the scan of the eye to be examined, which started in step S17, ends (S22). Step S22 corresponds to the end of step S2 in FIG.

[0302] After step S22, the ophthalmologic apparatus 1500 may be configured to execute any process such as evaluation information generation process, information display process, information storage process, or information analysis process in step S3 of Fig. 15. This completes the description of the operation example in Fig. 16.

[0303] An example of a specific configuration of an ophthalmic apparatus capable of functioning as the above-described ophthalmic apparatus 1000 or 1500 is shown in Fig. 18. Fig. 18 is a top view, in which the direction along the axis of the subject's eye E is defined as the Z direction, the direction perpendicular to this that is the left-right direction for the subject is defined as the X direction, and the direction perpendicular to both the X and Z directions (the up-down direction, the body axis direction) is defined as the Y direction.

[0304] The ophthalmic apparatus of this example is a slit lamp microscope system 1 having a configuration similar to that disclosed in Patent Document 3 (JP 2019-213733 A), and includes an illumination system 2, an imaging system 3, a video imaging system 4, an optical path coupling element 5, a movement mechanism 6, a control unit 7, a data processing unit 8, a communication unit 9, and a user interface 10. The cornea of ​​the subject's eye E is indicated by the symbol C, and the crystalline lens is indicated by the symbol CL. The anterior chamber corresponds to the region between the cornea C and the crystalline lens CL (the region between the cornea C and the iris). For details of each element of the slit lamp microscope system 1, please refer to Patent Document 3 (JP 2019-213733 A).

[0305] The combination of the illumination system 2, the imaging system 3, and the moving mechanism 6 is an example of the image acquisition unit 1010 of the ophthalmic apparatus 1000 (1500). The illumination system 2 is an example of the illumination system 1011, and the imaging system 3 is an example of the imaging system 1012.

[0306] The illumination system 2 projects slit light onto the anterior segment of the subject's eye E. Reference symbol 2a indicates the optical axis (illumination optical axis) of the illumination system 2. The imaging system 3 images the anterior segment onto which the slit light from the illumination system 2 is projected. Reference symbol 3a indicates the optical axis (imaging optical axis) of the imaging system 3. The optical system 3A guides light from the anterior segment of the subject's eye E onto which the slit light is projected to the imaging element 3B. The imaging element 3B receives the light guided by the optical system 3A on an imaging surface. The imaging element 3B includes an area sensor (such as a CCD area sensor or a CMOS area sensor) having a two-dimensional imaging area. The imaging element 3B is an example of the imaging element 1013 of the ophthalmic apparatus 1000 (1500).

[0307] The illumination system 2 and the imaging system 3 function as a Scheimpflug camera. The object plane along the illumination optical axis 2a, the imaging plane of the optical system 3A, and the imaging element 3B are configured to satisfy the Scheimpflug condition. That is, the YZ plane (including the object plane) passing through the illumination optical axis 2a, the principal plane of the optical system 3A, and the imaging plane of the imaging element 3B intersect on the same straight line. This allows imaging to be performed with at least the area from the posterior surface of the cornea C to the anterior surface of the crystalline lens CL (the anterior chamber) in focus. Furthermore, the illumination system 2 and the imaging system 3 allow imaging to be performed with at least the area from the vertex of the anterior surface of the cornea C (Z=Z1) to the vertex of the posterior surface of the crystalline lens CL (Z=Z2) in focus. The coordinate Z=Z0 indicates the intersection of the illumination optical axis 2a and the imaging optical axis 3a.

[0308] The video imaging system 4 is a video camera, and captures video of the anterior segment of the subject's eye E in parallel with the imaging of the subject's eye by the illumination system 2 and the imaging system 3. The optical path coupling element 5 couples the optical path of the illumination system 2 (illumination optical path) with the optical path of the video imaging system 4 (video imaging optical path).

[0309] 19 shows a specific example of an optical system including an illumination system 2, an imaging system 3, a moving image imaging system 4, and an optical path combining element 5. The optical system shown in Fig. 19 includes an illumination system 20 which is an example of the illumination system 2, a left imaging system 30L and a right imaging system 30R which are examples of the imaging system 3 (first imaging system 1014 and second imaging system 1015), a moving image imaging system 40 which is an example of the moving image imaging system 4, and a beam splitter 47 which is an example of the optical path combining element 5.

[0310] Reference symbol 20a indicates the optical axis (illumination optical axis) of the illumination system 20, reference symbol 30La indicates the optical axis (left photographing optical axis) of the left photographing system 30L, and reference symbol 30Ra indicates the optical axis (right photographing optical axis) of the right photographing system 30R. Angle θL indicates the angle between the illumination optical axis 20a and the left photographing optical axis 30La, and angle θR indicates the angle between the illumination optical axis 20a and the right photographing optical axis 30Ra. Coordinate Z=Z0 indicates the intersection of the illumination optical axis 20a, the left photographing optical axis 30La, and the right photographing optical axis 30Ra.

[0311] The movement mechanism 6 moves the illumination system 20, the left imaging system 30L, and the right imaging system 30R in the direction indicated by the arrow 49 (X direction).

[0312] An illumination light source 21 of the illumination system 20 outputs illumination light (e.g., visible light), and a positive lens 22 refracts the illumination light. A slit forming unit 23 forms a slit to allow a portion of the illumination light to pass through. The generated slit light is refracted by objective lens groups 24 and 25, reflected by a beam splitter 47, and projected onto the anterior segment of the subject's eye E.

[0313] The reflector 31L and imaging lens 32L of the left imaging system 30L guide light (light traveling in the direction of the left imaging system 30L) from the anterior segment onto which the slit light is projected by the illumination system 20 to the imaging element 33L. The imaging element 33L receives the guided light on an imaging surface 34L. The left imaging system 30L repeatedly captures images in parallel with the movement of the illumination system 20, left imaging system 30L, and right imaging system 30R by the movement mechanism 6. This allows multiple images of the anterior segment (a series of Scheimpflug images) to be obtained. The object plane along the illumination optical axis 20a, the optical system including the reflector 31L and imaging lens 32L, and the imaging surface 34L satisfy the Scheimpflug condition. The right imaging system 30R has a similar configuration and function.

[0314] The Scheimpflug image acquisition by the left imaging system 30L and the Scheimpflug image acquisition by the right imaging system 30R are performed in parallel with each other. The combination of the series of Scheimpflug images acquired by the left imaging system 30L and the series of Scheimpflug images acquired by the right imaging system 30R corresponds to the combination of the first Scheimpflug image group and the second Scheimpflug image group.

[0315] The control unit 7 can synchronize the repeated photographing by the left photographing system 30L and the repeated photographing by the right photographing system 30R. This allows a correspondence relationship to be obtained between a series of Scheimpflug images obtained by the left photographing system 30L and a series of Scheimpflug images obtained by the right photographing system 30R. Note that the control unit 7 or the data processing unit 8 may perform a process of determining the correspondence relationship between a plurality of anterior eye images obtained by the left photographing system 30L and a plurality of anterior eye images obtained by the right photographing system 30R.

[0316] The video imaging system 40 captures video of the anterior segment of the subject's eye E from a fixed position in parallel with imaging by the left imaging system 30L and imaging by the right imaging system 30R. Light transmitted through a beam splitter 47 is reflected by a reflector 48 and enters the video imaging system 40. The light entering the video imaging system 40 is refracted by an objective lens 41, and then an image is formed on the imaging surface of an imaging element 43 (area sensor) by an imaging lens 42. The video imaging system 40 is used for monitoring the movement of the subject's eye E, alignment, tracking, processing collected Scheimpflug images, etc.

[0317] Returning to FIG. 18, the movement mechanism 6 moves the illumination system 2 and the imaging system 3 integrally in the X direction. The control unit 7 controls each part of the slit lamp microscope system 1. The control unit 7 controls the illumination system 2, the imaging system 3, and the movement mechanism 6, and the video imaging system 4 in parallel, thereby allowing the image acquisition unit 1010 of the ophthalmic apparatus 1000 (1500) to execute slit scanning (collection of a series of Scheimpflug images) and video imaging (collection of a series of time-series images) in parallel. The control unit 7 also controls the illumination system 2, the imaging system 3, and the movement mechanism 6, and the video imaging system 4 in synchronization with each other, thereby synchronizing the slit scanning and video imaging.

[0318] When the imaging system 3 includes a left imaging system 30L and a right imaging system 30R, the control unit 7 can synchronize repeated imaging by the left imaging system 30L (collection of the first Scheimpflug image group) with repeated imaging by the right imaging system 30R (collection of the second Scheimpflug image group).

[0319] The control unit 7 includes a processor, a storage device, etc. The storage device stores computer programs such as various control programs. The functions of the control unit 7 are realized by cooperation between software such as the control programs and hardware such as the processor. The control unit 7 controls the illumination system 2, the imaging system 3, and the movement mechanism 6 to scan a three-dimensional region of the subject's eye E with slit light. For details of this control, see Patent Document 3 (JP 2019-213733 A).

[0320] The data processing unit 8 executes various types of data processing. The data processing unit 8 includes a processor, a storage device, etc. The storage device stores computer programs such as various data processing programs. The functions of the data processing unit 8 are realized by cooperation between software such as the data processing programs and hardware such as a processor. The data processing unit 8 may have the function of the image selection unit 1110 and / or the function of the evaluation processing unit 1030 of the ophthalmic apparatus 1000 (1500).

[0321] The communication unit 9 performs data communication between the slit lamp microscope system 1 and other devices. The user interface 10 includes any user interface device such as a display device and an operation device.

[0322] The slit lamp microscope system 1 shown in FIGS. 18 and 19 is merely an example, and the configuration for implementing the ophthalmic apparatus 1000 or 1500 is not limited to the slit lamp microscope system 1.

[0323] Some functions and effects of the ophthalmologic apparatus according to the embodiment will be described.

[0324] An ophthalmic apparatus (1000, 1500) according to an embodiment includes an image acquisition unit (1010) and a control unit (1020). The image acquisition unit includes an optical system (illumination system 1011, imaging system 1012) that satisfies the Scheimpflug condition and projects illumination light onto the subject's eye and captures the image using an imaging element (1013). The image acquisition unit, under the control of the control unit, acquires a series of images (a series of Scheimpflug images) while changing the projection position of the illumination light and the imaging position. In other words, under the control of the control unit, the image acquisition unit acquires a series of images by scanning the subject's eye while maintaining the optical system in a state where the Scheimpflug condition is satisfied. The control unit controls the image acquisition unit (optical system) so that the projection time of the illumination light onto the subject's eye is shorter than the exposure time of the imaging element, and so that at least a portion of the projection period of the illumination light onto the subject's eye overlaps with at least a portion of the exposure period of the imaging element.

[0325] According to the embodiment having such a configuration, exposure is substantially performed only during a projection period shorter than the exposure period. This reduces image blurring caused by movement of the scan position or eye movement during exposure, enabling the provision of high-quality images. Thus, the ophthalmic apparatus according to the embodiment contributes to improving the image quality of ophthalmic apparatuses that use optical scanning for imaging. This enables the ophthalmic apparatus according to the embodiment to perform high-quality image analysis, such as corneal detection and cell detection. Furthermore, the ophthalmic apparatus according to the embodiment can shorten the time that illumination light is projected onto the subject's eye, thereby reducing the subject's burden. Additionally, the ophthalmic apparatus according to the embodiment does not require repeated sudden starts and stops of the optical system for scanning, which prevents vibrations caused by such movements during scanning and thus does not adversely affect the quality of the captured image.

[0326] An ophthalmic apparatus (1) according to one embodiment includes an illumination system (2), an imaging system (3), a movement mechanism (6), and a control unit (7). The illumination system is configured to project illumination light onto the subject's eye. The imaging system is configured to capture an image of the subject's eye. The imaging system includes an image sensor (3B). The illumination system and the imaging system are configured to satisfy the Scheimpflug condition. The movement mechanism is configured to move the illumination system and the imaging system. The control unit is configured to control the illumination system, the imaging system, and the movement mechanism to cause the imaging system to collect a series of images. In controlling the imaging system to collect a series of images of the subject's eye, the control unit controls at least one of the illumination system and the imaging system so that the projection time of the illumination light onto the subject's eye is shorter than the exposure time of the image sensor and so that at least a portion of the projection period of the illumination light onto the subject's eye overlaps with at least a portion of the exposure period of the image sensor.

[0327] The advantages of such an embodiment include improving the image quality of an ophthalmic device that uses an optical scanning imaging method, performing high-quality image analysis, not imposing a heavy burden on the subject, and not generating vibrations that adversely affect the imaging quality.

[0328] Some optional aspects of the embodiments are described below. Note that the matters related to the optional aspects are not limited to the matters described below, but may be matters included in the present disclosure or matters derived from the matters included in the present disclosure. In addition, two or more optional aspects may be combined.

[0329] In an embodiment, the control unit may be configured to control the imaging system (image acquisition unit) to repeatedly expose the image sensor in controlling the imaging system (image acquisition unit) to acquire a series of images of the subject's eye. The control unit may be further configured to control the illumination system to modulate the intensity of illumination light in parallel with controlling the imaging system to repeatedly expose the image sensor. The control unit may be further configured to control the illumination system so that illumination light is intermittently projected onto the subject's eye.

[0330] In an embodiment, the control unit may be configured to perform synchronization control between the illumination system and the imaging system in controlling the imaging system (image acquisition unit) to collect a series of images of the subject's eye. The control unit may further be configured to perform synchronization control between the illumination system and the imaging system so that the total time of projection and non-projection of illumination light is equal to the total time of exposure and non-exposure of the image sensor.

[0331] In an embodiment, the control unit may be configured to control the illumination system (image acquisition unit) to change the projection time of the illumination light. The control unit may be further configured to control the illumination system (image acquisition unit) based on the speed at which the illumination system and the imaging system are moved by the movement mechanism to change the projection time of the illumination light. The control unit may be further configured to determine the speed at which the illumination system and the imaging system are moved by the movement mechanism based on an imaging mode selected from two or more preset imaging modes.

[0332] In an embodiment, the control unit may be configured to control the illumination system (image collection unit) to change the projection time of the illumination light, and may further be configured to control the illumination system (image collection unit) based on a shooting mode selected from two or more preset shooting modes to change the projection time of the illumination light.

[0333] In an embodiment, the control unit may be configured to control the imaging system (image acquisition unit) to change the exposure time of the image sensor. The control unit may be further configured to control the imaging system (image acquisition unit) based on the speed at which the illumination system and the imaging system are moved by the movement mechanism to change the exposure time of the image sensor. The control unit may be further configured to determine the speed at which the illumination system and the imaging system are moved by the movement mechanism based on an imaging mode selected from two or more preset imaging modes.

[0334] In an embodiment, the control unit may be configured to control the imaging system (image collection unit) to change the exposure time of the imaging element, and may further be configured to control the imaging system (image collection unit) based on an imaging mode selected from two or more preset imaging modes to change the exposure time of the imaging element.

[0335] In an embodiment, the ophthalmologic apparatus (1500) may further include an evaluation processing unit (1030). The evaluation processing unit is configured to generate evaluation information of the subject's eye based on a series of images of the subject's eye collected by the imaging system (image collection unit). The evaluation processing unit may further be configured to generate evaluation information of floaters present in the subject's eye as the evaluation information of the subject's eye.

[0336] In the embodiment, the illumination light projected onto the subject's eye by the illumination system (image collecting unit) may be slit light.

[0337] Those skilled in the art will understand that these optional aspects can further improve the image quality of an ophthalmic device that uses an optical scanning imaging method, and can provide applications for an ophthalmic device that uses an optical scanning imaging method.

[0338] <Other embodiments> Although the embodiments of the ophthalmic apparatus have been described above, the embodiments according to the present disclosure are not limited to ophthalmic apparatuses. Examples of embodiments other than ophthalmic apparatuses include a control method for an ophthalmic apparatus, a program, and a recording medium. As with the embodiments of the ophthalmic apparatus, these embodiments also achieve the effects of improving the image quality of an ophthalmic apparatus that uses an optical scanning imaging method, performing high-quality image analysis, not imposing a significant burden on the subject, and not generating vibrations that adversely affect the imaging quality.

[0339] A control method for an ophthalmic apparatus according to one embodiment is a method for controlling an ophthalmic apparatus including an illumination system, an imaging system, a movement mechanism, and a processor. The illumination system projects illumination light onto an eye to be examined, and the imaging system captures an image of the eye to be examined using an imaging element. The illumination system and the imaging system are configured to satisfy the Scheimpflug condition. The movement mechanism moves the illumination system and the imaging system. The method according to this embodiment causes the processor to concurrently execute the following three controls: (1) control of at least one of the illumination system and the imaging system to make the projection time of the illumination light onto the eye to be shorter than the exposure time of the imaging element; (2) control of at least one of the illumination system and the imaging system to overlap at least a portion of the projection period of the illumination light onto the eye to be examined with at least a portion of the exposure period of the imaging element; and (3) control of the illumination system, the imaging system, and the movement mechanism to cause the imaging system to collect a series of images of the eye to be examined.

[0340] A control method for an ophthalmic apparatus according to one embodiment is a method for controlling an ophthalmic apparatus including an image acquisition unit and a processor. The image acquisition unit includes an optical system that projects illumination light onto the subject's eye and captures images using an image sensor, the optical system satisfying the Scheimpflug condition. The image acquisition unit further acquires a series of images while changing the projection position of the illumination light and the image sensor. In other words, the image acquisition unit acquires a series of images by scanning the subject's eye while maintaining the optical system in a state where the Scheimpflug condition is satisfied. A program according to this embodiment causes a processor to execute the following two controls in parallel to cause the image acquisition unit to acquire a series of images of the subject's eye: (1) control of the image acquisition unit (optical system) to make the projection time of the illumination light onto the subject's eye shorter than the exposure time of the image sensor; and (2) control of the image acquisition unit (optical system) to overlap at least a portion of the projection period of the illumination light onto the subject's eye with at least a portion of the exposure period of the image sensor.

[0341] A program according to one embodiment causes a computer to control an ophthalmic apparatus. The ophthalmic apparatus includes an illumination system, an imaging system, and a movement mechanism. The illumination system projects illumination light onto the subject's eye, and the imaging system captures an image of the subject's eye using an imaging element. The illumination system and the imaging system are configured to satisfy the Scheimpflug condition. The movement mechanism moves the illumination system and the imaging system. The program according to this embodiment causes a computer to concurrently execute the following three controls: (1) control of at least one of the illumination system and the imaging system to make the projection time of the illumination light onto the subject's eye shorter than the exposure time of the imaging element; (2) control of at least one of the illumination system and the imaging system to overlap at least a portion of the projection period of the illumination light onto the subject's eye with at least a portion of the exposure period of the imaging element; and (3) control of the illumination system, the imaging system, and the movement mechanism to cause the imaging system to collect a series of images of the subject's eye.

[0342] A program according to one embodiment causes a computer to control an ophthalmic apparatus. The ophthalmic apparatus includes an image acquisition unit. The image acquisition unit includes an optical system that projects illumination light onto the subject's eye and captures images using an image sensor, the optical system satisfying the Scheimpflug condition. The image acquisition unit further acquires a series of images while changing the projection position of the illumination light and the capture position. In other words, the image acquisition unit acquires a series of images by scanning the subject's eye while maintaining the optical system in a state that satisfies the Scheimpflug condition. The program according to this embodiment causes a computer to execute the following two controls in parallel to cause the image acquisition unit to acquire a series of images of the subject's eye: (1) control of the image acquisition unit (optical system) to make the projection time of illumination light onto the subject's eye shorter than the exposure time of the image sensor; and (2) control of the image acquisition unit (optical system) to overlap at least a portion of the projection period of illumination light onto the subject's eye with at least a portion of the exposure period of the image sensor.

[0343] A recording medium according to one embodiment is a computer-readable non-transitory recording medium having recorded thereon a program for causing a computer to control an ophthalmic apparatus. The ophthalmic apparatus includes an illumination system, an imaging system, and a movement mechanism. The illumination system projects illumination light onto the subject's eye, and the imaging system captures an image of the subject's eye using an imaging element. The illumination system and the imaging system are configured to satisfy the Scheimpflug condition. The movement mechanism moves the illumination system and the imaging system. The program recorded on the recording medium according to this embodiment causes a computer to concurrently execute the following three controls: (1) control of at least one of the illumination system and the imaging system to make the projection time of the illumination light onto the subject's eye shorter than the exposure time of the imaging element; (2) control of at least one of the illumination system and the imaging system to overlap at least a portion of the projection period of the illumination light onto the subject's eye with at least a portion of the exposure period of the imaging element; and (3) control of the illumination system, the imaging system, and the movement mechanism to cause the imaging system to collect a series of images of the subject's eye.

[0344] A recording medium according to one embodiment is a computer-readable non-transitory recording medium having a program recorded thereon that causes a computer to control an ophthalmic apparatus. The ophthalmic apparatus includes an image acquisition unit. The image acquisition unit includes an optical system that satisfies the Scheimpflug condition and projects illumination light onto the subject's eye and captures images with an image sensor. The image acquisition unit further acquires a series of images while changing the projection position of the illumination light and the capture position. In other words, the image acquisition unit acquires a series of images by scanning the subject's eye while maintaining the optical system in a state that satisfies the Scheimpflug condition. The program recorded on the recording medium according to this embodiment causes a computer to execute the following two controls in parallel to cause the image acquisition unit to acquire a series of images of the subject's eye: (1) control the image acquisition unit (optical system) to shorten the projection time of illumination light onto the subject's eye compared to the exposure time of the image sensor; and (2) control the image acquisition unit (optical system) to overlap at least a portion of the projection period of illumination light onto the subject's eye with at least a portion of the exposure period of the image sensor.

[0345] In addition, the computer-readable non-transitory recording medium that can be used as a recording medium according to the embodiment may be a recording medium of any form, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0346] Any feature described in the embodiment of the ophthalmic apparatus may be combined with an embodiment other than the ophthalmic apparatus. For example, any feature described above as an optional aspect of the ophthalmic apparatus according to the embodiment may be combined with an embodiment of a control method for the ophthalmic apparatus, an embodiment of a program, an embodiment of a recording medium, etc. Furthermore, any feature described in the present disclosure may be combined with an embodiment of a control method for the ophthalmic apparatus, an embodiment of a program, an embodiment of a recording medium, etc.

[0347] This disclosure presents several embodiments and several exemplary aspects thereof. These embodiments and aspects are merely examples of the present invention. Therefore, any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the present invention can be applied to the embodiments and aspects presented in this disclosure. [Explanation of symbols]

[0348] 1. Slit lamp microscope system (ophthalmic equipment) 2. Lighting system 3. Photography 3B image sensor 6 Moving mechanism 7 Control Unit 1000, 1500 ophthalmology equipment 1010 Image Acquisition Unit 1011 Lighting system 1012 Photography 1013 image sensor 1020 control section 1030 Evaluation processing unit

Claims

1. an illumination system that projects illumination light onto the subject's eye; an imaging system for imaging the subject's eye; a movement mechanism that moves the illumination system and the imaging system; a control unit that controls the illumination system, the imaging system, and the moving mechanism to cause the imaging system to collect a series of images; Including, the imaging system includes an imaging element, the illumination system and the imaging system are configured to satisfy the Scheimpflug condition, the control unit controls at least one of the illumination system and the imaging system in the control for causing the imaging system to collect the series of images so that a projection time of the illumination light onto the eye to be examined is shorter than an exposure time of the imaging element, and so that at least a part of a projection period of the illumination light onto the eye to be examined and at least a part of an exposure period of the imaging element overlap. Ophthalmology equipment.

2. the control unit controls the imaging system to repeatedly expose the image sensor in the control for causing the imaging system to collect the series of images. The ophthalmic device of claim 1.

3. the control unit controls the illumination system to modulate the intensity of the illumination light in parallel with the control of the imaging system to repeat the exposure of the image sensor. The ophthalmic device of claim 2.

4. the control unit executes the control of the illumination system so that the illumination light is intermittently projected onto the subject's eye. The ophthalmic apparatus of claim 3.

5. the control unit executes synchronization control between the illumination system and the imaging system in the control for causing the imaging system to collect the series of images. The ophthalmic device of claim 1.

6. the control unit executes the synchronization control so that a total time of the projection time and non-projection time of the illumination light is equal to a total time of the exposure time and non-exposure time of the image sensor. The ophthalmic device of claim 5.

7. the control unit controls the illumination system to change the projection time of the illumination light. The ophthalmic device of claim 1.

8. the control unit controls the illumination system based on the speed of the illumination system and the imaging system moved by the movement mechanism to change the projection time of the illumination light. The ophthalmic device of claim 7.

9. the control unit determines the speeds of the illumination system and the imaging system based on an imaging mode selected from two or more preset imaging modes. The ophthalmic device of claim 8.

10. the control unit controls the illumination system based on a photographing mode selected from two or more preset photographing modes to change the projection time of the illumination light. The ophthalmic device of claim 7.

11. the control unit controls the imaging system to change the exposure time of the imaging element. The ophthalmic device of claim 1.

12. the control unit controls the imaging system based on the speeds of the illumination system and the imaging system moved by the movement mechanism to change the exposure time of the image sensor. The ophthalmic device of claim 11.

13. the control unit determines the speeds of the illumination system and the imaging system based on an imaging mode selected from two or more preset imaging modes. The ophthalmic device of claim 12.

14. the control unit controls the imaging system based on a photographing mode selected from two or more preset photographing modes to change the exposure time of the image sensor. The ophthalmic device of claim 11.

15. an evaluation processing unit that generates evaluation information of the subject's eye based on the series of images collected by the imaging system, The ophthalmic device of claim 1.

16. the evaluation processing unit generates evaluation information of floaters present in the subject's eye as the evaluation information of the subject's eye. The ophthalmic device of claim 15.

17. the illumination light is a slit light; The ophthalmic device of claim 1.

18. A method for controlling an ophthalmic apparatus including an illumination system that projects illumination light onto an eye to be examined, an imaging system that images the eye to be examined using an image sensor, a movement mechanism that moves the illumination system and the imaging system, and a processor, wherein the illumination system and the imaging system are configured to satisfy the Scheimpflug condition, and causing the processor to control at least one of the illumination system and the imaging system so that a projection time of the illumination light onto the subject's eye is shorter than an exposure time of the imaging element and so that at least a part of the projection period of the illumination light onto the subject's eye and at least a part of the exposure period of the imaging element overlap, and the processor controls the illumination system, the imaging system, and the moving mechanism to cause the imaging system to collect a series of images. method.

19. A program that causes a computer to execute control of an ophthalmic apparatus including an illumination system that projects illumination light onto an eye to be examined, an imaging system that images the eye to be examined by an image sensor, and a movement mechanism that moves the illumination system and the imaging system, wherein the illumination system and the imaging system are configured to satisfy the Scheimpflug condition, and causing the computer to control at least one of the illumination system and the photographing system so that a projection time of the illumination light onto the eye to be examined is shorter than an exposure time of the image sensor, and so that at least a part of the projection period of the illumination light onto the eye to be examined and at least a part of the exposure period of the image sensor overlap, and controlling the illumination system, the photographing system, and the moving mechanism to cause the photographing system to collect a series of images. program.

20. A computer-readable non-transitory recording medium having recorded thereon a program for causing a computer to execute control of an ophthalmic apparatus including an illumination system that projects illumination light onto an eye to be examined, an imaging system that images the eye to be examined using an image sensor, and a movement mechanism that moves the illumination system and the imaging system, wherein the illumination system and the imaging system are configured to satisfy the Scheimpflug condition, and causing the computer to control at least one of the illumination system and the photographing system so that a projection time of the illumination light onto the eye to be examined is shorter than an exposure time of the image sensor, and so that at least a part of the projection period of the illumination light onto the eye to be examined and at least a part of the exposure period of the image sensor overlap, and controlling the illumination system, the photographing system, and the moving mechanism to cause the photographing system to collect a series of images. Recording medium.

Images