Ophthalmologic apparatus

Abstract

To provide an ophthalmologic apparatus capable of automatically performing alignment adjustment to a position where an eye to be examined is shown when the eye to be examined is not shown in an anterior eye part image in an imaging start mode for the anterior eye part.SOLUTION: An ophthalmologic apparatus A includes a body part 20, an anterior eye part stereo camera 22, a jaw receiving part 30, and a control part 60. The control part 60 includes an alignment control part 63 for executing control to adjust relative positional relationships between an eye E to be examined and the body part 20 on the basis of an anterior eye part image. The alignment control part 63 includes an imaging start mode control part 631 for, when it is determined that the eye E to be examined is not shown in the anterior eye part image in an anterior eye part imaging start mode by the anterior eye part stereo camera 22, automatically calculating an estimated position of the eye E to be examined on the basis of the image recognition for a face part shown in the anterior eye part image following the determination, and executing control to move an optometry part toward the calculated estimated position of the eye E to be examined.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to ophthalmic devices. [Background technology]

[0002] An ophthalmic apparatus is provided with a variable magnification means for changing the magnification at which an image of the subject's face including the external eye part of the subject's eye is observed by an observation means. The variable magnification means provides a high-magnification image to the observation means when it is detected that the subject's eye is on the optical axis of the measurement means, and provides a low-magnification image to the observation means when it is detected that the subject's eye is not on the optical axis of the measurement means (see Patent Document 1).

[0003] An ophthalmic device for examining a subject's eye sets a region of interest in a captured image based on the position of an eye examination unit moved by an adjustment means. A calculation processing means processes an image signal in the region of interest, and controls the photographing conditions of a face photographing means based on the image signal in the region of interest. An ophthalmic device characterized in that the movement of the eye examination means by the adjustment means is controlled based on a photographed image acquired under the controlled photographing conditions is known (see Patent Document 2).

[0004] The apparatus includes an information acquisition unit that acquires position information on at least one of the subject's eye and the optical head unit as output data from the trained model by using an image of the subject's eye acquired using the optical head unit as input data for the trained model. An ophthalmic device is known in which a drive control unit controls the drive of at least one of the support unit and the optical head unit based on the acquired position information, and moves at least one of the subject's eye and the optical head unit to the position (see Patent Document 3). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 8-224213 [Patent Document 2] Patent No. 6843627 [Patent Document 3] Patent No. 7194136 Summary of the Invention [Problem to be solved by the invention]

[0006] Incidentally, when measuring the eye characteristics of the subject's eye with the subject's chin supported on the chin rest, alignment control is performed to adjust the relative positional relationship between the subject's eye and the main body when the anterior eye image is started, provided that the subject's eye is reflected in the anterior eye image. However, the subject's face may not be reflected in the anterior eye image in some cases, such as when the subject's face is tilted and not facing forward, when the subject's chin is not properly placed on the chin rest, or when the initial height position of the chin rest is shifted due to the age difference from the previous subject. Therefore, when the subject's eye is not reflected in the anterior eye image in the shooting start mode, there is a problem that a preparatory work is required to make the subject's eye reflected in the anterior eye image, such as the examiner correcting the inclination of the subject's face or adjusting the alignment manually.

[0007] In particular, when the examiner measures the eye characteristics by remote operation from a position away from the subject or from another room, it is difficult for the examiner to visually check the relative positional relationship between the subject's eye and the main unit or to manually operate the main unit. Therefore, if the subject's eye is not captured in the anterior segment image during remote operation, the examiner needs to move to the subject's position to perform preparation work, which requires work steps including the examiner's position movement.

[0008] In contrast, the technology disclosed in Patent Document 1 requires a variable magnification means, and if there is no variable magnification means, the problem cannot be solved in the shooting start mode. Patent Document 2 discloses a technology for detecting the position of the subject's eye reflected in the anterior eye image from the face photographing unit, determining whether the subject's eye has been detected in the anterior eye image from the anterior eye photographing optical system, and performing alignment control. Patent Document 3 discloses a technology for acquiring position information of at least one of the subject's eye and the optical head unit using an image of the subject's eye acquired using an optical head unit and a trained model, and performing alignment control. However, the technologies disclosed in Patent Documents 2 and 3 are both alignment technologies that require the subject's eye to be reflected in the anterior eye image, and therefore cannot solve the above problem when the subject's eye is not reflected.

[0009] The present disclosure has been made with attention to the above-mentioned problems, and aims to provide an ophthalmic device that automatically performs alignment adjustment to a position where the test eye is reflected when the test eye is not reflected in the anterior eye image in the anterior eye shooting start mode. [Means for solving the problem]

[0010] In order to achieve the above object, the ophthalmic device of the present disclosure includes a main body having a built-in measurement optical system for measuring ocular characteristics of a subject's eye with the subject supporting his / her chin on a chin rest, an anterior eye camera provided in the main body and configured to capture an anterior eye image by photographing the anterior eye of the subject, and a control unit for controlling each unit of the device. The control unit includes an alignment control unit configured to control adjustment of a relative positional relationship between the subject's eye and the main body based on the anterior eye image. The alignment control unit includes a photographing start mode control unit configured to, when it is determined that the subject's eye is not captured in the anterior eye image in a photographing start mode of the anterior eye camera, calculate a predicted position of the subject's eye based on image recognition of a face part captured in the anterior eye image automatically following the determination, or automatically after at least a notification following the determination, and control the subject's eye to move toward the calculated predicted position of the subject's eye. Effect of the Invention

[0011] In the ophthalmologic apparatus of the present disclosure, in an anterior eye imaging start mode, if the subject's eye is not reflected in the anterior eye image, alignment adjustment can be automatically performed to a position where the subject's eye is reflected. [Brief description of the drawings]

[0012] [Figure 1] 1 is a perspective view showing an external configuration of an ophthalmologic apparatus according to a first embodiment, viewed obliquely from a chin rest side. [Diagram 2] 1 is a perspective view showing an external configuration of an ophthalmologic apparatus according to a first embodiment, viewed obliquely from a control panel side. [Diagram 3] 1 is a front view showing the external configuration of an ophthalmologic apparatus according to a first embodiment, as viewed from the chin rest side toward the front of a main body unit. [Figure 4] 1 is a side view showing a schematic configuration of built-in components and accessories of an ophthalmic apparatus according to a first embodiment. [Diagram 5] 2 is a block diagram showing a control system configuration in the ophthalmologic apparatus of the first embodiment. FIG. [Figure 6] 11 is a flowchart showing a flow of alignment control processing in a shooting start mode control unit in the first embodiment. [Figure 7] 13 is a diagram showing an example of a shooting start mode alignment image displayed on the display screen of the control panel during shooting start mode control. FIG. [Figure 8] 11A and 11B are explanatory diagrams showing an example of image movement by calculating the predicted position of the subject's eye from a shooting start mode alignment image in which the subject's eye is not shown, and an example of changing to a low magnification image when facial features are not detected. [Figure 9] 11 is an explanatory diagram showing an example of detection of characteristic parts of a subject's face in a shooting start mode alignment image in which the subject's eye is not shown. FIG. [Figure 10] FIG. 13 is a diagram showing an example of an alignment image of a pupil height position displayed on the display screen of the control panel during coarse alignment control. [Figure 11] FIG. 13 is a diagram showing an example of an auto-alignment image of a pupil position displayed on the display screen of the control panel during precision alignment control. [Figure 12]FIG. 13 is a diagram showing an example of a manual alignment image of the pupil position displayed on the display screen of the control panel during precision alignment control. [Figure 13] 13 is a flowchart showing a flow of alignment control processing in a shooting start mode control unit in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The embodiment for carrying out the ophthalmic device according to the present disclosure will be described based on the first and second embodiments shown in the drawings. The first and second embodiments are examples of application to an ophthalmic device that observes, photographs, and records an anterior segment image of a subject's eye, a fundus image of the subject's eye, and a tomographic image of the fundus of the subject's eye, and provides them as electronic images for diagnosis. In each drawing, the X-axis indicates the left-right axis in the left-right direction (horizontal direction) when facing the main body of the ophthalmic device with the subject's eye as a reference, the Y-axis indicates the up-down axis in the up-down direction (vertical direction), and the Z-axis indicates the front-rear axis in the front-rear direction (depth direction) perpendicular to the X-axis and Y-axis. EXAMPLES

[0014] [Overall configuration of the device (Figs. 1 to 4)] The ophthalmologic apparatus A is called a "3D fundus image capturing apparatus" and, as shown in Figures 1 to 4, comprises a stand unit 10, a main body unit 20, a chin rest unit 30, a control panel unit 40, a measurement optical system 50, and a control unit 60.

[0015] The ophthalmic device A includes a fundus camera that captures a fundus image of the subject's eye E, and an OCT (short for "Optical Coherence Tomography") that captures a fundus tomographic image of the subject's eye E. Here, the "fundus camera" refers to a camera that images the fundus condition of the retina, optic nerve, capillaries, etc. at the back of the subject's eye E and captures the fundus image. "OCT" refers to an optical coherence tomography that uses the interference of light to image the tomographic layer of the retina present at the fundus of the subject's eye E and captures the fundus tomographic image.

[0016] The stand 10 is placed on an eye examination table T (not shown) whose height is adjustable. A main body 20 is supported on the top surface of the stand 10 so as to be movable in three axial directions, namely, the X-axis, the Y-axis, and the Z-axis. A chin rest 30 is fixed to the front surface of the stand 10. A power switch 11, a power inlet 12, a USB terminal 13, and a LAN terminal 14 are provided on the side surface of the stand 10. Note that USB is an abbreviation for "Universal Serial Bus" and LAN is an abbreviation for "Local Area Network". The USB terminal 13 is a terminal for connecting an external memory, and as shown in FIG. 4, an HDD (abbreviation for "Hard Disk Drive"), a USB memory, or the like is connected to the USB terminal 13. A personal computer 16 on which dedicated software or the like is installed is connected to the LAN terminal 14 via a LAN cable 15.

[0017] 4, a power supply unit 17 and an XYZ drive unit 18 are built into the internal space of the gantry 10. The power supply unit 17 includes a power switch 11, a power inlet 12, a USB terminal 13, a LAN terminal 14, etc. The XYZ drive unit 18 is a motor actuator having a motor and a motor drive circuit that drives the main body unit 20 in three axial directions of the X, Y and Z axes when the main body unit 20 is moved relative to the gantry 10 in alignment control.

[0018] The main body 20 is provided so as to be movable in the X-axis, Y-axis and Z-axis directions by an XYZ drive unit 18 relative to the stand 10 to which the chin rest 30 is fixed. In the main body 20, a measurement optical system 50 for measuring ocular characteristics of the subject's eye E with the subject supporting his / her chin on the chin rest 30 is built into a main body cover 21 that covers the entire main body 20. A control panel unit 40 is disposed at an upper position on the back surface of the main body cover 21 as shown in Figs. 1, 2 and 4. In addition to the measurement optical system 50, a control unit 60 is built into the internal space of the main body cover 21 as shown in Fig. 4.

[0019] 3, the main body cover 21 has, at its center, an objective lens 51 of a measurement optical system 50 facing the subject's eye E at its front position. In addition, the main body cover 21 has, at its periphery, an anterior eye stereo camera 22 (anterior eye camera), a peripheral fixation lamp 23, and an anterior eye observation filter 24.

[0020] The anterior eye stereo camera 22 is a camera that captures an anterior eye image by photographing the anterior eye of the subject. The anterior eye stereo camera 22 is composed of two cameras, a first camera 22a and a second camera 22b, which are arranged on both sides of the objective lens 51 with the lens optical axis tilted toward the anterior eye of the subject's eye E, which is the measurement target. The first camera 22a and the second camera 22b are variable magnification cameras that can switch between high and low magnification and change the magnification steplessly, and capture a right anterior eye image and a left anterior eye image that are cut out of a part of the face of the subject supported by the chin rest 30 according to the selection of the subject's eye E and the angle of view at that time. In addition, since the anterior eye stereo camera 22 is composed of two cameras, the first camera 22a and the second camera 22b, which are arranged with a determined width dimension in the X-axis direction and a determined tilt angle, the three-dimensional coordinate position of the subject's eye E can be specified by calculation processing based on the two anterior eye images.

[0021] The peripheral fixation lights 23 are fixation lights used to fix the line of sight of the subject's eye E by turning them on, and eight of them are arranged at equal intervals around the outer periphery of the objective lens 51. The anterior segment observation filters 24 are filters used to adjust the amount of light during anterior segment observation or anterior segment OCT, and two of them (four in total) are arranged in the vertical direction outside the first camera 22a and the second camera 22b.

[0022] The chin rest 30 is provided so that its height position (position in the Y-axis direction) can be adjusted relative to the chin rest support 31 fixed to the pedestal 10, and supports the chin of the subject. The chin rest 30 has a lifting rod 30a that is lifted and lowered by a built-in chin rest drive 32, a chin rest 30b fixed to the upper end position of the lifting rod 30a, and chin rest paper stopper pins 30c provided on both sides of the chin rest 30b. The chin rest drive 32 is a motor actuator having a motor and a motor drive circuit that drives the lifting rod 30a in the Y-axis direction when the chin rest 30 is moved in the Y-axis direction relative to the chin rest support 31 (= the pedestal 10) in alignment control.

[0023] The chin rest support part 31 has a face support frame part 33 fixed at both ends of the T-shape, the face of the subject with the chin supported by the chin rest part 30, in three directions. A pair of vertical frames extending in the Y-axis direction of the face support frame part 33 are provided with height marks 33a as a guide for the height position of the subject's eye E. A horizontal frame connecting the upper ends of the pair of vertical frames of the face support frame part 33 is provided with a removable forehead rest surface 33b made of silicone rubber or the like. Furthermore, a multi-stage bendable arm 34 is provided at the central upper position of the horizontal frame of the face support frame part 33, and an external fixation target 35 is provided at the tip of the arm 34.

[0024] The control panel unit 40 is disposed at the upper rear position of the main body cover 21, and has a display screen 41 that displays in color an anterior eye image of the subject's eye E from the anterior eye stereo camera 22 and an anterior eye observation image of the subject's eye E from the measurement optical system 50. The display screen 41 is a touch panel that allows the examiner to input an input operation to the control unit 60 by touching the displayed button images and images with his / her finger. The connection support unit 42 of the control panel unit 40 to the main body unit 20 has a combined support structure of bending support and rotation support that allows the display screen 41 to be set at any position in the entire circumferential direction relative to the main body unit 20, and also allows the inclination angle of the display screen 41 to be freely set. In other words, the control panel unit 40 provided in the main body unit 20 is used when the examiner is close to the examinee to perform an eye characteristic test. For this reason, the connection support unit 42 ensures a function that allows the examiner to position the display screen 41 at a position that is easy for the examiner to operate, regardless of where the examiner is located around the ophthalmic device A.

[0025] When the examiner performs the eye characteristic examination by remote operation from a position distant from the examinee, for example, as shown in Fig. 2, a remote operation tablet 40' having a communication function with the main body 20 in addition to an input operation function equivalent to that of the control panel 40 is used. Therefore, the remote operation tablet 40' also has a display screen 41' using a touch panel.

[0026] The measurement optical system 50 measures the ocular characteristics of the subject's eye E while the subject supports his / her chin on the chin rest 30, and includes a fundus camera unit 52 having an objective lens 51 and an OCT unit 53, as shown in FIG. 4. The fundus camera unit 52 includes an illumination optical system and a photographing optical system, and is a unit constituting a fundus camera that acquires a fundus image of the subject's eye E using a lens, an image sensor, etc. The OCT unit 53 is a unit constituting an OCT that acquires a fundus tomographic image of the subject's eye E using a wavelength-variable light source, a fiber coupler, etc. The measurement optical system 50 can acquire a fundus image and a fundus tomographic image of the subject's eye E, as well as an anterior eye observation image of the subject's eye E.

[0027] The control unit 60 controls each part of the device (fundus camera unit 52, OCT unit 53, chin rest unit 30, main body unit 20, etc.) based on various input operations including a touch operation on the display screen 41 of the control panel unit 40. As shown in Fig. 4, the control unit 60 has, as its hardware configuration, a control board 60a, a CPU board 60b, and an image board 60c.

[0028] [Control system configuration (Fig. 5)] The control system of the ophthalmologic apparatus A includes a control panel unit 40 (display unit 41), a measurement optical system 50 (fundus camera unit 52, OCT unit 53), and a control unit 60, as shown in FIG.

[0029] The control unit 60 includes a main control unit 61 that controls the fundus camera unit 52 and the OCT unit 53, a storage unit 62 that stores necessary data, and an alignment control unit 63. The alignment control unit 63 performs alignment control to adjust the relative positional relationship between the subject's eye E and the main body unit 20 (the objective lens 51 of the main body unit 20) based on the anterior eye image acquired by the anterior eye stereo camera 22. The alignment control unit 63 includes a photography start mode control unit 631, a coarse alignment control unit 632, and a fine alignment control unit 633. Hereinafter, the coarse alignment control and the fine alignment control that are performed when the subject's eye is reflected in the anterior eye image will be collectively referred to as "normal alignment control."

[0030] The alignment control unit 63 obtains anterior eye images by photographing the left and right sides of the subject's face from two directions using the anterior eye stereo camera 22 (first camera 22a, second camera 22b), and obtains anterior eye observation images by photographing the right and left eyes using the measurement optical system 50. The alignment control unit 63 adjusts the relative positional relationship between the subject's eye E and the objective lens 51 provided in the main body unit 20 by a drive command output to at least one of the XYZ drive unit 18 and the chin rest drive unit 32. Here, the XYZ drive unit 18 and the chin rest drive unit 32 are used when the adjustment movement amount is only the XYZ axis direction movement amount. On the other hand, when the adjustment movement amount includes the Y axis direction movement amount, the XYZ drive unit 18 and the chin rest drive unit 32 are used separately because the Y axis movement allowable range of the chin rest drive unit 32 is wider than the XYZ drive unit 18 movement allowable range. For example, during Y-axis movement, the chin rest driver 32 is used for coarse alignment control, and the XYZ driver 18 is used for fine alignment control.

[0031] When the imaging start mode control unit 631 determines that the subject's eye E is not captured in the anterior eye image during imaging start mode of the anterior eye using the anterior eye stereo camera 22, the imaging start mode control unit 631 automatically calculates a predicted position of the subject's eye E based on image recognition of the face part captured in the anterior eye image following the determination. Then, the imaging start mode control unit 631 performs control to move the subject's eye E toward the calculated predicted position. Furthermore, after starting control to move the subject's eye E toward the predicted position of the subject's eye E, when the subject's eye E is detected in the anterior eye image from the anterior eye stereo camera 22, the imaging start mode control unit 631 performs control to stop the movement at the detection stage and switch to normal alignment control.

[0032] In addition, when it is determined that the subject's eye E is not captured in the anterior eye image and the position of the subject's eye E cannot be predicted, the photography start mode control unit 631 lowers the magnification of the anterior eye stereo camera 22 and again acquires an anterior eye image from the anterior eye stereo camera 22. Here, in the first embodiment, "when the position of the subject's eye E cannot be predicted" refers to a case where a facial feature used for predicting the position of the subject's eye E is not detected from the anterior eye image.

[0033] When it is confirmed that the subject's eye E is reflected in the displayed anterior eye image, the coarse alignment control unit 632 performs rough alignment control so that the position of the pupil of the subject's eye E is located near the center of the image. This coarse alignment control is performed manually by the examiner while viewing the displayed anterior eye image and the anterior eye observation image.

[0034] When it is confirmed that the pupil position is near the center of the displayed image, the precision alignment control unit 633 performs alignment control for the pupil so that the pupil position is at the center of the image. This precision alignment control is performed by automatic control based on two anterior eye images from the anterior eye stereo camera 22, but it can also be performed by manual operation at the examiner's choice.

[0035] [Processing configuration and operation of shooting start mode control (Fig. 6)] The processing configuration of the imaging start mode control executed in the imaging start mode control unit 631 will be described with reference to the flowchart shown in Fig. 6. The imaging start mode control processing is started by a predetermined operation after it is confirmed that the subject sits in front of the ophthalmologic apparatus A with the power switch turned on and supports the chin on the chin rest 30. Here, the "predetermined operation" refers to, for example, tapping the capture start button 437 on the imaging start auto adjustment mode screen 43 (see Fig. 7) switched by tapping the imaging eye selection button on the imaging icon selection screen not shown.

[0036] In step S1, following the start, the anterior eye stereo camera 22 starts photographing the anterior eye, and the process proceeds to step S2. When the processing operation of the photographing start mode control starts, the anterior eye stereo camera 22 starts photographing the anterior eye, and after the start of photographing the anterior eye image, acquisition of the anterior eye image by a moving image continues.

[0037] In step S2, following the anterior eye photographing in step S1, or the adjustment movement to the position of the subject eye in step S6, or the low magnification photographing in step S7, it is determined whether or not the subject eye E is detected in the anterior eye image displayed on the photographing start auto adjustment mode screen 43. If it is determined as YES in step S2 (subject eye E is detected), the process proceeds to step S3. On the other hand, if it is determined as NO in step S2 (subject eye E is not detected), the process proceeds to step S4.

[0038] Here, the determination of whether the subject's eye E is detected in the anterior eye image is made by determining whether the pupil of the subject's eye E is reflected in the anterior eye image. That is, image processing is performed to convert the anterior eye image into a luminance image representing the level of luminance, and if a circular pupil with the lowest luminance is detected in the converted luminance image, it is determined that the subject's eye E is reflected in the anterior eye image, and that the subject's eye E is detected in the anterior eye image. On the other hand, if a circular pupil with the lowest luminance is not detected in the luminance image, it is determined that the subject's eye E is not reflected in the anterior eye image, and that the subject's eye E is not detected in the anterior eye image.

[0039] Furthermore, the determination in step S2 that the subject's eye E is detected in the anterior eye image means that the subject's eye E (pupil) is detected in the anterior eye image from both the first camera 22a and the second camera 22b. Therefore, even if the subject's eye E (pupil) is detected in only the anterior eye image from one of the first camera 22a and the second camera 22b, it is determined in step S2 that the subject's eye E is not detected in the anterior eye image.

[0040] In step S3, following the determination that the subject's eye E is detected in step S2, normal alignment control, that is, coarse alignment control is executed, followed by fine alignment control, and the process proceeds to END.

[0041] In step S4, following the determination in step S2 that the subject's eye E is not detected, a machine learning detection process is performed to determine whether or not facial features have been detected using an anterior eye image in which the subject's eye E is not shown and a trained feature detection model. If the determination is YES (feature detection) in step S4, the process proceeds to step S5. On the other hand, if the determination is NO (feature non-detection) in step S4, the process proceeds to step S7. That is, if it is determined that the subject's eye E is not shown in the anterior eye image, the trained feature detection model is used to automatically detect facial feature parts extracted from the anterior eye image in which the subject's eye E is not shown. Here, the "facial feature parts" refers to parts of the face (e.g., eyebrows, contour, nose, mouth, ears, etc.) that can be used to identify a part of the face other than the subject's eye E that is partially shown by image recognition, thereby predicting the position of the subject's eye E based on its positional relationship in the overall face image.

[0042] The "trained feature part detection model" refers to a model that is constructed in advance by executing machine learning using a machine learning data set that is generated by associating feature part image data with feature part information and a selected machine learning model. Here, the "feature part image data" refers to image data acquired by cutting out facial feature parts other than the eyes from a large number of face image data acquired as samples. The "feature part information" refers to information that indicates the name of the facial feature part represented by the acquired image data and its position relative to the entire face image. The "machine learning model" refers to a model selected from various machine learning algorithm models according to the required accuracy level of image recognition that recognizes the facial feature parts. As the "machine learning model", for example, a "convolutional neural network model" that uses a deep learning method for image recognition and is capable of detecting local features of an image is selected. The "trained feature part detection model" is constructed in advance by the personal computer 16, for example, and is stored and set so as to be readable by connection via the LAN cable 15 during control processing by the shooting start mode control unit 631. The trained feature part detection model may be updated as appropriate by changing or adding the machine learning data set.

[0043] In step S5, following the determination that the facial characteristic parts are detected in step S4, the predicted position of the subject's eye E is calculated, and the process proceeds to step S6. That is, when it is determined that the subject's eye E is not included in the anterior eye image, if the facial characteristic parts are detected from the anterior eye image in which the subject's eye E is not included, the predicted position of the subject's eye E is calculated based on the positional relationship between the detected characteristic parts and the overall face image of the subject's eye E. For example, in a two-dimensional coordinate plane of the XY axes, when the center position of the anterior eye image in which the subject's eye E is not included is set as the reference position (xo, yo), the predicted position of the subject's eye E is calculated as a predicted target position (xt, yt) which is the predicted pupil position of the subject's eye E in the overall face image.

[0044] In step S6, following the calculation of the predicted position of the subject's eye in step S5, the subject's eye is adjusted to the calculated predicted position, and the process proceeds to step S2. Here, the "adjustment movement to the predicted position of the subject's eye E" uses the XYZ drive unit 18 for movement in the X-axis direction, uses the XYZ drive unit 18 for movement in the Y-axis direction if the Y-axis movement amount is less than a predetermined amount, and uses the chin rest drive unit 32 if the Y-axis movement amount is greater than the predetermined amount. When the chin rest drive unit 32 is used to move the chin rest unit 30 up and down in the Y-axis direction, a voice announcement is made in advance to inform the subject that the chin rest unit 30 will be moved up or down so as not to surprise or cause discomfort to the subject due to a sudden movement.

[0045] In step S7, following the determination that no facial feature was detected in step S4, the magnification of the anterior eye stereo camera 22 is reduced for photographing, an anterior eye image is obtained again from the anterior eye stereo camera 22, and the process proceeds to step S2. That is, by photographing with the anterior eye stereo camera 22 at a reduced magnification, the angle of view of the anterior eye image is expanded overall compared to the angle of view of an anterior eye image at a normal high magnification, and if the subject places his / her chin on the chin rest 30, the subject's eye E can be reflected in the anterior eye image.

[0046] Next, the processing operation of the photography start mode control when starting to photograph the anterior eye segment with the anterior eye segment stereo camera 22 will be described with reference to FIG.

[0047] In step S1, the anterior eye is photographed, and in the next step S2, when the subject's eye E is detected in the anterior eye image at the start of photographing, the flow chart of Fig. 6 proceeds from step S1 to step S2 to step S3 to end. That is, when the subject's eye E is detected in the anterior eye image in step S2, the flow proceeds to step S3, where the coarse alignment control is performed by the coarse alignment control unit 632, followed by the fine alignment control by the fine alignment control unit 633. After the alignment control is completed, for example, autofocus control of the fundus is performed.

[0048] On the other hand, if the anterior eye is photographed in step S1 and the subject's eye E is not detected in the anterior eye image at the start of photographing in the next step S2, the flow chart of FIG. 6 proceeds from step S1 to step S2 to step S4. In step S4, a machine learning detection process using an anterior eye image in which the subject's eye E is not photographed and a learned feature detection model is used to determine whether or not a facial feature has been detected. If it is determined that a facial feature has been detected in step S4, the process proceeds from step S4 to step S5, and in step S5, a predicted position of the subject's eye E is calculated. In the next step S6, the subject's eye E is adjusted and moved to the calculated predicted position, and from step S6, the process returns to step S2, and in step S2, it is determined whether or not the subject's eye E has been detected in the anterior eye image by adjusting and moving the subject's eye E to the predicted position.

[0049] While the subject's eye E is not detected in the anterior eye image despite the adjustment movement of the subject's eye E to the predicted position in step S2, the flow of step S2 → step S4 → step S5 → step S6 is repeated. Then, when the subject's eye E is detected in the anterior eye image in step S2 due to an increase in the adjustment movement amount of the subject's eye E to the predicted position, the adjustment movement of the subject's eye E to the predicted position is stopped, and the flow proceeds from step S2 to step S3 → end.

[0050] Furthermore, if it is determined in step S2 that the subject's eye E is not detected in the anterior eye image at the start of shooting, and in step S4 that the facial feature required for predicting the position of the subject's eye E is not detected, the process proceeds from step S4 to step S7. In step S7, the magnification of the anterior eye stereo camera 22 is lowered for shooting, and an anterior eye image is acquired again from the anterior eye stereo camera 22, and the process proceeds to step S2. Then, in step S2, if the subject's eye E is detected in the anterior eye image by shooting with the anterior eye stereo camera 22 at the lower magnification, the process proceeds from step S2 to step S3 → end.

[0051] [Shooting start mode control function (Figs. 7 to 9)] The shooting start mode control operation will be described with reference to Fig. 7 to Fig. 9. In the shooting start mode control by automatic alignment, the display screen 41 of the control panel unit 40 or the display screen 41' of the remote operation tablet 40' becomes a shooting start auto adjustment mode screen 43 shown in Fig. 7.

[0052] The shooting start auto adjustment mode screen 43 has a menu button 431, a patient ID display area 432, a shooting information display area 433, a first moving image area 434, a second moving image area 435, a processing message display section 436, and a capture start button 437. When the menu button 431 is tapped, it transitions to a shooting icon selection screen. The patient ID display area 432 displays the patient ID. The shooting information display area 433 displays information on the photographed eye and the angle of view. The first moving image area 434 displays the anterior eye moving image from the first camera 22a of the anterior eye stereo camera 22. The second moving image area 435 displays the anterior eye moving image from the second camera 22b of the anterior eye stereo camera 22. The processing message display section 436 displays the current processing state ("Auto alignment in progress"). When the capture start button 437 is tapped, shooting starts. Note that "patient ID" refers to "subject ID".

[0053] For example, in the anterior eye image displayed in the first video area 434 and the anterior eye image displayed in the second video area 435, the subject's eye E (pupil) is not reflected in both the anterior eye images, but eyebrows B are reflected in both the anterior eye images, as shown in Fig. 7. In this case, eyebrows B, which are a facial feature, are detected from both the anterior eye images in the first video area 434 and the second video area 435. Then, as shown in the left part of Fig. 8, a predicted position of the subject's eye E is calculated based on the positional relationship between the detected eyebrows B and the subject's eye E in the overall face image, and the subject's eye E is adjusted and moved to the calculated predicted position. If this adjustment movement in the X-axis and Y-axis directions to the predicted position of the subject's eye E is replaced with the movement of the anterior eye image, the first video area 434 moves in the direction indicated by arrow C in Fig. 8, and becomes the first video area 434'. Then, when the movement amount in the direction indicated by the arrow C reaches a predetermined amount by adjusting the movement of the subject's eye E to the predicted position, the subject's eye E (pupil) is detected in the anterior eye image in the first video area 434'. Here, the facial feature may be any part of the face other than the subject's eye E, and is not limited to the eyebrows B, and may of course be the facial outline F, nose, and mouth N as shown in Fig. 9, or may be other parts such as the ears.

[0054] Furthermore, when the subject's eye E is not captured in the anterior eye image and a facial feature cannot be detected, the magnification of the anterior eye stereo camera 22 is lowered and an anterior eye image is acquired again from the anterior eye stereo camera 22. That is, by lowering the magnification of the anterior eye stereo camera 22, as shown in the right part of FIG. 8, the angle of view of the anterior eye image displayed in the first video area 434 is enlarged to the angle of view of the first video area 434'', and the subject's eye E is included in the angle of view of the anterior eye image displayed in the first video area 434''.

[0055] Incidentally, the background technology of alignment control in an ophthalmic apparatus is a control executed when the condition that at least the subject's eye E is reflected in the anterior eye image is satisfied, and when the subject's eye E is not reflected, it is not subject to alignment control and is left to the examiner's response. In contrast, the present inventors have focused on the fact that even in a situation where the subject's eye E is not reflected in the anterior eye image, it can be included in the alignment control as long as the facial part reflected in the anterior eye image can be grasped by image recognition.

[0056] That is, when it is determined that the subject's eye E is not reflected in the anterior eye image during the anterior eye imaging start mode by the anterior eye stereo camera 22, the alignment control unit 63 automatically calculates a predicted position of the subject's eye E based on image recognition of the face part reflected in the anterior eye image following the determination. Then, a configuration is adopted in which a photographing start mode control unit 631 performs control to move the subject's eye E toward the calculated predicted position.

[0057] Therefore, in the anterior eye imaging start mode, if the subject's eye E is not reflected in the anterior eye image, the XY axis alignment adjustment can be automatically performed to the position where the subject's eye E is reflected. As a result, the examiner does not need to perform a preparation work to make the subject's eye E appear in the anterior eye image by correcting the inclination of the subject's face or by manually adjusting the alignment, and the burden on the examiner is reduced. Furthermore, when the examiner measures the eye characteristics from a position away from the subject or by remote operation from another room, the examiner does not need to move to the subject's position to perform the preparation work, which reduces the number of work steps. Furthermore, compared to Patent Document 1, even if there is no magnification means, the XY axis automatic alignment adjustment can be performed to the position where the subject's eye E is reflected by predicting the position of the subject's eye E.

[0058] [Coarse alignment control function (Fig. 10)] The coarse alignment control action executed following the photography start mode control for starting photography of the anterior eye segment by the anterior eye segment stereo camera 22 will be described with reference to Fig. 10. The coarse alignment control is performed by the examiner's manual operation based on the coarse manual adjustment mode screen 44 shown in Fig. 10, which is displayed on the control panel unit 40 or the remote operation tablet 40'.

[0059] The rough manual adjustment mode screen 44 has a menu button 441, a patient ID display area 442, an imaging information display area 443, an operation method guide 444, a first video area 445, and a second video area 446. It further has a chin rest up and down movement button 447, a photographed eye selection button 448, an external fixation button 449, an advance button 450, and a capture start button 451. The operation method guide 444 displays the operation method of this screen. The first video area 445 displays an anterior eye observation image from the measurement optical system 50. The second video area 446 displays an image of the anterior eye stereo camera 22 closer to the photographed eye. The chin rest up and down movement button 447 moves the chin rest part 30 up and down by touch operation. The photographed eye selection button 448 selects the photographed eye. When tapped, the external fixation button 449 switches to the external fixation target 35, displays the ON / OFF state of the external fixation target 35, and switches ON / OFF of the external fixation target 35. When tapped, the advance button 450 switches to the advance mode.

[0060] The coarse alignment control is performed by Y-axis manual alignment adjustment using the anterior eye image and XY-axis manual alignment adjustment using the anterior eye observation image. The Y-axis manual alignment adjustment using the anterior eye image is performed by manually operating the chin rest up / down movement button 447 so as to align the height mark 33a of the chin rest part 30 with the height of the subject's eye E, using the lines L1 and L2 displayed in the second video area 446 as guides. The XY-axis manual alignment adjustment using the anterior eye observation image is performed by touching the pupil of the subject's eye E shown in the anterior eye observation image displayed in the first video area 445 so that the pupil of the anterior eye observation image fits into the frame G.

[0061] [Precise alignment control function (Fig. 11, Fig. 12)] The precision alignment control operation executed following the coarse alignment control will be described with reference to Figures 11 and 12. There are two types of precision alignment control: precision auto-alignment control for performing auto-alignment with respect to the pupil, and precision manual alignment control for performing manual alignment with respect to the pupil.

[0062] In the precise auto-alignment control, the display screen 41 of the control panel unit 40 or the display screen 41' of the remote operation tablet 40' becomes a precise auto-adjustment mode screen 46 shown in FIG. 11. The precise auto-adjustment mode screen 46 has a menu button 461, a patient ID display area 462, an imaging information display area 463, a video area 464, a manual mode button 465, and a processing message 466. The video area 464 displays an anterior eye image from the first camera 22a of the anterior eye stereo camera 22 and an anterior eye image from the second camera 22b in an upper area and a lower area. When the manual mode button 465 is tapped, the automatic alignment adjustment is stopped, and the screen is switched to a later-described precise manual adjustment mode screen 47 in which the photographed eye is adjusted manually. The processing message 466 displays the currently processing state ("auto-alignment in progress").

[0063] In the precise auto-alignment control for the pupil, the XY-axis alignment adjustment is automatically performed by the XYZ drive unit 18 so that the pupil mark M1 in the upper region and the pupil mark M2 in the lower region displayed in the video area 464 coincide with each other. The pupil marks M1 and M2 are marks that indicate the pupil position in the anterior eye image.

[0064] Next, the precision manual alignment control action executed by tapping the manual mode button 465 in the precision automatic alignment control will be described with reference to Fig. 12. When the manual mode button 465 is tapped, the display screen 41 of the control panel unit 40 or the display screen 41' of the remote operation tablet 40' switches from the precision automatic adjustment mode screen 46 shown in Fig. 11 to a precision manual adjustment mode screen 47 shown in Fig. 12.

[0065] The fine manual adjustment mode screen 47 has a menu button 471, a patient ID display area 472, an imaging information display area 473, an operation method guide 474, a first moving image area 475, and a second moving image area 476. It further has a processing message 477, a back button 478, and a capture start button 479. The operation method guide 474 displays the operation method of this screen. The first moving image area 475 displays an anterior eye image from the first camera 22a of the anterior eye stereo camera 22. The second moving image area 476 displays an anterior eye image from the second camera 22b of the anterior eye stereo camera 22. The processing message 477 displays the current processing status ("Manual Alignment"). The back button 478, when tapped, returns to the rough manual adjustment mode screen 44.

[0066] In the precise manual alignment control for the pupil, the pupil mark M1 of the anterior eye image displayed in the first video area 475 and the pupil mark M2 of the anterior eye image displayed in the second video area 476 are tapped. Based on this tapping operation, the XYZ driving unit 18 performs XY axis alignment adjustment so that the pupil mark M1 and the pupil mark M2 are each positioned at the center position of the anterior eye image.

[0067] As described above, the ophthalmologic apparatus A of the first embodiment provides the following advantages. (1) An ophthalmic device A including a main body 20, an anterior eye camera (anterior eye stereo camera 22), and a control unit 60. The control unit 60 includes an alignment control unit 63 that performs control to adjust the relative positional relationship between the subject's eye E and the main body 20 based on an anterior eye image. The alignment control unit 63 has a photographing start mode control unit 631 that, when it is determined that the subject's eye E is not reflected in the anterior eye image in an anterior eye photographing start mode using the anterior eye camera, automatically calculates a predicted position of the subject's eye E based on image recognition of a face part reflected in the anterior eye image, and controls the subject's eye E to move toward the calculated predicted position. Therefore, when the subject's eye E is not reflected in the anterior eye image in an anterior eye photographing start mode, alignment adjustment can be automatically performed to a position where the subject's eye E is reflected.

[0068] (2) After starting control to move the subject's eye E toward the predicted position of the subject's eye E, when the subject's eye E is detected in the anterior eye image from the anterior eye camera (anterior eye stereo camera 22), the photographing start mode control unit 631 stops the movement at the detection stage and switches to normal alignment control. Therefore, when the subject's eye E is detected in the anterior eye image by the photographing start mode control, it is possible to transition from the photographing start mode control to the normal alignment control without changing the existing normal alignment control.

[0069] (3) When it is determined that the subject's eye E is not included in the anterior eye image and the position of the subject's eye E cannot be predicted, the imaging start mode control unit 631 lowers the magnification of the anterior eye camera (anterior eye stereo camera 22) and acquires an anterior eye image from the anterior eye camera again. Therefore, when the subject's eye E is not included in the anterior eye image and the position of the subject's eye E cannot be predicted, the angle of view of the anterior eye image is enlarged, thereby increasing the probability that the subject's eye E will be detected in the anterior eye image.

[0070] (4) When it is determined that the subject's eye E is not shown in the anterior eye image, the shooting start mode control unit 631 detects facial feature parts from the anterior eye image in which the subject's eye E is not shown, and calculates the predicted position of the subject's eye E from the positional relationship between the detected feature parts and the overall face image of the subject's eye E. Therefore, based on the detection of the facial feature parts, the predicted position of the subject's eye E can be calculated with high accuracy from the positional relationship between the detected feature parts and the overall face image of the subject's eye E.

[0071] (5) The imaging start mode control unit 631 has a trained feature part detection model that has been constructed in advance by executing machine learning using a machine learning dataset of a large number of generated feature part images and a selected machine learning model, and detects facial feature parts based on image recognition using an anterior eye image that does not include the subject's eye E and the trained feature part detection model. Therefore, the image recognition method using the trained feature part detection model can accurately detect facial feature parts from an anterior eye image that does not include the subject's eye E. EXAMPLES

[0072] The second embodiment is an example in which facial feature parts are individually detected by image recognition, and the photographing start mode control is performed by setting a detection order in which the detection of eyebrows B has a higher priority than the detection of contour F and the detection of nose and mouth N. Note that the configuration of the ophthalmologic apparatus A in the second embodiment is the same as the configuration shown in Figs. 1 to 5 in the first embodiment, and therefore illustration and description thereof will be omitted.

[0073] [Processing configuration and operation of shooting start mode control (Fig. 13)] A process configuration of the shooting start mode control executed in the shooting start mode control unit 631 of the second embodiment will be described with reference to the flowchart shown in Fig. 13. Note that steps S21, S22, S23, S25, S26, and S27 in Fig. 13 are steps that perform the same processes as steps S1, S2, S3, S5, S6, and S7 in Fig. 6, and therefore description thereof will be omitted.

[0074] In step S24a, following the determination in step S22 that the subject's eye is not detected, a machine learning detection process is performed using an anterior eye image in which the subject's eye E is not shown and a trained eyebrow detection model (trained individual part detection model) to determine whether or not eyebrow B is detected. If step S24a determines YES (eyebrow B is detected), the process proceeds to step S25. On the other hand, if step S24a determines NO (eyebrow B is not detected), the process proceeds to step S24b. That is, if the subject's eye E is not shown in the anterior eye image, the top priority is given to detecting eyebrow B from the anterior eye image in which the subject's eye E is not shown using the trained eyebrow detection model. Then, the detection order of individual feature parts is specified such that when eyebrow B is not detected, the contour F is detected, and when the contour F is not detected, the nose and mouth N are detected.

[0075] Here, the "trained individual part detection model" refers to a model that is constructed in advance by dividing the characteristic parts into the individual parts of eyebrows B, contour F, and nose and mouth N, and executing machine learning using a machine learning dataset generated in large numbers for each part, and a selected machine learning model. In other words, as the "trained individual part detection model", a "trained eyebrow detection model", a "trained contour detection model", and a "trained nose and mouth detection model" that are specialized for detecting the individual parts of eyebrows B, contour F, and nose and mouth N are prepared in advance (see Figure 9).

[0076] In step S24b, following the determination of non-detection of eyebrow B in step S24a, a machine learning detection process is performed using an anterior eye image that does not include the subject's eye E and a trained contour detection model to determine whether or not a contour F has been detected. If the determination in step S24b is YES (contour F is detected), the process proceeds to step S25. On the other hand, if the determination in step S24b is NO (contour F is not detected), the process proceeds to step S24c.

[0077] In step S24c, following the non-detection determination of the contour F in step S24b, a machine learning detection process is performed using an anterior eye image that does not include the subject's eye E and a trained nose and mouth detection model to determine whether or not the nose and mouth N are detected. If the determination in step S24c is YES (nose and mouth N detected), the process proceeds to step S25. On the other hand, if the determination in step S24c is NO (nose and mouth N not detected), the process proceeds to step S27.

[0078] Next, the processing operation of the photography start mode control when starting to photograph the anterior eye segment with the anterior eye segment stereo camera 22 will be described with reference to FIG.

[0079] In step S21, the anterior eye is photographed, and in the next step S22, when the subject's eye E is detected in the anterior eye image at the start of photographing, the flow proceeds from step S21 to step S22 to step S23 to end in the flowchart of Fig. 13. That is, when the subject's eye E is detected in the anterior eye image in step S2, the flow proceeds to step S3, where the coarse alignment control is performed by the coarse alignment control unit 632, followed by the fine alignment control by the fine alignment control unit 633.

[0080] On the other hand, if the anterior eye is photographed in step S21 and the subject's eye E is not detected in the anterior eye image at the start of photographing in the next step S22, the flow chart of FIG. 13 proceeds from step S21 to step S22 to step S24a. In step S24a, whether or not the eyebrow B is detected is determined by a machine learning detection process using an anterior eye image in which the subject's eye E is not reflected and a trained eyebrow detection model. If it is determined that the eyebrow B is detected in step S24a, the flow proceeds from step S24a to step S25, where the predicted position of the subject's eye E is calculated. In the next step S26, the subject's eye E is adjusted and moved to the calculated predicted position, and from step S26, the flow returns to step S22, and in step S22, it is determined whether or not the subject's eye E is detected in the anterior eye image by adjusting and moving the subject's eye E to the predicted position. Then, when the subject's eye E is detected in the anterior eye image in step S22, the adjustment movement of the subject's eye E to the predicted position is stopped, and the process proceeds from step S22 to step S23 →END.

[0081] Next, if it is determined in step S22 that the subject's eye E is not detected in the anterior eye image at the start of shooting, and in step S24a that the eyebrows B are not detected, the process proceeds from step S24a to step S24b. In step S24b, it is determined whether or not the contour F is detected by a machine learning detection process using an anterior eye image in which the subject's eye E is not reflected and a learned contour detection model. If it is determined in step S24b that the contour F is detected, the process proceeds from step S24b to step S25, in which the predicted position of the subject's eye E is calculated. In the next step S26, the subject's eye E is adjusted and moved to the calculated predicted position, and from step S26, the process returns to step S22, and in step S22, it is determined whether or not the subject's eye E is detected in the anterior eye image by adjusting and moving the subject's eye E to the predicted position. Then, when the subject's eye E is detected in the anterior eye image in step S22, the adjustment movement of the subject's eye E to the predicted position is stopped, and the process proceeds from step S22 to step S23 →END.

[0082] Next, if it is determined in step S22 that the subject's eye E is not detected in the anterior eye image at the start of shooting and in step S24b that the contour F is not detected, the process proceeds from step S24b to step S24c. In step S24c, whether or not the nose and mouth N are detected is determined by a machine learning detection process using an anterior eye image in which the subject's eye E is not reflected and a trained nose and mouth detection model. If it is determined in step S24c that the nose and mouth N are detected, the process proceeds from step S24c to step S25, where the predicted position of the subject's eye E is calculated. In the next step S26, the subject's eye E is adjusted and moved to the calculated predicted position, and from step S26, the process returns to step S22, and in step S22, it is determined whether or not the subject's eye E is detected in the anterior eye image by adjusting and moving the subject's eye E to the predicted position. Then, when the subject's eye E is detected in the anterior eye image in step S22, the adjustment movement of the subject's eye E to the predicted position is stopped, and the process proceeds from step S22 to step S23 →END.

[0083] Next, if it is determined in step S24c that the nose and mouth N are not detected, the process proceeds from step S24c to step S27. In step S27, the magnification of the anterior eye stereo camera 22 is lowered for photographing, and an anterior eye image is acquired again from the anterior eye stereo camera 22, and the process proceeds to step S22. Then, in step S22, if the subject's eye E is detected in the anterior eye image photographed with the anterior eye stereo camera 22 at the lower magnification, the process proceeds from step S22 to step S23 and then to the end.

[0084] [Shooting start mode control function] In the second embodiment, facial characteristic parts are detected separately by image recognition, and the detection order is set to give priority to the detection of eyebrows B over the detection of the contour F and the detection of the nose and mouth N. The reason for this is based on the inventor's finding that when the subject's eye E is not detected in the anterior eye image, eyebrows B are most frequently detected among the facial characteristic parts.

[0085] That is, when the face of the subject is tilted and not facing forward, even if one of the subject's left and right eyes, the subject's eye E, which is at a higher height position, is detected in the anterior eye image, the other of the left and right eyes, which is at a lower height position, is not detected in the anterior eye image, but the eyebrow B is detected. Also, when the previous subject was an adult and the current subject is a child, the chin rest part 30 is set at a height position in the Y-axis direction at which the left and right eyes E are detected in the anterior eye image of the adult's face, and the subject is then switched to a child, the left and right eyebrows B are detected in the anterior eye image.

[0086] Therefore, by setting the detection order to give priority to the detection of eyebrows B over the detection of the contour F and the detection of the nose and mouth N, it is possible to detect eyebrows B at an earlier timing than when one of multiple characteristic parts in the face is extracted and detected by image recognition. Furthermore, when eyebrows B are detected at an earlier timing, the time required to end the photographing start mode control after detecting eyebrows B, through the calculation of the predicted position of the eye E to be examined and the adjustment movement to the predicted position, can be shortened.

[0087] As described above, the ophthalmologic apparatus A of the second embodiment has the following advantages in addition to the advantages (1) to (4) of the first embodiment.

[0088] (6) The shooting start mode control unit 631 has learned individual part detection models for the eyebrows B, the contour F, the nose, and the mouth N that have been constructed in advance by executing machine learning using a machine learning dataset of many generated characteristic part images and a selected machine learning model. When detecting facial characteristic parts based on image recognition using an anterior eye image in which the subject's eye E is not shown and the learned individual part detection model, the order of detection for each characteristic part is set such that detection of the eyebrows B takes precedence over detection of the contour F and detection of the nose and mouth N. Therefore, the image recognition method using the learned individual part detection model can accurately detect individual parts of the face from an anterior eye image in which the subject's eye E is not shown, and by prioritizing detection of the eyebrows B, which are frequently detected, the time required for shooting start mode control can be shortened.

[0089] The ophthalmic device of the present disclosure has been described above based on Examples 1 and 2. However, the specific configuration is not limited to these Examples, and design changes and additions are permitted as long as they do not deviate from the gist of the invention according to each claim in the scope of the claims.

[0090] In the first and second embodiments, the photographing start mode control unit 631 has been described as an example in which, when it is determined that the subject's eye E is not reflected in the anterior eye image in the photographing start mode of the anterior eye using the anterior eye stereo camera 22, the photographing start mode control unit 631 automatically performs alignment control based on image recognition of the face part following the determination. However, the photographing start mode control unit 631 may be an example in which, when it is determined that the subject's eye is not reflected in the anterior eye image in the photographing start mode of the anterior eye using the anterior eye camera, the photographing start mode control unit 631 automatically performs alignment control based on image recognition of the face part after at least notifying the subject. That is, the photographing start mode control unit 631 may automatically perform alignment control based on image recognition of the face part after notifying the subject. Alternatively, the photographing start mode control unit 631 may automatically perform alignment control based on image recognition of the face part after notifying the subject and receiving a predetermined operation by the examiner following the determination that the subject's eye is not reflected. Here, the "notification" refers to notifying the subject in advance by an announcement or the like that the main body part or the chin rest part will move due to the execution of the alignment control.

[0091] In the first and second embodiments, the photographing start mode control unit 631 starts control to move the subject's eye E toward the predicted position of the subject's eye E, and then stops the movement when the subject's eye E is detected in the anterior eye image from the anterior eye stereo camera 22, and switches to normal alignment control. However, the photographing start mode control unit may be configured to switch to normal alignment control after confirming that the subject's eye is detected in the anterior eye image after ending control to move the subject's eye E toward the predicted position of the subject's eye.

[0092] In the first embodiment, an example of proceeding to low magnification photography when facial characteristic features cannot be detected is shown. In the second embodiment, an example of proceeding to low magnification photography when eyebrows B, contour F, and nose and mouth N cannot be detected is shown. However, in the first and second embodiments, when facial characteristic features can be detected but the predicted position of the subject's eye cannot be calculated, this may also be included in "when the position of the subject's eye E cannot be predicted" and may be an example of proceeding to low magnification photography.

[0093] In the first embodiment, the photographing start mode control unit 631 detects facial feature parts based on image recognition using an anterior eye image in which the subject's eye E is not shown and a learned feature part detection model. In the second embodiment, the photographing start mode control unit 631 detects eyebrows B, contour F, nose and mouth N individually based on image recognition using an anterior eye image in which the subject's eye E is not shown and a learned individual part detection model. However, the photographing start mode control unit may detect facial feature parts and individual parts of the face by a pattern matching or landmark matching method without using a learned feature part detection model or a learned individual part detection model by machine learning. Here, "pattern matching" refers to a method of preparing a pattern map in which feature parts are digitized in advance by a shape pattern, and determining whether the shape pattern acquired by image processing of an anterior eye image in which the subject's eye is not shown matches the pattern map to identify the feature parts. In addition to the shape pattern, pattern matching may be performed including a color pattern and a brightness pattern. "Landmark matching" is a technique in which facial landmarks (numbered feature points) that are important key points in detecting facial features such as the position of the eyes and nose are determined in advance, and characteristic parts are identified by the landmark numbers obtained from an anterior eye image in which the test eye is not shown.

[0094] In the first and second embodiments, the photographing start mode control unit 631 is exemplified as starting the photographing start mode control process by a predetermined operation after it is confirmed that the subject sits in front of the ophthalmologic apparatus A with the power switch turned on and supports the chin on the chin rest 30. However, the photographing start mode control unit may be exemplified as starting the photographing start mode control process when it is detected by a sensor or a switch that the subject supports the chin on the chin rest and presses the forehead against it.

[0095] In the first and second embodiments, an anterior eye stereo camera 22 is used as the anterior eye camera. However, the anterior eye camera is not limited to a stereo camera, and may be, for example, a monocular camera. Also, a camera without a zoom function for changing the magnification may be used.

[0096] In the first and second embodiments, an application example to an ophthalmic apparatus A that observes, photographs, and records an anterior segment image of a subject's eye, a fundus image of the subject's eye, and a tomographic image of the fundus of the subject's eye, and provides them as electronic images for diagnosis, has been shown. However, the application to the ophthalmic apparatus is not limited to the ophthalmic apparatus A. In other words, the alignment control technology of the present invention can be applied to any ophthalmic apparatus that requires alignment control for adjusting the relative positional relationship between the subject's eye and the apparatus main body, regardless of whether the apparatus is a subjective ophthalmic apparatus or an objective ophthalmic apparatus. [Explanation of symbols]

[0097] A Ophthalmology equipment 18 XYZ drive unit 20 Main body 22 Anterior Eye Stereo Camera (Anterior Eye Camera) 22a Camera No. 1 22b 2nd Camera 30 Chin rest 32 Chin rest drive unit 50 Measurement optical system 60 Control section 63 Alignment control unit 631 Shooting start mode control section E. Examined eye

Claims

1. An ophthalmologic device comprising: a main body having a built-in measurement optical system for measuring ocular characteristics of a subject's eye while the subject supports the chin on a chin rest; an anterior eye camera provided in the main body for acquiring an anterior eye image by photographing the anterior eye of the subject; and a control unit for controlling each unit of the device, the control unit includes an alignment control unit that performs control to adjust a relative positional relationship between the subject's eye and the main body unit based on the anterior eye image, The alignment control unit includes a photographing start mode control unit that, when it is determined that the subject's eye is not reflected in the anterior eye image during a photographing start mode of the anterior eye camera, calculates a predicted position of the subject's eye based on image recognition of a face part reflected in the anterior eye image automatically following the determination, or automatically after at least a notification following the determination, and controls the subject's eye to move toward the calculated predicted position of the subject's eye. An ophthalmic device comprising:

2. 2. The ophthalmic apparatus according to claim 1, The imaging start mode control unit starts control to move the subject's eye toward a predicted position where the subject's eye is present, and then when the subject's eye is detected in the anterior eye image from the anterior eye camera, stops the movement at the detection stage and switches to normal alignment control. An ophthalmic device comprising:

3. 2. The ophthalmic apparatus according to claim 1, When it is determined that the subject's eye is not included in the anterior eye image and the position of the subject's eye cannot be predicted, the photography start mode control unit reduces the magnification of the anterior eye camera and acquires the anterior eye image from the anterior eye camera again. An ophthalmic device comprising:

4. 4. The ophthalmic apparatus according to claim 1, When it is determined that the subject's eye is not included in the anterior eye image, the photography start mode control unit detects facial characteristic parts from the anterior eye image in which the subject's eye is not included, and calculates a predicted position of the subject's eye based on a positional relationship between the detected characteristic parts and the subject's eye in an overall face image. An ophthalmic device comprising:

5. 5. The ophthalmic apparatus according to claim 4, the imaging start mode control unit has a learned feature portion detection model that has been constructed in advance by executing machine learning using a machine learning data set of a large number of generated feature portion images and a selected machine learning model; Detecting the facial feature based on image recognition using the anterior eye image in which the subject's eye is not shown and the trained feature detection model. An ophthalmic device comprising:

6. 5. The ophthalmic apparatus according to claim 4, the shooting start mode control unit has a learned individual part detection model for eyebrows, a contour, a nose, and a mouth that has been constructed in advance by executing machine learning using a machine learning data set of a large number of generated characteristic part images and a selected machine learning model; When detecting the facial feature parts based on image recognition using the anterior eye image in which the subject's eye is not shown and the trained individual part detection model, the detection of the eyebrows is prioritized over the detection of the contour and the detection of the nose and mouth in an order setting for each feature part. An ophthalmic device comprising:

Images