Inspection system, inspection device, inspection method, and inspection program

Abstract

An examination system (1000) comprises: a camera (16); a processor (21) for examining the wetness of an eye, on the basis of captured images obtained as a result of the camera (16) imaging the eye, and an estimation model (232) that includes a neural network (2321); and a display (15) that displays the results of the examination executed by the processor (21).

Description

Technical Field

[0001] The present disclosure relates to an inspection system, an inspection device, an inspection method, and an inspection program for inspecting the moisture level of the eye. The present disclosure also relates to an inspection system, an inspection device, an inspection method, and an inspection program for inspecting the stability of the tear film. Furthermore, the present disclosure relates to an inspection system, an inspection device, an inspection method, and an inspection program for inspecting the presence or absence of dry eye.

Background Art

[0002] In recent years, due to the effects of aging, room drying caused by the use of air conditioners, the use of personal computers, and the wearing of contact lenses, the number of people complaining of eye discomfort has been increasing. Such ophthalmic diseases include, for example, dry eye.

[0003] Dry eye is a disease in which tears do not evenly cover the surface of the eye (for example, the cornea) due to insufficient tear volume or an imbalance in tear quality. Patients with dry eye may experience eye discomfort, visual function abnormalities, or damage to the eye surface.

[0004] Japanese Patent Application Laid-Open No. 7-136120 (Patent Document 1) discloses an ophthalmic device capable of diagnosing dry eye.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The ophthalmic device disclosed in Japanese Patent Publication No. 7-136120 captures interference patterns formed by light reflected from the surface of the eye using a camera. By observing the interference patterns captured by the camera of the ophthalmic device, the user can check for the presence or absence of dry eye.

[0007] However, the ophthalmic device disclosed in Japanese Patent Publication No. 7-136120 is a device specifically designed for users with sufficient knowledge to determine the presence or absence of dry eye, such as ophthalmologists. On the other hand, there is a need for technology that allows ordinary people or patients with little knowledge of ophthalmic diseases to more easily self-check their own eyes.

[0008] This disclosure has been made to solve the above-mentioned problems and aims to provide an examination system, examination device, examination method, and examination program that enable self-checking of the eyes. [Means for solving the problem]

[0009] This disclosure relates to an inspection system for examining the moisture level of the eye. The inspection system comprises a camera, an inspection unit that examines the moisture level of the eye based on an image captured by the camera when it photographs the eye and an estimation model including a neural network, and a display that displays the inspection results from the inspection unit.

[0010] Preferably, the estimation model is trained to examine the degree of eye moisture from the captured image, based on training data that includes the captured image and the results of the eye moisture level test.

[0011] Preferably, the captured image includes a first still image taken immediately after blinking and a second still image taken after a predetermined time has elapsed since blinking.

[0012] Preferably, each of the first and second still images is extracted from a video obtained by the camera capturing the eye.

[0013] Preferably, each of the first and second still images is a face image including at least the eyes. The eye image is extracted from the face image contained in each of the first and second still images.

[0014] Preferably, the inspection unit inspects the degree of eye moisture based on the degree of fluctuation of the eye image extracted from the first still image and the degree of fluctuation of the eye image extracted from the second still image.

[0015] Preferably, the display further shows the eye image included in the captured image. Preferably, the display further shows information about recommended eye drops.

[0016] Preferably, the display further outputs information about the ophthalmology clinic. Preferably, the information regarding ophthalmology clinics includes at least one of the following: information that encourages ophthalmology clinics to provide medical care; information regarding medical care provided at ophthalmology clinics; and information regarding recommended ophthalmology clinics.

[0017] Preferably, the display shows the results of a check for at least one of the subjective symptoms related to the eyes and the possibility that the eyes have an ophthalmic disease, as well as the test results based on the examination conducted by the testing department.

[0018] Preferably, the inspection system further comprises an inspection device and a server device capable of communicating with the inspection device. The inspection device includes a camera and a display. The server device includes an inspection unit.

[0019] This disclosure relates to an inspection device for examining the moisture level of the eye. The inspection device comprises a camera, an inspection unit that examines the moisture level of the eye based on an image captured by the camera when it photographs the eye and an estimation model including a neural network, and a display that displays the inspection results from the inspection unit.

[0020] The present disclosure relates to an inspection apparatus for inspecting the degree of eye moisture. The inspection apparatus includes a camera, a communication device for communicating with a server device including an estimation model including a neural network, and a display. The communication device transmits a captured video obtained by the camera capturing an eye to the server device. The communication device receives an inspection result of the degree of eye moisture acquired based on a captured image of the eye extracted from the captured video and the estimation model by the server device. The display displays the inspection result.

[0021] The present disclosure relates to an inspection method for inspecting the degree of eye moisture by a computer. The inspection method includes a step of inputting a captured image obtained by a camera capturing an eye, a step of inspecting the degree of eye moisture based on the captured image and an estimation model including a neural network, and a step of outputting an inspection result by the inspecting step.

[0022] The present disclosure relates to an inspection program for inspecting the degree of eye moisture. The inspection program causes a computer to execute a step of inputting a captured image obtained by a camera capturing an eye, a step of inspecting the degree of eye moisture based on the captured image and an estimation model including a neural network, and a step of outputting an inspection result by the inspecting step.

[0023] The present disclosure relates to an inspection system for inspecting the stability of the tear film. The inspection system includes a camera, an inspection unit that inspects the stability of the tear film based on a captured image obtained by the camera capturing an eye and an estimation model including a neural network, and a display that displays an inspection result by the inspection unit.

[0024] The present disclosure relates to an inspection system for inspecting the presence or absence of dry eye. The inspection system includes a camera, an inspection unit that inspects the presence or absence of dry eye based on a captured image obtained by the camera capturing an eye and an estimation model including a neural network, and a display that displays an inspection result by the inspection unit.

Advantages of the Invention

[0025] According to the inspection system, inspection device, inspection method, and inspection program of the present disclosure, a user can perform a self-check of the eyes.

Brief Description of the Drawings

[0026] [Figure 1] It is a diagram for explaining an inspection of the eye moisture level using the inspection device according to the present embodiment. [Figure 2] It is a diagram showing the configuration of the inspection system according to the present embodiment. [Figure 3] It is a diagram for explaining the learning of the estimation model in the learning phase. [Figure 4] It is a diagram showing the configuration of the inspection system in the operation phase. [Figure 5] It is a diagram showing the user interface screen displayed on the display of the inspection device during self-check. [Figure 6] It is a diagram showing the user interface screen displayed on the display of the inspection device during self-check. [Figure 7] It is a diagram showing the user interface screen displayed on the display of the inspection device during self-check. [Figure 8] It is a diagram showing the user interface screen displayed on the display of the inspection device during self-check. [Figure 9] It is a diagram showing the user interface screen displayed on the display of the inspection device during self-check. [Figure 10] It is a diagram showing the user interface screen displayed on the display of the inspection device during self-check. [Figure 11] It is a diagram showing the user interface screen displayed on the display of the inspection device during self-check. [Figure 12] It is a diagram showing the user interface screen displayed on the display of the inspection device during self-check. [Figure 13]This diagram shows the user interface screen displayed on the inspection device's screen during a self-check. [Figure 14] This diagram shows the user interface screen displayed on the inspection device's screen during a self-check. [Figure 15] This diagram shows the user interface screen displayed on the inspection device's screen during a self-check. [Figure 16] This is a flowchart illustrating the eye moisture level test process performed by the server device. [Figure 17] This is a diagram to explain the test for eye moisture level. [Figure 18] This figure shows the configuration of an inspection system according to another embodiment. [Modes for carrying out the invention]

[0027] Embodiments of this disclosure will be described in detail below with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and their descriptions will not be repeated. In this disclosure, the term "eye" is used to include not only the eyeball but also the surrounding tissues of the eye, such as the eyelid, inner corner of the eye, and outer corner of the eye. The term "eye" is used primarily to refer to the human eyeball. The term "pupil" is used primarily to refer to the black part of the human eye. "Moisture level of the eye" includes not only the moisture level of the eyeball but also the moisture level of the pupil or black part of the eye.

[0028] [Overview of the eye moisture level test] Figure 1 is a diagram illustrating the examination of eye moisture level using the examination device 1 according to this embodiment. Eyes with dry eye disease have unstable tear conditions, such as insufficient tear secretion compared to normal eyes, or sufficient secretion but poor tear quality causing tears to evaporate quickly. Therefore, in cases of dry eye, even if the tear condition is stable immediately after blinking, the tear condition becomes unstable after a period of time has passed without blinking.

[0029] If people experiencing the eye discomfort described above can self-check their eyes, such technology can increase their motivation to visit an ophthalmologist or purchase eye drops, leading to the early detection, prevention, and treatment of eye diseases.

[0030] Therefore, in this embodiment, user 10 can obtain objective eye (pupil) moisture level test results by launching an application program (hereinafter also referred to as the "self-check app") for performing a self-check of the eyes using the inspection device 1, and operating the inspection device 1 according to the self-check app.

[0031] Specifically, as shown in Figure 1, the inspection device 1 is equipped with a camera 16. The lens of the camera 16 is located on the back side of the inspection device 1 (opposite the side where the display 15 is located). The inspection device 1 captures a video of the user 10's face using the camera 16. The video obtained by the camera 16 (hereinafter also referred to as the "recorded video") includes multiple still images of the face obtained in chronological order. The inspection device 1 is connected to a server device 2, which will be described later, in a communication manner. The inspection device 1 transmits the data of the recorded video obtained by the camera 16 to the server device 2. The server device 2 extracts multiple still images from the recorded video acquired from the inspection device 1. Furthermore, the server device 2 uses AI (Artificial Intelligence) image recognition to extract the eye area from each of the multiple still images and identifies the changes in the extracted eye images in chronological order to inspect the moisture level of the user 10's eyes. The server device 2 transmits the inspection results to the inspection device 1. The inspection device 1 displays the inspection results obtained from the server device 2 on the display 15.

[0032] This allows user 10 to objectively check the moisture level of their eyes by using the testing device 1 to examine the state (stability) of their tears. In this way, user 10 can easily perform a self-check of their eyes using the testing device 1, regardless of their level of knowledge.

[0033] The test results output by the testing device 1 include not only the determination of the degree of eye moisture, but also the determination of the stability of the tear film covering the surface of the eye. Furthermore, the test results output by the testing device 1 include the determination of whether or not dry eye is present, that is, whether or not dry eye is present, or whether or not dry eye is suspected. Here, the degree of eye (pupil) moisture refers to the degree to which the surface of the eye is moistened with tears. The lower the degree of moisture, the more unstable the state of the tears becomes, and unevenness occurs in the tear film. On the other hand, the higher the degree of moisture, the more the surface of the eye is constantly moistened with tears, and the state of the tears, i.e., the tear film, remains stable without a shortage of tear components. In addition, the lens of the camera 16 may be positioned on the side of the testing device 1 where the display 15 is located. In this case, the user 10 should point the display 15 side of the testing device 1 in front of them and record a video of their face.

[0034] [Configuration of the inspection system] Figure 2 is a diagram showing the configuration of the inspection system 1000 according to this embodiment. As shown in Figure 2, the inspection system 1000 comprises a plurality of inspection devices 1 (in the example in Figure 2, inspection device 1A, inspection device 1B, and inspection device 1C) and a server device 2 that is connected to each of the plurality of inspection devices 1 in a manner that enables communication.

[0035] The inspection device 1 is configured according to a general-purpose computer architecture. In this embodiment, the inspection device 1 is exemplified by a portable terminal such as a smartphone that can be carried by user 10. However, the inspection device 1 may also be a device other than a smartphone, such as a desktop computer, a laptop computer, or a tablet computer.

[0036] The inspection device 1 comprises a processor 11, a communication device 12, a memory 13, an input interface 14, a display 15, and a camera 16.

[0037] The processor 11 is a computing entity (computer) that performs various processes according to various programs. The processor 11 is composed of at least one of the following: a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), a GPU (Graphics Processing Unit), and an MPU (Multi Processing Unit). The processor 11 may also be composed of processing circuits.

[0038] The communication device 12 transmits and receives data (information) to and from the server device 2 via a wired or wireless connection. In this embodiment, the communication device 12 transmits and receives data (information) to and from the communication device 22 of the server device 2 via wireless communication over the network 5. Specifically, during a self-check, the communication device 12 transmits a video recording, including images of the eyes acquired by the camera 16, to the server device 2 via the network 5. During a self-check, the communication device 12 receives data, including the results of the eye moisture level test, from the server device 2 via the network 5.

[0039] Memory 13 consists of volatile memory such as DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory), or non-volatile memory such as ROM (Read Only Memory). Memory 13 stores various programs and data, such as a self-check application 131 for performing eye self-checks. The self-check application 131 is an application program for the user 10 to check the moisture level of their eyes themselves, and includes a program for taking pictures of the eyes with the camera 16.

[0040] The input interface 14 is an interface that accepts input from the user 10, such as buttons and a touch panel. The input interface 14 outputs a signal based on the user's input to the processor 11.

[0041] The display 15 is a display device such as a liquid crystal display, plasma display, or organic EL (Electro-Luminescence) display, and displays a predetermined screen based on the control of the processor 11.

[0042] Camera 16 records the subject in video. The video data 236 of the video recorded by camera 16 is transmitted to server device 2 via network 5.

[0043] Server device 2 is configured according to a general-purpose computer architecture. In this embodiment, server device 2 is a server device owned by a manufacturer that provides user 10 with a self-check application 131 for checking the moisture level of the eyes. Server device 2 comprises a processor 21, a communication device 22, and memory 23.

[0044] The processor 21 is an example of a "test unit." The processor 21 is an arithmetic unit that executes various processes (for example, the test process described later) according to various programs (for example, the test program 231 described later). The processor 21 is composed of at least one of the following: a CPU, FPGA, GPU, and MPU. The processor 21 may also be composed of arithmetic circuits.

[0045] The communication device 22 transmits and receives data (information) to and from each of the multiple inspection devices 1 via wired or wireless connection. In this embodiment, the communication device 22 transmits and receives data (information) to and from the communication device 12 of the inspection device 1 via wireless communication over the network 5. Specifically, during a self-check, the communication device 22 receives a video recording, including images of the eyes taken by the camera 16, from the inspection device 1 via the network 5. During a self-check, the communication device 22 transmits data, including the results of the eye moisture level test, to the inspection device 1 via the network 5.

[0046] Memory 23 consists of volatile memory such as DRAM and SRAM, or non-volatile memory such as ROM. Memory 23 stores various programs and data, including an examination program 231 for examining the moisture level of the eyes, an estimation model 232 used to examine the moisture level of the eyes, eye drop information 233 containing information about eye drops, ophthalmology clinic information 234 containing information about ophthalmology clinics, user information 235 containing information about each user 10 of the multiple examination devices 1, video data 236 of videos captured by the camera 16 of the examination device, and a calculation table 237 for calculating a score used when determining the results of a self-check. Here, the examination program is a program for examining the moisture level of the eyes by analyzing videos of the eyes acquired from the examination device 1 using AI.

[0047] [Training the estimation model during the learning phase] Figure 3 is a diagram illustrating the learning of the estimation model 232 during the learning phase. The learning phase is a pre-training phase in which the estimation model 232 is trained before the self-check app 131 is provided to the user 10's inspection device 1. As shown in Figure 3, the estimation model 232 is trained by the learning device 31 to examine the moisture level of the eye from the captured image of the eye.

[0048] For training the estimated model 232, known algorithms such as supervised learning, unsupervised learning, and reinforcement learning can be used. In this embodiment, the learning device 31 trains the estimated model 232 using supervised learning with the training data 4.

[0049] Training data 4 is prepared in advance for training the estimation model 232 and includes video recordings of eyes and the results of eye moisture level tests. For example, the designer of the examination program 231 records videos of the eyes of multiple people with different levels of eye moisture, associates the obtained video recordings of the eyes with the results of eye moisture level tests (ground truth data), and uses this as training data 4. The designer prepares multiple such training data sets 4 in advance.

[0050] The estimated model 232 includes a neural network 2321 and parameters 2322 used by the neural network 2321. The neural network 2321 can be any known neural network used in deep learning-based image recognition, such as a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or an LSTM (Long Short-Term Memory Network). The estimated model 232 performs deep learning using the neural network 2321 described above. The parameters 2322 include weighting coefficients used in the calculations performed by the neural network 2321. Note that the estimated model 232 is not limited to models trained by deep learning using a neural network; it may also be trained by other machine learning methods.

[0051] The learning device 31 accepts an input of an eye image from the training data 4. Based on the input eye image and the estimation model 232 which includes the neural network 2321, the learning device 31 performs processing to examine the degree of eye moisture represented by the eye image.

[0052] Specifically, the learning device 31 acquires a still image immediately after blinking and a still image taken after a predetermined time (for example, 5 seconds) has elapsed since blinking. Each of the two acquired still images is a still image that includes at least an image of the eye. The learning device 31 inputs the two acquired still images into the estimation model 232. The estimation model 232 estimates the moisture level of the user's eyes by identifying the time-series changes in the two eye images through image recognition. The learning device 31 acquires the estimated result of the eye moisture level obtained by the estimation model 232.

[0053] The learning device 31 may also estimate the moisture level of the eyes based on multiple still images obtained from immediately after blinking until a predetermined time (for example, 5 seconds) has elapsed. For example, the learning device 31 may acquire all still images obtained for each frame from immediately after blinking until a predetermined time (for example, 5 seconds) has elapsed, and estimate the moisture level of the user's eyes by identifying the changes in the time series of the multiple acquired eye images.

[0054] The learning device 31 trains the estimation model 232 based on the estimated result of the eye moisture level and the ground truth data included in the training data 4 (the result of pre-measurement of the eye moisture level represented by the captured image). Specifically, the learning device 31 trains the estimation model 232 by adjusting the parameters 2322 (for example, weighting coefficients) so that the estimated result of the eye moisture level obtained by the estimation model 232 approaches the ground truth data.

[0055] [Configuration of inspection equipment during the operational phase] Figure 4 shows the configuration of the inspection device 1 during the operational phase. The operational phase is the phase in which the self-check application 131 is provided to the user 10's inspection device 1, and then the degree of eye moisture is estimated using the estimation model 232. As shown in Figure 4, the server device 2 stores the estimation model 232 learned by the learning device 31 shown in Figure 3 in the memory 23. For example, the server device 2 retrieves the estimation model 232 from the learning device 31 and stores the retrieved estimation model 232 in the memory 23. Note that the learning device 31 may be the server device 2, and the functions of the learning device 31 described above may be functions of the processor 21 of the server device 2.

[0056] The processor 21 of the server device 2 includes an input unit 211, a processing unit 212, and an output unit 213.

[0057] The input unit 211 receives a video recording input that includes captured images obtained when the camera 16 of the inspection device 1 photographs the user's eyes. The processing unit 212 performs processing to examine the moisture level of the eyes represented by the captured images, based on the captured images extracted from the video recording input from the input unit 211 and an estimation model 232 that includes a neural network 2321. As mentioned above, the estimation model 232 is not limited to one that is learned by deep learning using a neural network, but may also be learned by other machine learning methods.

[0058] Specifically, the processing unit 212 acquires a still image of the eye immediately after blinking and a still image of the eye after a predetermined time (for example, 5 seconds) has elapsed since blinking. The still image of the eye immediately after blinking is an example of the "first still image". The still image of the eye after a predetermined time (for example, 5 seconds) has elapsed since blinking is an example of the "second still image". Each of the two acquired still images is a still image that includes at least an image of an eye. The processing unit 212 inputs the two acquired still images into the estimation model 232. The estimation model 232 estimates the moisture level of the user's eyes by identifying the time-series changes in the two eye images through image recognition. The processing unit 212 acquires the estimated result of the eye moisture level obtained by the estimation model 232.

[0059] The processing unit 212 may also estimate the moisture level of the eyes based on multiple still images obtained from immediately after blinking until a predetermined time (for example, 5 seconds) has elapsed. For example, the processing unit 212 may acquire all still images obtained for each frame from immediately after blinking until a predetermined time (for example, 5 seconds) has elapsed, and estimate the moisture level of the user's eyes by identifying the changes in the time series of the multiple acquired eye images.

[0060] The output unit 213 outputs the test results obtained by the processing unit 212 to the testing device 1. The display 15 of the testing device 1 provides the user 10 with the test results of eye moisture level by displaying a screen showing the obtained test results of eye moisture level.

[0061] [Specific examples of eye self-checks] Referring to Figures 2 and 5-15, a specific example of a user 10 performing an eye self-check using the examination device 1 will be described. Figures 5-15 show the user interface screen displayed on the display 15 of the examination device 1 during a self-check. The user can operate icons displayed on the display 15 using the input interface 14. Note that the user interface screens shown in Figures 5-15 are examples and can be modified as appropriate by the designer of the self-check application 131. Hereafter, the act of user 10 performing an eye examination using the self-check application 131 will also be referred to as a "self-check".

[0062] As shown in Figure 5(A), when user 10 launches the self-check application 131 using the inspection device 1, a login screen is displayed on the display 15.

[0063] The login screen includes fields for entering an ID and password to identify user 10. User 10 obtains an ID and password from server device 2 by registering their own information (for example, name, gender, age, phone number, email address, etc.) with server device 2. User 10 may also set their own desired ID and password. The information registered by user 10 with server device 2 is stored in memory 23 as user information 235.

[0064] On the login screen, when user 10 enters their ID and password, inspection device 1 sends the ID and password to server device 2. Once server device 2 authenticates user 10 based on the ID and password, inspection device 1 displays a menu screen on display 15, as shown in Figure 5(B). The menu screen includes an icon 151 for starting a self-check, an icon 152 for viewing the history of past self-checks, and an icon 153 for making various settings.

[0065] When user 10 operates icon 151, the inspection device 1 displays a risk check screen on display 15 for user 10 to perform a risk check, as shown in Figures 5(C) and (D). The risk check is a check to determine the possibility that user 10 has an ophthalmic disease. In this embodiment, the risk check includes multiple check items such as user 10's gender, age, daily usage time of PCs and smartphones, presence or absence of stiff shoulders, presence or absence of headaches, presence or absence of contact lenses, presence or absence of eye drops, and whether or not the user has been diagnosed with dry eye in the past.

[0066] Once user 10 completes the risk check, the inspection device 1 displays a selection screen on the display 15, as shown in Figure 6(E). The selection screen includes an icon 148 for viewing a tutorial and an icon 149 for taking a picture.

[0067] When user 10 operates icon 148, the inspection device 1 displays a tutorial screen on display 15, as shown in Figures 6(F1) to 7(F6). The tutorial screen includes images to explain to user 10 how to photograph the eye for self-check.

[0068] As shown in Figure 6(F1), the inspection device 1 displays a message on the display 15 prompting user 10 to rest their eyes while recording video, such as "Please look outside and rest your eyes." Furthermore, the inspection device 1 displays a message on the display 15 informing user 10 that natural light will be used for video recording, such as "We will record using natural light during the daytime."

[0069] As shown in Figure 6(F2), the inspection device 1 displays a message on the display 15 informing the user 10 of the standing position when recording the video, such as, "Stand directly in front of the window, get within one arm's length of your body, and face the window while recording."

[0070] As shown in Figure 7(F3), the inspection device 1 displays a message on the display 15 informing the user 10 of the relative positions of the user 10 and the camera 16 during video recording, such as, "Please check the shooting distance. Make a thumbs-up sign with your hand and straighten your thumb as much as possible."

[0071] As shown in Figure 7(F4), the inspection device 1 displays a message on the display 15 informing the user 10 of the positional relationship between the user 10 and the camera 16 during video recording, such as, "Please lightly place your thumb on your cheekbone and place the little finger side against the back of the device."

[0072] As shown in Figure 7(F5), the inspection device 1 displays an image on the display 15 of what it looks like when the thumb is correctly and lightly placed on the cheekbone.

[0073] As shown in Figure 7(F6), the inspection device 1 displays an image on the display 15 of what happens when the thumb is mistakenly placed lightly on the cheekbone. Specifically, the display 15 shows an example where the lower eyelid is being pulled by the thumb placed on the cheekbone.

[0074] In the selection screen shown in Figure 6(E), when user 10 operates icon 149, the inspection device 1 displays a self-check screen on the display 15, as shown in Figures 8(G1) to 9(G8). The self-check screen includes images for measuring the moisture level of the eyes based on the captured video, which is recorded by the camera 16 and used to record a video of the user's eyes.

[0075] As shown in Figure 8(G1), the inspection device 1 displays a message on the display 15 prompting user 10 to take a video of their face with their thumb lightly resting on their cheekbone, such as, "Blink slowly, and then immediately start recording a video for about 5 seconds. Please refrain from blinking while recording." As a result, user 10 takes a video of their face, including their eyes, for 5 seconds using the camera 16 while looking at the scenery outside in natural daylight.

[0076] As shown in Figure 8(G2), the inspection device 1 displays the image being captured by the camera 16 on the display 15. When the camera 16 has finished capturing the image, as shown in Figure 8(G3), the inspection device 1 displays a message on the display 15 to inform the user 10 that the capture is complete, such as "Capture complete".

[0077] As shown in Figure 8(G4), after the shooting is complete, the inspection device 1 displays the image taken immediately after the start of shooting on the display 15. At this time, the inspection device 1 detects the pupil (black of the eye) based on the still image taken immediately after the start of shooting and extracts an image of the peripheral part of the eye including the pupil (for example, the eye itself). The inspection device 1 indicates the extracted image portion with a frame. The user 10 can adjust this frame by moving, enlarging, or shrinking it so that at least the image portion of the eye is within the frame. By selecting icon 147, the user 10 can crop the image portion within the frame and extract at least the captured image of the eye. Alternatively, the user 10 can retake the image by selecting icon 146.

[0078] As shown in Figure 9(G5), the inspection device 1 displays the message "Please check if the image is suitable for the examination" on the display 15. As shown in Figure 9(G6), the inspection device 1 displays the captured image of the eye on the display 15 five seconds after it has been captured. Furthermore, the inspection device 1 displays on the display 15 an example of a captured image of the eye when the pupil (black of the eye) detection and focusing are appropriate, and an example of a captured image of the eye when the pupil (black of the eye) detection and focusing are not appropriate.

[0079] As shown in Figure 9(G7), the inspection device 1 displays on the display 15 an icon 154 indicating that an image suitable for the inspection has been acquired, along with a message asking the user 10 whether an image suitable for the inspection has been acquired, such as "Is this an appropriate image for the inspection?", and an icon 155 indicating that the eye video should be re-recorded.

[0080] When the user operates icon 154, the AI ​​starts measuring the moisture level of the eyes. As shown in Figure 9(G8), during the self-check, the testing device 1 displays a message on the display 15 to inform the user 10 that the evaluation is in progress, such as "AI evaluation in progress." Furthermore, the testing device 1 displays a message on the display 15 prompting the user 10 to check for subjective symptoms related to the eyes, such as "Check subjective symptoms here," along with an icon 156 for the user 10 to perform the subjective symptom check.

[0081] When user 10 operates icon 156, the testing device 1 displays multiple subjective symptom check screens on the display 15 in stages, as shown in Figures 10(H1) and (H2).

[0082] As shown in Figures 10(H1) and (H2), the testing device 1 displays 12 questions on the display 15 as a subjective symptom check, asking about the user 10's subjective symptoms related to the eyes, such as "My eyes feel dry" and "My eyes tire easily." The user 10 can input the subjective symptom check results by checking the items for which they experience subjective symptoms. As shown in Figure 10(H1), the user 10 can also carry over subjective symptom check results that they have entered in the past.

[0083] Once user 10 has completed inputting all self-reported symptoms, as shown in Figure 11(H3), the testing device 1 displays a message to inform user 10 that the self-check results have been obtained, such as "Thank you for your responses! The image-based test results are ready," along with an icon 157 on the display 15 for viewing the test results.

[0084] When user 10 operates icon 157, the inspection device 1 sequentially displays multiple inspection result screens showing the self-check inspection results on the display 15, as shown in Figures 12 and 13.

[0085] Figure 12 shows the test results when the user 10's eye moisture level is low. As shown in Figure 12(J1), the testing device 1 displays on the display 15 an image 1511 showing the overall evaluation of the test results, an image 1521 showing the tear quality check results, and an image 1531 showing a magnified portion of the eye.

[0086] Image 1511 includes a message indicating that User 10's eye moisture level is low, such as "Your eye moisture level is low," and an overall assessment that encourages User 10 to visit an ophthalmologist, such as "We recommend you see an ophthalmologist as soon as possible. You may find out the true cause of the decreased moisture level in your eyes."

[0087] Image 1521 includes a diagram that informs user 10 that the quality of their tears is unstable.

[0088] Image 1531 includes a magnified image of user 10's eye used for self-check. Furthermore, this magnified image shows the portion of the eye that was the target of image recognition during the self-check. This allows user 10 to confirm the image of their own eye used for self-check on the display 15.

[0089] As shown in Figure 12(J2), the testing device 1 displays an image 1541 on the display 15 showing the results of the self-check based on the subjective symptom check shown in Figure 10.

[0090] Image 1541 includes a message informing user 10 of the results of a self-reported symptom check, such as, "You may have unstable tearing if you checked 5 or more items."

[0091] Figure 13 shows the test results when the user 10's eye moisture level is moderate. As shown in Figure 13(K1), the testing device 1 displays on the display 15 an image 1512 showing the overall evaluation of the test results, an image 1522 showing the tear quality check results, and an image 1532 showing a magnified portion of the eye.

[0092] Image 1512 includes a message indicating that User 10's eye moisture level is moderate, such as "Your eye moisture level is 'medium'," and an overall assessment message encouraging User 10 to take care of their eyes, including using eye drops, such as "Why not try using eye drops from a drugstore? If you are concerned about your symptoms, please consult an ophthalmologist."

[0093] Image 1522 includes a diagram that informs user 10 that the quality of their tears is generally normal.

[0094] Image 1532 includes a magnified image of user 10's eye used for self-checking. Furthermore, this magnified image shows the portion of the eye that was the target of image recognition during the self-checking. This allows user 10 to confirm the image of their own eye used for self-checking on the display 15.

[0095] As shown in Figure 13(K2), the testing device 1 displays an image 1542 on the display 15 showing the results of the self-check based on the subjective symptom check shown in Figure 10.

[0096] Image 1542 includes a message that informs user 10 of the results of a self-reported symptom check, such as, "You may have unstable tearing based on item 2."

[0097] As shown in Figure 14(L), the inspection device 1 displays an image 1543 on the display 15 showing the inspection results of the self-check based on the risk check shown in Figures 5(C) and (D).

[0098] Image 1543 includes a message that informs user 10 of the items included in the risk check that increase the risk of dry eye, such as, "If any of the following apply to you, you are at high risk of dry eye, so please be careful."

[0099] As shown in Figures 12(J1) and 13(K1), the inspection device 1 displays icons 158 on the display 15 on the inspection results screen for viewing corrective actions based on the inspection results.

[0100] In the test results screen when user 10's eye moisture level is low (the test results screen in Figure 12(J1)), if user 10 operates icon 158, the testing device 1 displays a countermeasures screen on the display 15, showing countermeasures based on the test results, as shown in Figure 15(M1).

[0101] The screen for dealing with low eye moisture levels for user 10 includes, as information about ophthalmology clinics, at least one of the following: information encouraging consultation at an ophthalmology clinic, information about consultation at an ophthalmology clinic, and information about recommended ophthalmology clinics. In this disclosure, the term "consultation" is used to include "diagnosis" and "treatment." That is, the information about ophthalmology clinics may include at least one of the following: information encouraging consultation (diagnosis or treatment) at an ophthalmology clinic, information about consultation (diagnosis or treatment) at an ophthalmology clinic, and information about recommended ophthalmology clinics.

[0102] Specifically, as shown in Figure 15(M1), the screen for dealing with low eye moisture levels in user 10 displays two messages on display 15: one encouraging user 10 to seek medical attention at an ophthalmologist's office, such as "We recommend you see an ophthalmologist first," and another informing user 10 about the medical treatment at the ophthalmologist's office, such as "We will diagnose dry eye by checking your symptoms through a medical interview and performing tests on the quality of your tears."

[0103] Furthermore, the examination device 1 displays an icon 159 on the display 15 to inform user 10 of the nearest ophthalmology clinic as information about recommended ophthalmology clinics. When user 10 operates the icon 159, for example, the examination device 1 obtains information about user 10's nearest ophthalmology clinic from the server device 2 via the network 5 and displays the obtained information on the display 15. The examination device 1 may also search for user 10's nearest ophthalmology clinic via an internet connection through the network 5 and display the search results on the display 15. The examination device 1 may also search for user 10's nearest ophthalmology clinic based on location information it possesses. The examination device 1 may also search for user 10's nearest ophthalmology clinic based on information entered by user 10 (for example, user 10's address).

[0104] In this way, based on the self-check results, the testing device 1 provides user 10 with information regarding ophthalmology clinics, including information encouraging consultation at an ophthalmology clinic, information regarding consultations at ophthalmology clinics, and information regarding recommended ophthalmology clinics. As a result, user 10 is encouraged to go to an ophthalmology clinic and can also obtain information about the nearest ophthalmology clinic.

[0105] As shown in Figure 15(M1), the inspection device 1 may display on the display 15 all of the following information regarding ophthalmology clinics: information encouraging consultations by ophthalmology clinics, information regarding consultations at ophthalmology clinics, and information regarding recommended ophthalmology clinics. Alternatively, it may display on the display 15 at least one of these pieces of information regarding ophthalmology clinics.

[0106] Furthermore, in the screen showing how to deal with moderate or high levels of eye moisture in the user 10, the inspection device 1 may display at least one of the following on the display 15 as information about ophthalmology clinics: information encouraging a visit to an ophthalmology clinic, information about a visit to an ophthalmology clinic, and information about a recommended ophthalmology clinic. In this way, even if the user 10's eye moisture level is moderate or high, the inspection device 1 can prompt the user 10 to visit an ophthalmology clinic as a precaution.

[0107] In the test results screen when user 10's eye moisture level is moderate (the test results screen in Figure 13(K1)), when user 10 operates icon 158, the testing device 1 displays a countermeasures screen on the display 15, showing countermeasures based on the test results, as shown in Figure 15(M2).

[0108] The screen for user 10 with moderate eye moisture levels includes information on recommended eye drops.

[0109] Specifically, as shown in Figure 15(M2), when user 10's eye moisture level is moderate, the treatment screen displays on display 15 a message introducing recommended eye drops to user 10, such as "Recommended over-the-counter medicine for self-care," and a message introducing the function of the recommended eye drops to user 10, such as "For those who want to maintain eye moisture, eye drops with high moisturizing function."

[0110] Furthermore, the testing device 1 displays an icon 160 on the display 15 to inform the user 10 of detailed information about recommended eye drops. When the user 10 operates the icon 160, the testing device 1, for example, obtains detailed information about the recommended eye drops from the server device 2 via the network 5 and displays the obtained information on the display 15. The testing device 1 may also search for detailed information about the recommended eye drops via the internet connection through the network 5 and display the search results on the display 15. The eye drops recommended by the testing device 1 may be eye drops or eye ointment.

[0111] In this way, the testing device 1 provides user 10 with information on recommended eye drops based on the self-check test results. This allows user 10 to obtain information on the optimal eye drops for self-care of their eyes.

[0112] Furthermore, the inspection device 1 may display information on recommended eye drops on the display 15 when the screen for dealing with low eye moisture levels of user 10 is displayed. In this way, when user 10 has low eye moisture levels, the device can prompt user 10 to seek medical attention at an ophthalmologist's office and provide user 10 with information on the most suitable eye drops for self-care of their eyes.

[0113] [Eye moisture level test] Figure 16 is a flowchart of the eye moisture level test process performed by server device 2. The test process shown in Figure 16 is performed by the processor 21 of server device 2 executing the test program 231. Server device 2 executes the test process shown in Figure 16 when predetermined start conditions are met. Examples of start conditions include the user 10 operating the icon 151 to start a self-check, as shown in Figure 5(B). In Figure 16, "S" is used as an abbreviation for "STEP".

[0114] As shown in Figure 16, the server device 2 acquires the risk check results (S1). Specifically, the server device 2 acquires the user 10's risk check results input from the inspection device 1 via the network 5. The server device 2 determines whether or not a video recording including images of the user 10's eyes obtained by the camera 16 of the inspection device 1 has been input (S2). The video recording of the eyes includes a still image of the eye immediately after blinking and a still image of the eye after a predetermined time (for example, 5 seconds) has elapsed since blinking. If the server device 2 has not received a video recording (NO in S2), it repeats the process in S2.

[0115] When a recorded video is input to Server Device 2 (YES in S2), it extracts a still image of the eye immediately after blinking and a still image of the eye after a predetermined time (for example, 5 seconds) has elapsed since blinking from the recorded video (S3). Server Device 2 extracts the degree of fluctuation of each of the two eye images by image recognition (S4). Based on the time-series changes in the degree of fluctuation in the two eye images, Server Device 2 checks the moisture level of the eye (S5).

[0116] Here, referring to Figure 17, we will explain the process for examining the moisture level of the eyes based on the recorded video. Figure 17 is a diagram illustrating the examination of eye moisture level.

[0117] As shown in Figure 17, the inspection device 1 uses the camera 16 to start recording at timing t1 immediately after blinking, and then stops recording at timing t2, 5 seconds later. The inspection device 1 transmits the recorded video data to the server device 2.

[0118] The server device 2 extracts from the video footage acquired from the inspection device 1 two still images: one taken at timing t1 immediately after blinking (an example of the "first still image"), and another taken at timing t2 a predetermined time after blinking (5 seconds in the example in Figure 17) (an example of the "second still image") (this corresponds to the process S3 in Figure 16). As shown in Figure 8(G2), each of these two still images contains an image of the user 10's face.

[0119] Server device 2 extracts an eye image 301 from a still image at timing t1 and an eye image 302 from a still image at timing t2 using image recognition. Server device 2 extracts the degree of fluctuation of eye image 301 at timing t1 and the degree of fluctuation of eye image 302 at timing t2 (processing corresponding to S4 in Figure 16).

[0120] In cases of dry eye, even if the tear film is stable immediately after blinking, it becomes unstable after a period of time has passed without blinking. Therefore, in cases of dry eye, the degree of fluctuation in the eye image after a period of time without blinking is greater than that of the eye image immediately after blinking. In other words, the greater the degree of fluctuation in the eye image, the more unstable the tear film is, and such an eye image suggests that the eyes are not sufficiently moist and / or that the person has or may have dry eye.

[0121] For example, in the eye image 301 at timing t1, the outside scenery (in this example, the city background) reflected on the surface of the eye (e.g., the cornea) is generally clearly depicted, but in the eye image 302 at timing t2, the outside scenery (city background) reflected on the surface of the eye (cornea) is depicted in a blurred manner. "Degree of fluctuation in the eye image" refers to the degree to which the objects reflected on the surface of the eye (e.g., the cornea) shown in the image are blurred, smudged, or wavering.

[0122] Server device 2 uses estimation model 232 to compare the eye image 301 at timing t1 with the eye image 302 at timing t2, and by observing the changes in the degree of fluctuation as described above, it checks the moisture level of user 10's eyes (processing corresponding to S5 in Figure 16).

[0123] Furthermore, timing t2 is not limited to the 5 seconds immediately following a blink; any timing that allows for the examination of eye moisture level based on changes in the degree of fluctuation in the eye image is acceptable.

[0124] Furthermore, the eye image 302 to be compared with the eye image 301 immediately after blinking is not limited to one image, but may be multiple images. For example, the server device 2 may extract the eye image 301 at timing t1 immediately after blinking, then extract the eye image 302 at timing t2 after a first predetermined time (for example, 5 seconds), and then extract the eye image at timing t3 after a second predetermined time (for example, 7 seconds), and by observing the change in the degree of fluctuation in these multiple eye images, the moisture level of the eye can be examined. In this way, the server device 2 can examine the moisture level of the eye based on multiple eye images extracted at each of the multiple timings that have elapsed since immediately after blinking, thereby enabling a more accurate examination of the moisture level of the eye.

[0125] Furthermore, the inspection device 1 is not limited to the process of extracting multiple eye images from a video of the face (eyes). For example, the inspection device 1 may acquire a first still image (photograph) by taking a picture of the user 10's face at timing t1 immediately after blinking, and then acquire a second still image (photograph) by taking a picture of the user 10's face at timing t2 after a predetermined time (for example, 5 seconds) has elapsed. The server device 2 may then extract eye images from each of the multiple still images acquired in this manner.

[0126] Returning to Figure 16, after processing in S5, the server device 2 acquires the subjective symptom check results (S6). Specifically, the server device 2 acquires the subjective symptom check results of user 10, which are input from the testing device 1, via the network 5.

[0127] Server device 2 calculates the score used to determine the final self-check result (S7).

[0128] Here, we will explain how to calculate the points used to determine the self-check results. In the calculation table for calculating points (illustration omitted), predetermined points are assigned to the results of the eye moisture level check using images, predetermined points are assigned to each item of the subjective symptom check, and predetermined points are assigned to each item of the risk check. Server device 2 refers to the calculation table and calculates points using an additive method based on the results of the eye moisture level check using images, the subjective symptom check results, and the risk check results. For example, if user 10 checks the item "My eyes feel dry" in the subjective symptom check, server device 2 adds predetermined points, and if user 10 selects "Yes" for the item "Use of eye drops" in the risk check, server device 2 adds predetermined points. In this way, server device 2 obtains the total score based on the results of the eye moisture level check using images, the subjective symptom check results, and the risk check results.

[0129] Server device 2 extracts a judgment message as a result of the self-check based on the score obtained by referring to the calculation table and a predetermined standard value.

[0130] For example, if the score is above the standard value, server device 2 extracts the message "Your eye moisture level is low" and the message "We recommend you see an ophthalmologist as soon as possible. You may find out the true cause of the decreased moisture level in your eyes," as shown in Figure 12, and outputs the results to testing device 1.

[0131] Thus, since the server device 2 generates the final results by considering the results of the eye moisture level test using images, the risk check results, and the subjective symptom check results, it can output self-check results with greater accuracy than if the final results were generated based solely on the results of the eye moisture level test using images.

[0132] Returning to Figure 16, Server Device 2 determines whether the score calculated in S7 is above the standard value (S8). If the score is above the standard value, i.e., if the eye moisture level is low (YES in S8), Server Device 2 obtains information about the ophthalmology clinic as shown in Figure 15 (M1) and includes it in the test results (S9).

[0133] If the score calculated in S7 is below the standard value, i.e., if the eye moisture level is moderate or high (NO in S8), the server device 2 obtains information on recommended eye drops as shown in Figure 15 (M2) and includes it in the test results (S10).

[0134] Furthermore, even if the eye moisture level is low, Server Device 2 may acquire information on recommended eye drops, as shown in Figure 15(M2), and include it in the test results. Even if the eye moisture level is moderate or high, Server Device 2 may acquire information on ophthalmology clinics, as shown in Figure 15(M1), and include it in the test results. Moreover, Server Device 2 is not limited to generating the final result based on the subjective symptom check results, the risk check results, and the image-based eye moisture level test results. Server Device 2 may generate the final result based on at least one of the subjective symptom check results and the risk check results, and the image-based eye moisture level test results.

[0135] Server device 2 outputs the generated inspection results to inspection device 1 (S11). As a result, the inspection results shown in Figures 13 to 15 are displayed on the display 15 of inspection device 1. After that, server device 2 terminates this process.

[0136] In this way, the server device 2 executes the inspection process according to the inspection program 231, allowing the user 10 to easily perform a self-check of their eyes using the inspection device 1.

[0137] [Another embodiment] In this embodiment, the server device 2 performed the inspection process to check the moisture level of the user's eyes 10, but the inspection device 1 may also perform the inspection process to check the moisture level of the user's eyes 10.

[0138] Figure 18 shows the configuration of an inspection system 1000a according to another embodiment. As shown in Figure 18, the inspection device 1a may store the inspection program 231, estimation model 232, eye drop information 233, ophthalmology clinic information 234, and calculation table 237 that were provided by the server device 2 shown in Figure 2 in the memory 13. The processor 11 of the inspection device 1a may then inspect the moisture level of the eye based on the captured images included in the video footage obtained by the camera 16 capturing the eye, and the estimation model 232 which includes a neural network 2321. Furthermore, the inspection results may be displayed on the display 15 as the final inspection result, along with the eye drop information 233 or the ophthalmology clinic information 234. In other words, the inspection device 1a may perform the processing corresponding to the inspection processing of the server device 2 shown in Figure 16.

[0139] As described above, the inspection device 1 can examine the degree of eye moisture based on changes in the degree of fluctuation of the eye image. Here, a large degree of fluctuation in the eye image means that the stability of the tear film is reduced (dry eye). Therefore, the inspection device 1 can also perform an examination of tear film stability (dry eye) based on changes in the degree of fluctuation of the eye image.

[0140] Furthermore, tear film stability can also be evaluated by measuring the time from when the eye is opened until the tear film on the surface of the eye breaks down (also known as tear film break-up time (BUT)). In this disclosure, the inspection device 1 can also measure BUT and evaluate tear film stability based on changes in the degree of fluctuation of the eye image. In particular, when fluorescent dyes are not used, the inspection device 1 can also measure non-invasive tear film break-up time (NIBUT) and evaluate tear film stability.

[0141] Thus, the descriptions of the examination system 1000, examination device 1, examination method, and examination program for examining the moisture content of the eye are applicable to the examination system, examination device, examination method, and examination program for examining the stability of the tear film.

[0142] Furthermore, the descriptions of the examination system 1000, examination device 1, examination method, and examination program for examining the moisture level of the eyes are applicable to the examination system, examination device, examination method, and examination program for examining the presence or absence of dry eye, respectively.

[0143] [summary] This disclosure relates to an inspection system 1000 for examining the moisture level of the eye (tear film stability, presence or absence of dry eye). As shown in Figures 2 and 4, the inspection system 1000 includes a camera 16, a processor 21 (inspection unit) that examines the moisture level of the eye (tear film stability, presence or absence of dry eye) based on captured images obtained by the camera 16 photographing the eye and an estimation model 232 including a neural network 2321, and a display 15 that displays the inspection results from the processor 21.

[0144] This allows user 10 to objectively examine the moisture level of their eyes (tear film stability, presence or absence of dry eye) by taking a picture of their eyes using the camera 16 of the inspection device 1. In this way, user 10 can easily perform a self-check of their eyes using the inspection device 1.

[0145] The term "captured image" may refer to either a "still image" (a so-called photograph) acquired by camera photography, or a "still image" included in a "video" acquired by camera photography. The inspection unit is configured to examine the degree of eye moisture (stability of the tear film, presence or absence of dry eye) based on at least a "still image" (an image included in a photograph or video).

[0146] Preferably, as shown in Figure 3, the estimation model 232 is trained to examine the degree of eye moisture (tear film stability, presence or absence of dry eye) from the captured images, based on training data 4 which includes captured images and the results of examinations on the degree of eye moisture (tear film stability, presence or absence of dry eye).

[0147] This allows the inspection system 1000 to use a pre-trained estimation model 232 to examine the moisture level of user 10's eyes (tear film stability, presence or absence of dry eye) from the captured images.

[0148] Preferably, as shown in Figure 17, the captured image includes a first still image at timing t1 immediately after blinking and a second still image at timing t2, after a predetermined time (for example, 5 seconds) has elapsed since blinking.

[0149] This allows the inspection system 1000 to examine the moisture level of user 10's eyes (tear film stability, presence or absence of dry eye) by comparing a still image taken immediately after blinking with a still image taken a predetermined time (for example, 5 seconds) after blinking.

[0150] Preferably, the first still image and the second still image are extracted from a video obtained by the camera 16 capturing the eye.

[0151] This allows user 10 to simply record a video of their eyes using camera 16, and the examination system 1000 can then extract two still images from the resulting video and compare them to examine the moisture level of user 10's eyes (tear film stability, presence or absence of dry eye). Therefore, user 10 does not need to take a still image with camera 16 immediately after blinking and a still image taken a predetermined time (for example, 5 seconds) after blinking, making it possible to examine the moisture level of their eyes (tear film stability, presence or absence of dry eye) as easily as possible.

[0152] Preferably, each of the first and second still images is a face image including at least the eyes. The eye image is extracted from the face image contained in each of the first and second still images.

[0153] This allows the inspection system 1000 to examine the moisture level of user 10's eyes (tear film stability, presence or absence of dry eye) by extracting an image of the eye from each of the two still images and comparing the two. Therefore, user 10 does not need to photograph only the eye area with the camera 16, and can have their eye moisture level (tear film stability, presence or absence of dry eye) examined as easily as possible.

[0154] Preferably, as shown in S4 and S5 of Figure 16, the processor 21 checks the degree of eye moisture (tear film stability, presence or absence of dry eye) based on the degree of fluctuation of the eye image extracted from the first still image and the degree of fluctuation of the eye image extracted from the second still image.

[0155] This allows the inspection system 1000 to examine the moisture level of user 10's eyes (tear film stability, presence or absence of dry eye) by comparing the degree of fluctuation of the eye images extracted from each of the two still images. Therefore, the inspection system 1000 can examine the moisture level of the eyes (tear film stability, presence or absence of dry eye) with greater accuracy.

[0156] Preferably, as shown in Figures 12(J1) and 13(K1), the display 15 further displays the eye image included in the captured image.

[0157] This allows user 10 to view the image of their own eye used for self-checking on the display 15. The image of the eye displayed on the display 15 may be the second still image at timing t2 shown in Figure 17.

[0158] Preferably, as shown in Figure 15(M2), the display 15 further displays information regarding recommended eye drops.

[0159] This allows user 10 to obtain information about the best eye drops for self-care of their eyes.

[0160] Preferably, as shown in Figure 15(M1), the display 15 further outputs information about the ophthalmology clinic.

[0161] This prompts user 10 to visit an ophthalmologist and to obtain information about ophthalmologists.

[0162] Preferably, as shown in Figure 15(M1), the information regarding ophthalmology clinics includes at least one of the following: information that encourages ophthalmology clinics to provide medical care, information regarding medical care provided at ophthalmology clinics, and information regarding recommended ophthalmology clinics.

[0163] This prompts user 10 to visit an ophthalmologist and allows them to obtain information about recommended ophthalmologists.

[0164] Preferably, as shown in Figures 12 and 13, the display 15 displays the results of a check for at least one of subjective symptoms related to the eyes and the possibility that the eyes have an ophthalmic disease, along with the test results based on the tests performed by the processor 11.

[0165] This allows user 10 to obtain the results of an eye moisture level test (tear film stability, presence or absence of dry eye) by considering at least one of the subjective symptom check results and the risk check results.

[0166] Preferably, as shown in Figure 2, the inspection system 1000 further comprises an inspection device 1 and a server device 2 capable of communicating with the inspection device 1. The inspection device 1 includes a camera 16 and a display 15. The server device 2 includes a processor 21 (inspection unit).

[0167] This allows user 10 to easily perform a self-check of their eyes using the inspection device 1 and server device 2 included in the inspection system 1000.

[0168] This disclosure relates to an inspection device 1a for examining the moisture level of the eye (tear film stability, presence or absence of dry eye). As shown in Figure 18, the inspection device 1a comprises a camera 16, a processor 11 (inspection unit) that examines the moisture level of the eye (tear film stability, presence or absence of dry eye) based on captured images obtained by the camera 16 photographing the eye and an estimation model 232 including a neural network 2321, and a display 15 that displays the inspection results from the processor 11.

[0169] This allows user 10 to objectively examine the moisture level of their eyes (tear film stability, presence or absence of dry eye) by taking a picture of their eyes using the camera 16 of the examination device 1a. In this way, user 10 can easily perform a self-check of their eyes using the examination device 1a.

[0170] This disclosure relates to an inspection device 1 for examining the moisture level of the eye (tear film stability, presence or absence of dry eye). As shown in Figure 2, the inspection device 1 comprises a camera 16, a communication device 12 for communicating with a server device 2 equipped with an estimation model 232 including a neural network 2321, and a display 15. The communication device 12 transmits the captured video obtained by the camera 16 capturing images of the eye to the server device 2. The communication device 12 receives the inspection results of the moisture level of the eye (tear film stability, presence or absence of dry eye) obtained by the server device 2 based on the captured images of the eye extracted from the captured video and the estimation model 232. The display 15 displays the inspection results.

[0171] This allows user 10 to objectively examine the moisture level of their eyes (tear film stability, presence or absence of dry eye) by taking a picture of their eyes using the camera 16 of the inspection device 1. In this way, user 10 can easily perform a self-check of their eyes using the inspection device 1.

[0172] This disclosure relates to an examination method for examining the moisture level of the eye (tear film stability, presence or absence of dry eye) using a processor 21 (computer). The examination method includes the steps of: inputting an image obtained by a camera 16 photographing the eye (S2 in Figure 16); examining the moisture level of the eye (tear film stability, presence or absence of dry eye) based on the image and an estimation model 232 including a neural network 2321 (S5 in Figure 16); and outputting the examination results from the examination step (S11 in Figure 16).

[0173] This allows user 10 to objectively examine the moisture level of their eyes (tear film stability, presence or absence of dry eye) by taking a picture of their eyes using the camera 16 of the inspection device 1. In this way, user 10 can easily perform a self-check of their eyes using the inspection device 1.

[0174] This disclosure relates to an examination program 231 for examining the moisture level of the eye (tear film stability, presence or absence of dry eye). The examination program 231 causes a processor 21 (computer) to receive an image captured by a camera 16 capturing an eye (S2 in Figure 16), to perform an examination of the moisture level of the eye (tear film stability, presence or absence of dry eye) based on the captured image and an estimation model 232 including a neural network 2321 (S5 in Figure 16), and to output the examination results from the examination step (S11 in Figure 16).

[0175] This allows user 10 to objectively examine the moisture level of their eyes (tear film stability, presence or absence of dry eye) by taking a picture of their eyes using the camera 16 of the inspection device 1. In this way, user 10 can easily perform a self-check of their eyes using the inspection device 1.

[0176] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of this disclosure is indicated by the claims rather than by the description of the embodiments above, and all modifications within the meaning and scope of the claims are intended to be included. [Explanation of symbols]

[0177] 1,1A,1B,1C,1a Inspection device, 2 Server device, 4 Training data, 5 Network, 10 User, 11,21 Processor, 12,22 Communication device, 13,23 Memory, 14 Input interface, 15 Display, 16 Camera, 31 Learning device, 131 Self-check app, 146,147,148,149,151,152,153,154,155,156,157,158,159,160 Icon, 211 Input unit, 212 Processing unit, 213 Output unit, 231 Inspection program, 232 Estimation model, 233 Eye drop information, 234 Ophthalmology clinic information, 235 User information, 236 Video data, 301, 302, 1511, 1512, 1521, 1522, 1531, 1532, 1541, 1542, 1543; Images, 1000, 1000a; Inspection system, 2321 neural network, 2322 parameters.

Claims

1. This is a testing system for examining the moisture level of the eyes. Camera and, An inspection unit that examines the moisture level of the eye based on the change in the degree of fluctuation of the object reflected on the surface of the eye, based on an estimated model including a neural network and an image captured by the camera taking a picture of the eye in natural light, An inspection system comprising a display that shows the inspection results from the aforementioned inspection unit.

2. The inspection system according to claim 1, wherein the estimation model is trained to inspect the moisture level of the eye from the captured image based on training data including the captured image and the results of the eye moisture level inspection.

3. The inspection system according to claim 1 or claim 2, wherein the captured image includes a first still image taken immediately after blinking and a second still image taken after a predetermined time has elapsed since blinking.

4. The inspection system according to claim 3, wherein each of the first still image and the second still image is extracted from a video obtained by the camera capturing the eye.

5. Each of the first still image and the second still image is an image of a face including at least the eyes, The inspection system according to claim 4, wherein the image of the eye is extracted from the image of the face contained in the first still image and the second still image, respectively.

6. The inspection system according to claim 5, wherein the inspection unit inspects the moisture level of the eye based on the difference between the degree of fluctuation of the object projected onto the surface of the eye extracted from the first still image and the degree of fluctuation of the object projected onto the surface of the eye extracted from the second still image.

7. The inspection system according to claim 1 or 2, wherein the display further displays the image of the eye included in the captured image.

8. The inspection system according to claim 1 or 2, wherein the display further displays information regarding recommended eye drops.

9. The examination system according to claim 1 or 2, wherein the display further outputs information related to an ophthalmology clinic.

10. The examination system according to claim 9, wherein the information regarding ophthalmology clinics includes at least one of the following: information that encourages consultations by ophthalmology clinics, information regarding consultations at ophthalmology clinics, and information regarding recommended ophthalmology clinics.

11. The examination system according to claim 1 or 2, wherein the display shows the result of checking at least one of subjective symptoms relating to the eye and the possibility that the eye has an ophthalmic disease, and the examination result based on the examination by the examination unit.

12. Inspection equipment and The inspection device further comprises a server device capable of communicating with the aforementioned inspection device, The inspection device includes the camera and the display, The inspection system according to claim 1 or claim 2, wherein the server device includes the inspection unit.

13. A testing device for examining the moisture level of the eyes, Camera and, An inspection unit that examines the moisture level of the eye based on the change in the degree of fluctuation of the object reflected on the surface of the eye, based on an estimated model including a neural network and an image captured by the camera taking a picture of the eye in natural light, An inspection device comprising a display for displaying the inspection results from the aforementioned inspection unit.

14. A testing device for examining the moisture level of the eyes, Camera and, A communication device for communicating with a server device equipped with an estimation model including a neural network, Equipped with a display, The aforementioned communication device is The camera captures the eye under natural light, and transmits the captured video, including the object reflected on the surface of the eye, to the server device. The server device receives the results of the eye moisture level test, which is performed based on the changes in the degree of fluctuation of the object projected onto the surface of the eye, using the captured image of the eye extracted from the captured video and the estimation model. The display is an inspection device that displays the inspection results.

15. A computer-based method for examining the moisture level of the eyes, The process involves inputting an image obtained by a camera photographing the eye in natural light, which includes an object projected onto the surface of the eye; The steps include: examining the degree of moisture in the eye based on the captured image and an estimation model including a neural network, from the change in the degree of fluctuation of the object projected onto the surface of the eye; An inspection method comprising the step of outputting the inspection results obtained from the inspection step.

16. This is a testing program that examines the moisture level of the eyes. On the computer, The process involves inputting an image obtained by a camera photographing the eye in natural light, which includes an object projected onto the surface of the eye; The steps include: examining the degree of moisture in the eye based on the captured image and an estimation model including a neural network, from the change in the degree of fluctuation of the object projected onto the surface of the eye; An inspection program that performs the steps of: performing the inspection step and outputting the inspection results from the aforementioned inspection step.

17. A testing system for examining the stability of the tear film, Camera and, An inspection unit that examines the stability of the tear film based on changes in the degree of fluctuation of the object projected onto the surface of the eye, using an estimated model including a neural network and an image captured by the camera when it photographs the eye in natural light, An inspection system comprising a display that shows the inspection results from the aforementioned inspection unit.

18. A testing system for checking for the presence or absence of dry eye, Camera and, An inspection unit that examines the presence or absence of dry eye based on changes in the degree of fluctuation of the object projected onto the surface of the eye, using an estimated model including a neural network and an image captured by the camera when it photographs the eye in natural light, An inspection system comprising a display that shows the inspection results from the aforementioned inspection unit.

Images