Visual function monitoring system and visual function monitoring method

Abstract

To provide a visual function monitoring system capable of continuously performing monitoring by using daily visual behavior of a user, and visual function monitoring method.SOLUTION: Simulated disorder data input means 2 receives input of data relating to the simulated visual disorder. Visual task presentation means 3 receives input of a visual task and presents the received visual task. Sensing means 4 senses visual behavior of the user. Learning model generation means 5 generates a learning model representing a degree of correlation between the visual function of the user and the visual behavior data by using the visual behavior data sensed during calibration and simulated disorder data. Storage means 6 stores the learning model, the visual behavior data sensed during monitoring, and evaluation data. Evaluation means 7 evaluates the visual function of the user based on the visual behavior data sensed during the monitoring and the learning model. Output means 8 outputs evaluation data relating to abnormality of the visual function.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for monitoring a user's visual function using the user's daily visual behavior. [Background technology]

[0002] A mobile device for performing a vision test is known (see Patent Document 1), in which an individual's vision is tested using a test pattern displayed on the screen of the mobile device, and parameters influencing the test result are evaluated before and / or during the test.

[0003] Also, a visual inspection device that improves the inspection accuracy when using a touch screen is known (see Patent Document 2). This is a device that includes a touch panel that presents a visual target to the subject and has the subject tap the visual target within a predetermined response time.

[0004] Furthermore, a system is known that allows elderly people to evaluate their own eyesight and hearing in a home environment (see Patent Document 3). In this system, graphics and text are shown to the subject on a display, and the subject's eye movements while looking at the graphics and text are collected and analyzed by an eye movement sensor to evaluate the subject's eyesight.

[0005] The techniques disclosed in Patent Documents 1 to 3 require the user to perform a specific visual task in order to carry out each visual function test, which may hinder continuous use. In addition, because only the results at the time of visual function testing are available, only intermittent evaluations can be performed, and dynamic changes in visual function cannot be captured. Continuous and continuous tracking of changes in visual function is extremely important in preventing severe visual function abnormalities. Furthermore, the techniques disclosed in Patent Documents 1 to 3 require the user to be voluntary and proactively involved in the system. In other words, the user must have a strong will to start the system and must spend a lot of time and effort to operate it, which tends to make using the system troublesome. Many visual function abnormalities are difficult to detect early because the initial subjective symptoms are mild, and as a result, there is a possibility that the user will not be willing to use the system in the first place. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Special Publication No. 2021-519195 [Patent Document 2] Patent Publication No. 2022-025240 [Patent Document 3] Special Publication No. 2018-500957 Summary of the Invention [Problem to be solved by the invention]

[0007] In view of the above circumstances, an object of the present invention is to provide a visual function monitoring system and a visual function monitoring method that are capable of continuous monitoring using a user's daily visual behavior. [Means for solving the problem]

[0008] In order to solve the above problems, the visual function monitoring system of the present invention includes visual behavior data obtained by sensing the visual behavior of a user under simulated visual impairment, a calibration unit that uses the simulated impairment data to generate a learning model that represents the degree of correlation between the user's visual function and the visual behavior data, and a monitoring unit that uses visual behavior data sensed under normal conditions and the learning model to capture changes in the user's visual function. That is, the system generates a learning model that represents the degree of correlation between the user's visual function and the visual behavior data, performs calibration, and then monitors the user's visual function using the learning model. Specifically, the system includes the following means. 1) A simulated disability data input means for accepting input of data relating to a simulated visual disability. 2) A sensing means for sensing the visual behavior of a user. 3) A learning model generation means for generating a learning model that uses visual behavior data obtained by sensing a user's visual behavior under visual impairment and simulated impairment data to generate a learning model that represents the degree of correlation between the user's visual function and the visual behavior data. 4) An evaluation method for evaluating a user's visual function using visual behavior data sensed from the user's visual behavior under normal conditions and a learning model. 5) A storage means for storing the learning model, visual behavior data sensed during monitoring, and evaluation data of the user's visual function. 6) An output means for outputting evaluation data regarding at least abnormalities in visual function.

[0009] In the calibration unit, the learning model generating means generates a customized learning model that is most suitable for the user, not for a wide range of users, regarding the degree of correlation between the user's visual function and visual behavior data, so that a highly accurate learning model can be generated. In this specification, visual function refers to input-related functions such as visual acuity, refraction, eye movement, binocular vision function, accommodation function, and color vision. Since the monitoring is performed after calibration in advance, the monitoring can be performed by utilizing the visual behavior that is performed unconsciously on a daily basis, and has the advantage that the user does not need to voluntarily operate the system, display anything, or interrupt the current task unless an abnormality in visual function is detected. In addition, by continuously monitoring daily visual behavior, visual function abnormalities can be detected early.

[0010] 1) Means of inputting simulated fault data The input of simulated disability data can be accepted, for example, by accepting input from a known PC or the like. Alternatively, input may be accepted from devices such as a smartphone or smart glasses through wired or wireless communication. A set of prescription lenses that change the optics of the human eye to induce refractive error is preferably used to simulate the visual disability that is the basis of the data input. Also, instead of lenses that change the refractive power, prism lenses that simulate strabismus or fog lenses that simulate cataracts can be used. These can monitor the signs of strabismus or cataracts rather than refractive errors. The simulated visual disability is not limited to refractive error, strabismus, or cataracts, but broadly includes disorders related to visual function.

[0011] 2) Sensing methods The visual behavior of the user to be sensed includes a wide range of behaviors using the visual function, regardless of the presence or absence of eye movement. Therefore, for example, it is not limited to eye movement in which the eyes move due to the action of extraocular muscles, or the movement of the skin, skeleton, or organs related to eye movement, but also states in which eye movement is not performed can be sensed as the visual behavior of the user. In addition, behaviors using the visual function include conscious or unconscious behaviors, and include not only waking but also sleeping. Visual behavior data is data obtained by sensing visual behavior, and is data that is the basis for evaluating the visual function, such as the user's refractive power, eye position, crystal clarity, and the presence or absence and degree of visual field defects. Specifically, examples include electrooculography, deformation data of the skin around the eyes, and data obtained from captured images of the eyes, including gaze measurement data. "Under visual impairment" refers to a state simulating a visual impairment, and "under normal circumstances" is intended to include a wide range of states other than those simulating a visual impairment, that is, states in daily life, and is not limited by time or place. As the sensing means, a wide variety of known visual behavior sensors can be applied, but an electro-oculogram (EOG) sensor is preferably used. An EOG sensor detects potential fluctuations accompanying eye movement from electrodes attached around the eye socket. Using this EOG sensor, the calibration unit records the visual behavior of the user while performing a visual task using a lens or the like, and the monitoring unit records the visual behavior of the user while using a device that executes this system in daily life.

[0012] In addition to the EOG sensor, optical reflection sensors that collect deformation data of the skin around the eye and optical eye trackers that collect data obtained from captured images of the eye, such as gaze measurement data, can be used alone or in combination. All of these record the subject's visual behavior while using the lenses during calibration, and while the subject uses the device that runs the system in their daily life during monitoring. The EOG sensor, optical reflection sensor, and eye tracker can all be unobtrusively integrated into the smart eyewear to collect data. For example, if the sensing means is installed in smart glasses, visual function tests can be continuously performed while the user is engaged in daily activities without bothering them. If any abnormality in vision is detected, the system can promptly notify the user, making it highly convenient.

[0013] 3) Means of generating learning models Using the visual behavior data and the simulated disability data, a learning model is generated that represents the degree of correlation between the visual function of the user and the visual behavior data. This makes it possible to generate a highly accurate learning model using only the visual behavior data of the user, without necessarily using visual behavior data of multiple users.

[0014] 4) Evaluation methods Since the evaluation is performed using a learning model adapted to the user, a highly accurate evaluation is possible. The evaluation is performed by comparing visual behavior data obtained by sensing the user's visual behavior under normal conditions with the visual behavior data stored in the learning model, and then evaluating the user's visual function as normal or abnormal based on the degree of correlation between the visual behavior data stored in the learning model and the user's visual function. The abnormality in visual function here is related to the visual impairment simulated in the calibration unit.

[0015] 5) Storage means It is also possible to store data other than the learning model, the visual behavior data sensed during monitoring, and the evaluation data. For example, simulated failure data during calibration, sensed visual behavior data, data related to visual tasks described later, etc. may be stored, and when calibration is performed multiple times, data related to past calibrations may be stored. Furthermore, it is also possible to store a learning model that represents the degree of correlation between the visual function and visual behavior data of other users.

[0016] 6) Output method Since the monitoring is performed continuously, the output means may output not only the evaluation data related to abnormalities in visual function, but may output all visual behavior data obtained in the measurement, including evaluation data related to normal visual behavior. The output method may be a wide variety of visual, auditory, or tactile methods. For example, the evaluation data may be displayed as a numerical value or a level, or the abnormality may be displayed as a message, notified by voice, or notified by a vibration function. The output may be displayed on the device itself in which the sensing means is provided, or may be displayed on another device, such as a smartphone, by wired or wireless communication.

[0017] In the visual function monitoring system of the present invention, it is preferable that the calibration unit further comprises a visual task presenting means for receiving an input of a visual task including at least one of a reading task, a fixation task, and a tracking task, and presenting the received visual task to the user. By providing the visual task presenting means, a more accurate learning model can be generated. The visual task induces the user's eye movement by moving the user's gaze in a predetermined direction or not moving the gaze for a predetermined period of time, and is used to measure the user's eye movement during calibration. Therefore, it is preferable that the content of the visual task covers daily visual behavior. The reading task is a task in which a predetermined text displayed on a display is read vertically or horizontally. The fixation task is a task in which the gaze is fixed on a target such as a pointer displayed on the display, and the tracking task is a task in which the gaze is moved in accordance with the movement of a target such as a pointer displayed on the display. The visual task is presented, for example, by displaying it on a display device such as a known display or VR goggles. The input of the visual task is accepted by a known PC or the like, and the visual task is preferably output from the PC or the like to a display or the like and presented by being displayed on the display or the like.

[0018] In the visual function monitoring system of the present invention, the simulated impairment data may be data on refractive error caused by a change in light induced on the eye, and the abnormality in the visual function of the user evaluated by the evaluation means may be a decrease in refractive power. With this configuration, the refractive error of the user can be detected with high accuracy. In the visual function monitoring system of the present invention, the visual behavior data may be at least any of data obtained from an image of the eye, including electrooculography, data on deformation of the skin around the eye, and gaze measurement data. The data obtained from an image of the eye includes not only gaze measurement data but also a wide range of image information related to the state of the eye other than the gaze, such as changes in the pupil of the eye.

[0019] In the visual behavior data used to generate a learning model in the learning model generation means of the visual function monitoring system of the present invention, and in the visual behavior data used for evaluation in the evaluation means, it is preferable that the sensed visual behavior data is preprocessed and normalized by removing DC components, high frequency components, and linear trends. The visual behavior data sensed by the calibration unit is pre-processed before generating a learning model. Similarly, the visual behavior data sensed by the monitoring unit is pre-processed before evaluation.

[0020] In the visual function monitoring system of the present invention, a calibration section is constituted by a simulated impairment data input means, a sensing means, a learning model generation means, and a storage means, and a monitoring section is constituted by a sensing means, an evaluation means, a storage means, and an output means. That is, among the above 1) to 6) means, 1) simulated fault data input means, 2) sensing means, 3) learning model generation means, and 5) storage means constitute a calibration unit, and 2) sensing means, 4) evaluation means, 5) storage means, and 6) output means constitute a monitoring unit, and 2) sensing means and 5) storage means are used for both the calibration unit and the monitoring unit. The calibration unit generates a model that best suits the user, rather than a wide range of users, regarding the degree of correlation between the user's visual function and visual behavior data, thereby generating a highly accurate learning model. In addition, the monitoring unit does not include 1) a means for inputting simulated disability data, and the visual function can be continuously evaluated using a device that the user wears on a daily basis.

[0021] The visual function monitoring method of the present invention includes a calibration step of generating a learning model expressing the degree of correlation between the user's visual function and the visual behavior data using visual behavior data obtained by sensing the visual behavior of a user under simulated visual impairment and the simulated impairment data, and a monitoring step of capturing a change in the user's visual function using the visual behavior data sensed under normal conditions and the learning model. Specifically, the method includes the following steps. A) Calibration Step A-1) A simulated disability data input step for accepting data input regarding a simulated visual disability. A-2) A first sensing step of sensing the visual behavior of a user under visual impairment. A-3) A learning model generation step for generating a learning model that represents the degree of correlation between the user's visual function and the visual behavior data using visual behavior data sensed from the user's visual behavior under visual impairment and simulated impairment data. A-4) A first memorization step of memorizing the learning model. B) Monitoring step B-1) A second sensing step of sensing the user's visual behavior under normal conditions. B-2) An evaluation step of evaluating the user's visual function using visual behavior data sensed from the user's visual behavior under normal conditions and a learning model. B-3) A second storage step for storing the visual behavior data and the evaluation data. B-4) An output step of outputting evaluation data regarding at least abnormalities in visual function.

[0022] In the visual function monitoring method of the present invention, a calibration step is provided before the monitoring step, thereby obtaining a learning model that represents the degree of correlation between the user's visual function and visual behavior data. This learning model is trained by recording visual behavior in a state where a visual disorder such as refractive error is simulated using a lens. This has the advantage that the user does not need to be actively involved in the monitoring process. Although it is sufficient to perform the calibration step only once, the accuracy can be improved by repeating the calibration step while the device is in use, i.e., during the monitoring step.

[0023] It is preferable that the visual function monitoring method of the present invention further comprises a visual task presenting step of accepting an input of a visual task including at least one of a reading task, a fixation task, or a tracking task, and presenting the accepted visual task. The learning model generated in the calibration step can be made more accurate by being trained by using lenses to simulate visual impairments such as refractive errors, performing visual tasks that cover everyday visual behavior, and recording the associated visual behavior.

[0024] In the visual function monitoring method of the present invention, the simulated impairment data is data relating to refractive error caused by induced changes in light on the eye, and the abnormality in the user's visual function evaluated in the evaluation step may be a decrease in refractive power. In the visual function monitoring method of the present invention, the visual behavior data may be at least any one of data obtained from captured images of the eye, including electrooculography, deformation data of the skin around the eye, and gaze measurement data.

[0025] In the visual function monitoring method of the present invention, the learning model generation step and the evaluation step perform preprocessing of the sensed visual behavior data, and it is preferable that the visual behavior data used to generate the learning model in the learning model generation step and the visual behavior data used as the evaluated visual behavior data in the evaluation step are both preprocessed data. Effect of the Invention

[0026] The visual function monitoring system and visual function monitoring method of the present invention have the advantage of being able to continuously monitor the user's daily visual behavior. [Brief description of the drawings]

[0027] [Figure 1] Functional block diagram of the visual function monitoring system according to the first embodiment [Diagram 2] Configuration image of the visual function monitoring system of the first embodiment [Diagram 3] Schematic flow diagram of the visual function monitoring method of Example 1 [Figure 4] Flowchart of preprocessing of visual behavior data [Diagram 5] Graph showing accuracy of visual tasks [Figure 6] Graph showing detection accuracy of the visual function monitoring system of Example 1 [Figure 7] Electrode arrangement diagram of EOG sensor [Figure 8] Electrode arrangement diagram of smart glasses DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. Note that the scope of the present invention is not limited to the following examples and illustrated examples, and many modifications and variations are possible. EXAMPLES

[0029] Fig. 1 shows a functional block diagram of a visual function monitoring system of Example 1. As shown in Fig. 1, the visual function monitoring system 1 is composed of a simulated disability data input means 2, a visual task presentation means 3, a sensing means 4, a learning model generation means 5, a storage means 6, an evaluation means 7, and an output means 8. The simulated disability data input means 2 receives input of data related to a simulated visual disability. The simulated disability data in this embodiment is data related to refractive error caused by induced changes in light on the eye. The visual task presentation means 3 receives input of visual tasks including at least one of a reading task, a fixation task, and a tracking task, and presents the received visual task. The sensing means 4 senses the visual behavior of the user. The visual behavior data in this embodiment is the electrooculogram.

[0030] The learning model generating means 5 uses the visual behavior data sensed during calibration and the simulated disability data to generate a learning model that represents the degree of correlation between the visual function and visual behavior data of the user. The storage means 6 stores the learning model, the visual behavior data sensed during monitoring, and the evaluation data. The evaluation means 7 evaluates the visual function of the user based on the visual behavior data sensed during monitoring and the learning model. In this embodiment, the abnormality of the user's visual function evaluated by the evaluation means 7 is a decrease in refractive power. The output means 8 outputs evaluation data related to at least the abnormality of the visual function. The simulated obstacle data input means 2, the visual task presentation means 3, the sensing means 4, the learning model generation means 5 and the storage means 6 constitute the calibration unit 1a. The sensing means 4, the storage means 6, the evaluation means 7 and the output means 8 constitute the monitoring unit 1b.

[0031] FIG. 2 shows a configuration image diagram of the visual function monitoring system of Example 1. As shown in FIG. 2, the visual function monitoring system 1 of Example 1 is composed of smart glasses 9, a PC 10, a smartphone 11, a display 30, a server 60, an eye examination frame 90, and lenses (90a, 90b). Specifically, the calibration unit 1a is composed of the PC 10, the display 30, the server 60, the eye examination frame 90, and lenses (90a, 90b). In addition, the monitoring unit 1b is composed of the smart glasses 9, the smartphone 11, and the server 60. The PC 10 and the display 30 communicate with each other by wire or wirelessly. In addition, the PC 10, the smartphone 11, and the server 60 can transmit and receive data via the Internet 12.

[0032] (About the calibration section) The trial frame 90 is provided with an eye movement sensor (not shown) as sensing means 4, which records the user's eye movement while the user performs a visual task using the lenses during calibration, and in this case, an EOG sensor is provided to measure the user's electrooculography. As an example of the EOG electrode configuration, two ground electrodes (GND) and eight electrodes are used to record the eye movements of both eyes. Figure 7 is an explanatory diagram of the electrode arrangement of the EOG sensor, where (1) shows a front view and (2) shows a right side view. Note that Figure 7 only shows the arrangement of the electrodes, and does not show the cables, eye trial frame 90, etc. As shown in Fig. 7(1), the electrodes for vertical and horizontal eye movements are arranged at vertical and horizontal positions around the eyes, i.e., at the positions above, below, left and right of each eye. Specifically, electrodes (13a, 13b) are arranged as vertical electrodes for the right eye 15a of the user 15, and electrodes (13c, 13d) are arranged as horizontal electrodes. Similarly, electrodes (13e, 13f) are arranged as vertical electrodes for the left eye 15b of the user 15, and electrodes (13g, 13h) are arranged as horizontal electrodes. Two ground electrodes (14a, 14b) are also placed. As shown in FIG. 7(1), ground electrode 14b is placed on the mastoid bone behind the ear to minimize inter-subject variability. Although not shown, ground electrode 14a is also placed on the mastoid bone behind the ear. In this embodiment, the electrodes (13a to 13h) and the ground electrodes (14a, 14b) are attached to the trial frame 90, but it is not essential that the electrodes are fixed to the trial frame. Therefore, for example, as shown in Fig. 7, the electrodes (13a to 13h) and the ground electrodes (14a, 14b) may be placed directly on the skin of the user 15, and then calibration may be performed by wearing a trial frame or the like that is not provided with a sensor.

[0033] 2 are provided with a short-range wireless communication device such as Bluetooth (registered trademark) (not shown), and measurement values ​​can be transmitted in real time to the PC 10. As described above, the sensing means 4 does not need to be fixed to the eye trial frame 90 or the smart glasses 9, and may be attached to the user's face or the like, separately from the eye trial frame 90, etc.

[0034] The lenses (90a, 90b) are a lens set that simulates visual function abnormalities, and are structured so that they can be attached to the trial frame 90 for eye examination. The trial frame 90 is similar to known eye examination glasses. Therefore, the power of the lenses attached to the trial frame 90 is not limited to one type, and lenses of many different powers can be combined to perform multiple visual tasks. Prism lenses or fog lenses may also be attached. Furthermore, a shielding plate may be used if necessary.

[0035] The PC 10 is provided with a simulated disability data input means 2 that accepts input of data related to a simulated visual disability, and a learning model generation means 5 that performs preprocessing of visual behavior data and training of a learning model that represents the degree of correlation between the user's visual function and visual behavior data. A deep neural network (DNN) is used as the learning model generation means 5. The server 60 also functions as a storage means 6 that stores the learning model. The display 30 displays the visual task, and the PC 10 and the display 30 constitute visual task presentation means 3. The PC 10 and the display 30 may be an integrated PC, or may be a tablet terminal, a smartphone, or the like.

[0036] (About the monitoring department) 2, the monitoring unit 1b of the first embodiment is realized by smart glasses 9 and a smartphone 11. The smart glasses 9 are provided with an eye movement sensor as the sensing means 4 for recording the eye movement of the user during monitoring, and here an EOG sensor for measuring the electrooculogram of the user is provided. FIG. 8 is an explanatory diagram of the electrode arrangement of smart glasses. As shown in FIG. 8, the electrodes in the monitoring unit 1b are arranged in the same manner as the electrodes in the calibration unit 1a (see FIG. 7), and the electrodes for vertical and horizontal eye movements are arranged in vertical and horizontal positions around the eyes, that is, in positions above, below, left and right of each eye. Specifically, for the right eye side electrodes, electrodes (13a, 13b) are arranged on the inner surface of the rim 9a as vertical electrodes, and electrode 13c is arranged as one of the horizontal electrodes. In addition, electrode 13d is arranged at the position of pad 9b as the other electrode in the horizontal direction. Similarly, for the left eye side electrodes, electrodes (13e, 13f) are arranged on the inner surface of the rim 9a as vertical electrodes, and electrode 13g is arranged as one of the horizontal electrodes. In addition, electrode 13h is arranged at the position of pad 9b as the other electrode in the horizontal direction. The two ground electrodes (14a, 14b) are positioned on the inner surface of the tip 9c, such that when the smart glasses 9 are worn, the ground electrodes (14a, 14b) are positioned on the mastoid bone behind the user's ear. Note that, unlike this embodiment, for example, any of the electrodes may be arranged on the temple 9d or the bridge 9e. The electrode arrangement shown in FIG.

[0037] The smart glasses 9 are equipped with a short-range wireless communication device such as Bluetooth (not shown), and can transmit measurement values ​​to the smartphone 11 in real time. The smartphone 11 includes an evaluation means 7 for evaluating the visual function of the user based on pre-processing of the visual behavior data and a learning model that represents the degree of correlation between the visual function of the user and the visual behavior data, and an output means 8 for outputting evaluation data of the visual function test. In addition, the server 60 functions as a storage means 6 for storing the visual behavior data and the evaluation data during monitoring.

[0038] Fig. 3 shows a schematic flow diagram of the visual function monitoring method of the first embodiment. As shown in Fig. 3, first, in the calibration step, input of data related to a simulated visual impairment is accepted (step S01: simulated impairment data input step). Specifically, the user inputs data related to the lenses (90a, 90b) attached to the eye test frame 90 to the PC 10, and the PC 10 accepts the data related to the lenses (90a, 90b). When performing calibration using a plurality of lens sets, data related to the replaced lenses (90a, 90b) is input to the PC 10 each time, and the PC 10 accepts this.

[0039] An input of a visual task is accepted, and the accepted visual task is presented (step S02: visual task presentation step). Specifically, a user inputs data related to the visual task to the PC 10, and the PC 10 accepts the data related to the visual task, and displays the accepted visual task on the display 30. A plurality of visual tasks may be input, and all of a reading task, a fixation task, and a tracking task may be input so that daily visual behavior can be comprehensively reproduced.

[0040] The visual behavior of the user during calibration is sensed (step S03: first sensing step). Specifically, a user (not shown) wears an eye trial frame 90 to which lenses (90a, 90b) are attached, and performs a visual task displayed on the display 30. Since an EOG sensor is provided in the eye trial frame 90, the electrooculography of the user performing the visual task can be measured in real time.

[0041] The sensed visual behavior data is preprocessed (step S04: first preprocessing step). FIG. 4 shows a flow diagram of preprocessing of visual behavior data. As shown in FIG. 4, the sensed raw data is input (step S21). Next, DC components are removed to facilitate frequency analysis (step S22). High frequency noise is removed (step S23). Linear trends are removed (step S24). Finally, the data is normalized to make it easier to handle (step S25). The flow of preprocessing of visual behavior data shown in FIG. 4 is applied to both the first preprocessing step (step S04) in the calibration step and the second preprocessing step (step S08) in the monitoring step.

[0042] Using the preprocessed visual behavior data, the simulated disability data, and the visual task, a learning model that represents the degree of correlation between the visual function and the visual behavior data of the user is generated (step S05: model generation step). The learning model is trained using a deep neural network.

[0043] The learning model is stored (step S06: first storage step). Specifically, data related to the learning model generated in the PC 10 is transmitted to the server 60 via the Internet 12, and the data related to the learning model received in the server 60 is stored. In this embodiment, the server 60 functions as the storage means 6, but in other forms, a storage device provided in the PC 10 may function as the storage means 6 here.

[0044] Next, in the monitoring step, the visual behavior of the user during monitoring is sensed (step S07: second sensing step). Specifically, the user goes about his or her daily life while wearing the smart glasses 9, and visually recognizes various real spaces 31. Since the smart glasses 9 are provided with an EOG sensor, the user's electrooculography can be continuously measured without the user being aware of it while going about his or her normal daily life. The sensed visual behavior data is transmitted in real time to the smartphone 11 carried by the user. Note that, unlike this, the sensed visual behavior data may be transmitted intermittently, such as being transmitted to the smartphone 11 at a predetermined timing.

[0045] The sensed visual behavior data is preprocessed (step S08: second preprocessing step). Specifically, the smartphone 11 preprocesses the received visual behavior data. The specific process of step S08 is the same as the preprocessing of step S04 as described above, and is as shown in FIG.

[0046] The visual function of the user is evaluated based on the preprocessed visual behavior data and the learning model (step S09: evaluation step). The smartphone 11 reads data related to the learning model from the server 60 and uses the data for the evaluation. Specifically, in response to a transmission request from the smartphone 11 to the server 60 for data related to the learning model, the server 60 transmits the data related to the learning model to the smartphone 11 via the Internet 12, and the smartphone 11 receives the data. Since the learning model represents the degree of correlation between the visual function and visual behavior data of the user, the smartphone 11 can evaluate the visual function of the user by comparing the learning model with the visual behavior data after preprocessing.

[0047] The visual behavior data during monitoring and the evaluation data are stored (step S10: second storage step). Specifically, the visual behavior data and the evaluation data are transmitted from the smartphone 11 to the server 60 via the Internet 12, and the received visual behavior data and the evaluation data are stored in the server 60. In this embodiment, the server 60 functions as the storage means 6, but in other forms, a storage device provided in the smartphone 11 may function as the storage means 6 here. Note that the visual behavior data to be stored may be only the visual behavior data before or after preprocessing, or may be the visual behavior data before and after preprocessing.

[0048] The evaluation data is output (step S11: output step). Specifically, the evaluation results may be displayed in a time series as graphs or numerical values ​​on the screen of the smartphone 11, or when an abnormality in visual function is detected, a notification of the abnormality may be given to the user by a screen display, a voice, a vibration, or the like. Data on the evaluation results may be sent from the smartphone 11 to the smart glasses 9, and the smart glasses 9 that receive the evaluation data may give a notification. Output data such as the number and frequency of notifications may be stored in a storage device provided in the smartphone 11, or may be stored in the server 60 via the Internet 12.

[0049] (About verification testing) A verification test was conducted using the visual function monitoring system of Example 1. Figure 5 is a graph showing the accuracy of visual tasks when the visual function monitoring system of Example 1 is used, and shows the accuracy of each of the fixation task, the tracking task, and the overall task combining the fixation task and the tracking task for five subjects (A to E). The closer the accuracy is to 1.00, the higher the accuracy is. As shown in FIG. 5, high accuracy was confirmed for all subjects (A to E) in the fixation task, the tracking task, and the overall task.

[0050] 6 is a graph showing the detection accuracy of the visual function monitoring system of Example 1. The vertical axis shows the actual measurement results, and the horizontal axis shows the evaluation results using the visual function monitoring system 1 of Example 1. The values ​​on both the vertical and horizontal axes represent myopia power (diopters). 6, the accuracy of agreement between the actual myopia power and the myopia power evaluated by the visual function monitoring system 1 at -3.0D was 0.999, the accuracy of agreement at -2.0D was 0.997, the accuracy of agreement at -1.0D was 0.978, the accuracy of agreement at 0D was 0.997, the accuracy of agreement at 1.0D was 0.990, the accuracy of agreement at 2.0D was 0.992, and the accuracy of agreement at 3.0D was 0.998. From the above, it was found that the accuracy of the evaluation by the visual function monitoring system 1 of Example 1 was high.

[0051] (Other Examples) 1) The smartphone 11 may include each of the means provided in the PC 10, the display 30, and the server 60 in the first embodiment. 2) In the calibration unit 1a, smart glasses 9 may be used instead of the trial frame 90. 3) Instead of using a wearable device such as the smart glasses 9 in the first embodiment, an in-camera provided on a smartphone or tablet terminal may function as the sensing means 4 to sense the daily usage state of the smartphone. Also, a single terminal such as a smartphone or tablet terminal may be configured to include all of the simulated disability data input means 2, visual task presentation means 3, sensing means 4, learning model generation means 5, storage means 6, evaluation means 7, and output means 8. In such a case, the visual tasks and evaluation results may be displayed on a display provided on the smartphone or the like. Also, a visual disability may be simulated on the display. [Industrial Applicability]

[0052] The present invention is useful as a technique for continuously testing a user's visual function during normal activities of daily life. [Explanation of symbols]

[0053] 1. Visual function monitoring system 1a Calibration section 1b Monitoring Department 2. Means of inputting simulated fault data 3 Visual task presentation methods 4. Sensing Methods 5. Learning model generation method 6 Memory means 7. Evaluation methods 8 Output Method 9. Smart Glasses 9a Rim 9b Pad 9c Modern 9d Temple 9e Bridge 10PCS 11 Smartphones 12. Internet 13a~13h Electrode 14a,14b Ground electrode 15 users 15a Right eye 15b left eye 30 Display 31 Real Space 60 Servers 90 Eye Test Frame 90a, 90b Lenses

Claims

1. A calibration unit that generates visual behavior data obtained by sensing the visual behavior of a user under a simulated visual impairment and generates a learning model that represents a degree of correlation between the visual function of the user and the visual behavior data using the simulated impairment data; A visual function monitoring system comprising a monitoring unit that uses visual behavior data sensed under normal conditions and the learning model to capture changes in a user's visual function.

2. The calibration unit includes: A simulated disability data input means for receiving input of data relating to a simulated visual disability; A sensing means for sensing a visual behavior of a user; A learning model generating means for generating the learning model using visual behavior data obtained by sensing the visual behavior of the user under the visual impairment and the simulated impairment data; Equipped with The monitoring unit is The sensing means; An evaluation means for evaluating the visual function of the user by using visual behavior data obtained by sensing the visual behavior of the user under normal conditions and the learning model; an output means for outputting evaluation data relating to at least visual function abnormality; The visual function monitoring system according to claim 1 , further comprising:

3. The visual function monitoring system according to claim 2, further comprising a memory means for storing the learning model, the visual behavior data sensed in the monitoring unit, and evaluation data of the user's visual function.

4. The calibration unit includes: The visual function monitoring system described in claim 2 or 3, further characterized in that it further comprises a visual task presentation means for accepting input of a visual task including at least one of a reading task, a fixation task, or a tracking task, and presenting the accepted visual task to the user.

5. The simulated impairment data is data relating to refractive errors caused by induced changes in light on the eye; 3. The visual function monitoring system according to claim 2, wherein the abnormality in the user's visual function is a decrease in refractive power.

6. The visual function monitoring system according to claim 2, characterized in that the visual behavior data is at least any one of data obtained from captured images of the eye, including electrooculography, data on deformation of the skin around the eye, and gaze measurement data.

7. The visual behavior data used for generating a learning model by the learning model generating means and the visual behavior data used for evaluation by the evaluation means, The sensed visual behavior data is pre-processed, 3. The visual function monitoring system according to claim 2, wherein the data is normalized by removing direct current components, high frequency components, and linear trends.

8. A calibration step of generating a learning model representing a degree of correlation between the user's visual function and the visual behavior data using visual behavior data obtained by sensing the user's visual behavior under a simulated visual impairment and the simulated impairment data; A visual function monitoring method comprising a monitoring step of capturing changes in a user's visual function using the visual behavior data sensed under normal conditions and the learning model.

9. The calibration step includes: a simulated disability data input step of accepting input of data relating to a simulated visual disability; A first sensing step of sensing a visual behavior of the user under the visual impairment; a learning model generation step of generating the learning model using visual behavior data obtained by sensing the visual behavior of the user under the visual impairment and the simulated impairment data; Equipped with The monitoring step includes: A second sensing step of sensing a visual behavior of the user under normal circumstances; An evaluation step of evaluating the visual function of the user using visual behavior data obtained by sensing the visual behavior of the user under normal conditions and the learning model; an output step of outputting evaluation data regarding at least abnormality in visual function; The visual function monitoring method according to claim 8, further comprising:

Images