Eye fatigue evaluation device, eye fatigue evaluation method, and program
The eye fatigue evaluation device analyzes eye shape features from facial images to accurately assess fatigue, addressing limitations of existing methods by providing timely alerts and reducing strain through statistical analysis and error correction.
Patent Information
- Application Number
- JP2020214134
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-12-23
- Publication Date
- 2024-11-11
- Estimated Expiration
- 2040-12-23
AI Technical Summary
Existing eye fatigue evaluation methods, such as blink measurement, are inadequate for accurately assessing eye fatigue, as they do not effectively capture the underlying biological changes in eye shape and movement that occur due to fatigue.
An eye fatigue evaluation device and method that calculates feature quantities representing the shape of the eyes based on feature points from facial images, using statistical distributions to determine eye fatigue levels, incorporating error correction for head movement and image variations.
Provides a more accurate assessment of eye fatigue by analyzing eye shape features, reducing the risk of eye strain and associated symptoms through timely alerts and breaks, applicable to various devices like smartphones and computers.
Smart Images

Figure 0007580265000004 
Figure 0007580265000005 
Figure 0007580265000006
Abstract
Description
[Technical field]
[0001] An embodiment of the present invention relates to an eye fatigue evaluation device, an eye fatigue evaluation method, and a program. [Background technology]
[0002] Conventionally, a technique for measuring blinking has been known as a technique for measuring eye fatigue (see, for example, Patent Document 1). This technique is a blink measuring device for determining a biological state of a measurement subject based on the blinking behavior of the measurement subject, and includes an imaging unit that captures an image of the measurement subject's eye, and an eyelid opening / closing measuring unit that calculates a blink feature amount related to the blinking behavior based on the image of the eye captured by the imaging unit, and determines the type of blinking behavior based on the blink feature amount. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2010-273954 A Summary of the Invention [Problem to be solved by the invention]
[0004] An object of the present invention is to provide an eye fatigue evaluation device, an eye fatigue evaluation method, and a program that are capable of evaluating eye fatigue. [Means for solving the problem]
[0005] One aspect of the present invention is an eye fatigue evaluation device for evaluating eye fatigue of a subject, the device including: a calculation unit that calculates a feature amount representing an eye shape based on information that identifies eye feature points obtained from an image of the subject's face; and a determination unit that determines eye fatigue of the subject based on the feature amount calculated by the calculation unit. The determination unit determines eye fatigue of the subject based on a statistic derived based on a distribution of features representing an eye shape calculated by the calculation unit at a predetermined time. This is an eye fatigue evaluation device.
[0006] One aspect of the present invention is a method for evaluating eye fatigue executed by a computer, the method comprising the steps of: calculating a feature amount representing an eye shape based on information identifying eye feature points obtained from an image of a face of a subject; and judging eye fatigue of the subject based on the feature amount calculated in the calculating step. In the determining step, eye fatigue of the subject is determined based on a statistic derived based on a distribution of the feature amount representing the eye shape calculated in the calculating step for a predetermined time. This is a method for evaluating eye fatigue.
[0007] According to one aspect of the present invention, a computer is caused to execute a step of calculating a feature amount representing an eye shape based on information identifying eye feature points obtained from an image of a face of a subject, and a step of determining eye fatigue of the subject based on the feature amount calculated in the step of calculating. In the determining step, eye fatigue of the subject is determined based on a statistic derived based on a distribution of the feature amount representing the eye shape calculated in the calculating step for a predetermined time. This is a program. Effect of the Invention
[0008] According to the embodiments of the present invention, it is possible to provide an eye fatigue evaluation device, an eye fatigue evaluation method, and a program capable of evaluating eye fatigue. [Brief description of the drawings]
[0009] [Figure 1] 1 is a diagram showing an eye fatigue evaluation system according to an embodiment of the present invention. [Diagram 2] 5 is a diagram showing an example of feature points detected by the eye fatigue evaluation device according to the present embodiment. FIG. [Figure 3A] FIG. 4 is a diagram illustrating an example of a process of the eye fatigue evaluation device according to the present embodiment. [Figure 3B] FIG. 4 is a diagram illustrating an example of a process of the eye fatigue evaluation device according to the present embodiment. [Figure 4] FIG. 2 is a diagram showing Example 1 of statistical data of aspect ratios calculated by the eye fatigue evaluation device according to the present embodiment. [Diagram 5] FIG. 11 is a diagram showing Example 2 of statistical data of aspect ratios calculated by the eye fatigue evaluation device according to the present embodiment. [Figure 6] 11 is a diagram showing an example of a time change in probability density distribution of a right eye aspect ratio calculated by the eye fatigue evaluation device according to the present embodiment. FIG. [Figure 7]1 is a diagram showing an example of an aspect ratio interval, a probability density, and an eye fatigue index calculated by the eye fatigue evaluation device according to the present embodiment. FIG. [Figure 8] 5 is a diagram showing an example of a time change in eye fatigue index calculated by the eye fatigue evaluation device according to the present embodiment. FIG. [Figure 9] FIG. 4 is a flow chart showing an example of the operation of the eye fatigue evaluation system according to the present embodiment. [Figure 10] FIG. 13 is a diagram showing an example of a display of an evaluation result of the eye fatigue evaluation system according to the present embodiment. [Figure 11] 13A and 13B are diagrams illustrating an example of a process of an eye fatigue assessment device according to a modified example of an embodiment. [Figure 12] FIG. 11 is a diagram showing Example 1 of statistical data of the right opening angle calculated by the eye fatigue evaluation device according to the modified example of the embodiment. [Figure 13] FIG. 11 is a diagram showing Example 2 of statistical data of the left opening angle calculated by the eye fatigue evaluation device according to the modified example of the embodiment. [Figure 14] 13A and 13B are diagrams showing an example of intervals, probability densities, and eye fatigue indexes of opening angles calculated by the eye fatigue evaluation device according to the modified example of the embodiment. [Figure 15] FIG. 11 is a diagram showing an example of a time change in eye fatigue index calculated by an eye fatigue evaluation device according to a modified example of an embodiment. [Figure 16] FIG. 11 is a flowchart showing an example of the operation of the eye fatigue evaluation system according to the modified example of the embodiment. [Figure 17] FIG. 13 is a diagram showing an example of a display of an evaluation result of the eye fatigue evaluation system according to the modified example of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] <Embodiment> Hereinafter, an eye fatigue evaluation system according to an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a diagram showing the eye fatigue evaluation system according to the present embodiment. The eye fatigue evaluation system 100 includes an imaging unit 1, a display device 2, and an eye fatigue evaluation device 10. The imaging unit 1 and the display device 2 are connected to the eye fatigue evaluation device 10. The imaging unit 1 captures an image of a subject S. The display device 2 displays the image captured by the imaging unit 1. The eye fatigue evaluation device 10 allows the subject S to check the positional relationship between the imaging unit 1 and the face of the subject S by displaying an image captured by the imaging unit 1 on the display device 2. The eye fatigue evaluation device 10 displays a gaze point on the display device 2 and has the subject S gaze at the gaze point. This configuration allows the subject S to keep his / her line of sight constant. The eye fatigue evaluation device 10 evaluates the eye fatigue of the subject S based on the image captured by the imaging unit 1.
[0011] In one example of the eye fatigue evaluation system 100, the imaging unit 1 is fixed on a desk. The subject S sits on a chair and gazes at the gaze point displayed on the display device 2. The imaging unit 1 creates multiple image data of the subject S by imaging the subject S for a predetermined time. In this embodiment, the explanation will be continued assuming that the predetermined time is 60 seconds as an example. The eye fatigue evaluation device 10 acquires multiple image data of the subject S created by the imaging unit 1. If the imaging unit 1 can capture 30 frames per second, 1800 frames can be captured in 60 seconds, so the eye fatigue evaluation device 10 acquires approximately 1800 pieces of image data. The eye fatigue evaluation device 10 records the acquired multiple image data of the subject S.
[0012] The eye fatigue assessment device 10 calls up the recorded image data into a memory and processes the image data called up into the memory to detect the face of the subject S from each of the images of the subject S. For example, the eye fatigue assessment device 10 obtains information specifying a rectangle (quadrangle) enclosing the face of the subject S as a result of detecting the face of the subject S from the image of the subject S. The eye fatigue evaluation device 10 detects facial feature points (landmarks) of the subject S from each of the multiple images of the subject S based on information identifying a rectangle enclosing the face of the subject S obtained from each of the multiple images of the subject S.
[0013] The eye fatigue evaluation device 10 detects eye feature points of the subject S from each of the multiple images of the subject S based on the detection results of the facial feature points of the subject S obtained from each of the multiple images of the subject S. The eye fatigue evaluation device 10 calculates features representing the shape of the subject S's eyes for each of the multiple images of the subject S based on the detection results of the feature points of the subject S's eyes obtained from each of the multiple images of the subject S. The eye fatigue evaluation device 10 evaluates the eye fatigue of the subject S based on feature amounts representing the eye shape of the subject S calculated for each of a plurality of images of the subject S.
[0014] The eye fatigue evaluation device 10 will be described in detail below. The eye fatigue evaluation device 10 includes an input / output I / F 11, a CPU (Central Processing Unit) 12, a memory 13, a recording device 14, a data reading unit 15, and a bus 16. The bus 16 connects the input / output I / F 11, the CPU 12, the memory 13, the recording device 14, and the data reading unit 15 to one another. The input / output I / F 11 is an interface for inputting and outputting information between the image capturing unit 1 and the display device 2 and between the eye fatigue evaluation device 10 and the image capturing unit 1 and the display device 2 . The data reading unit 15 acquires image data of the subject S from the imaging unit 1 via the bus 16. The data reading unit 15 causes the acquired image data to be recorded in the recording device 14 in association with the identification information of the subject S.
[0015] The CPU 12 executes a computer program (software) stored in the recording device 14 to function as a processing unit 12-1, a first detection unit 12-2, a second detection unit 12-3, a calculation unit 12-4, a determination unit 12-5, and a warning unit 12-6. The processing unit 12-1 acquires a plurality of image data that are recorded in association with the identification information of the subject S from among the image data recorded in the recording device 14. The processing unit 12-1 calls up the acquired plurality of image data into the memory 13. The processing unit 12-1 processes each of the plurality of image data called up into the memory 13 to acquire an image of the subject S.
[0016] The first detection unit 12-2 acquires a plurality of images of the subject S from the processing unit 12-1. The first detection unit 12-2 executes software stored in the recording device 14 to detect the face of the subject S from each of the acquired plurality of images of the subject S. As face detection software, various software are provided as libraries. In this embodiment, the description will continue with a case where OpenCV's Haar Cascades is used as an example of face detection software. OpenCV's Haar Cascades provides both a learning machine and a detector. In this embodiment, the learning machine has previously learned the relationship between images of people and images of people's faces based on multiple images of people and images of people's faces contained in each of the multiple images of people, and we will continue to explain the case where the detector detects images of people's faces from images of people using the learning results of the learning machine's relationship between images of people and images of people's faces.
[0017] As a result of detecting the face of the subject S from each of the multiple images captured of the subject S, the first detection unit 12-2 detects a rectangle (quadrant) enclosing the facial portion of the subject S in each of the multiple images captured of the subject S. Here, an example of information specifying the rectangle enclosing the facial portion of the subject S is coordinates VC(x, y, w, h). In the information specifying the rectangle enclosing the facial portion of the subject S, x and y are information specifying the X-coordinate and Y-coordinate of the upper left corner of the rectangle, respectively, and w and h are information specifying the width and height of the rectangle, respectively. The first detection unit 12-2 acquires coordinates VC from each of the multiple images captured of the subject S. By acquiring the coordinates VC from each of the multiple images captured of the subject S, the first detection unit 12-2 can acquire (identify) the position of the face of the subject S from each of the multiple images captured of the subject S based on each of the acquired coordinates VC. The first detection unit 12-2 may change the size of the image of the subject S based on each of the acquired coordinates VC in order to make the size of the rectangle represented by the coordinates VC constant. Below, the case where the first detection unit 12-2 changes the size of the image of the subject S based on each of the coordinates VC will be described. By configuring in this way, it is possible to correct errors caused by differences in the size of the rectangle surrounding the face of the subject S specified by the coordinates VC.
[0018] The second detection unit 12-3 acquires from the first detection unit 12-2 a plurality of images of the subject S and information specifying the coordinates VC corresponding to each of the plurality of images of the subject S. The second detection unit 12-3 detects an image of the face of the subject S from each of the plurality of images of the subject S based on the acquired plurality of images of the subject S and a plurality of pieces of information specifying the coordinates VC. The second detection unit 12-3 detects facial feature points of the subject S from each of the detected images of the face of the subject S. In this embodiment, as an example, the description will continue of a case where an ensemble of regression trees is used to detect facial feature points of the subject S. By using the ensemble of regression trees, it is possible to detect facial feature points of the subject S with high accuracy in real time using regression tree analysis. In this embodiment, the learning machine has previously learned the relationship between the human face images and the human face feature points based on the human face images and the human face feature points contained in each of the multiple human face images, and the detector detects the human face feature points from the human face images using the learning result of the relationship between the human face images and the human face feature points by the learning machine. In this embodiment, the explanation will be continued for the case where 68 human face feature points are detected from the human face image, as an example.
[0019] The second detection unit 12-3 detects 68 feature points FP01 to FP68 from one face image of the subject S. The second detection unit 12-3 acquires the coordinates of each of the detected feature points FP01 to FP68. Here, an example of the coordinates is a coordinate for identifying the position of the subject S in the captured image. 2 is a diagram showing an example of feature points detected by the eye fatigue evaluation device according to the present embodiment, in which each of the 68 feature points is indicated by a point and a number that identifies the point. The second detection unit 12-3 acquires the coordinates of each of the feature points corresponding to the left eye and the coordinates of each of the feature points corresponding to the right eye from the coordinates of each of the acquired feature points FP01 to FP68. Specifically, the second detection unit 12-3 acquires the coordinates of each of the feature points FP37 to FP42 corresponding to the left eye, and acquires the coordinates of each of the feature points FP43 to FP48 corresponding to the left eye. The second detection unit 12-3 creates array data including information specifying the coordinates of each of the acquired feature points FP37 to FP42 and coordinates specifying the coordinates of each of the acquired feature points FP43 to FP48. Returning to FIG. 1, the explanation will be continued.
[0020] The calculation unit 12-4 acquires the array data created by the second detection unit 12-3. The calculation unit 12-4 acquires the coordinates of each of the multiple feature points corresponding to the left eye portion and the coordinates of each of the multiple feature points corresponding to the right eye portion included in the acquired array data. The calculation unit 12-4 calculates a feature amount representing the shape of the left eye and a feature amount representing the shape of the right eye based on the coordinates of each of the multiple feature points corresponding to the left eye portion and the coordinates of each of the multiple feature points corresponding to the right eye portion acquired. In this embodiment, the aspect ratio represented by the ratio of the length between the height and width of the left eye image (hereinafter referred to as the "left eye aspect ratio") is used as an example of the feature amount representing the shape of the left eye, and the aspect ratio represented by the ratio of the length between the height and width of the right eye image (hereinafter referred to as the "right eye aspect ratio") is used as an example of the feature amount representing the shape of the right eye.
[0021] 3A and 3B are diagrams showing an example of the process of the eye fatigue evaluation device according to the present embodiment, in which feature points P1 to P6 corresponding to the right eye are shown as an example. As shown in Fig. 3A, an example of the feature point P1 (corresponding to FP37 in Fig. 2) corresponds to the outer corner of the right eye, an example of the feature point P2 (corresponding to FP38 in Fig. 2) corresponds to the intersection of the right boundary between the white and black of the eye and the upper eyelid in the right eye, and examples of the feature point P2 (corresponding to FP38 in Fig. 2) and the feature point P3 (corresponding to FP39 in Fig. 2) correspond to the points that divide the boundary between the upper eyelid and the eye into three equal parts between the outer corner of the eye at the feature point P1 and the inner corner of the eye at the feature point P4 in the right eye. As shown in Fig. 3A, an example of the feature point P4 (corresponding to FP40 in Fig. 2) corresponds to the inner corner of the eye in the right eye, and examples of the feature point P5 (corresponding to FP41 in Fig. 2) and the feature point P6 (corresponding to FP42 in Fig. 2) correspond to the points that divide the boundary between the lower eyelid and the eye into three equal parts between the outer corner of the eye at the feature point P1 and the inner corner of the eye at the feature point P4 in the right eye.
[0022] Calculation unit 12-4 calculates the right eye aspect ratio REAR using equation (1) based on the coordinates of each of feature points P1 to P6 corresponding to the right eye. REAR=(||P2-P6||+||P3-P5||) / 2||P1-P4|| (1) The calculation unit 12-4 calculates the left eye aspect ratio LEAR based on the coordinates of each of the six feature points corresponding to the left eye. The details of the process by which the calculation unit 12-4 calculates the left eye aspect ratio LEAR are omitted here because the process for calculating the right eye aspect ratio REAR described above can be applied. The calculation unit 12-4 estimates the movement (direction) of the head of the subject S with respect to each of the X-axis, Y-axis, and Z-axis using the coordinates of a predetermined feature point among the feature points FP01 to FP68 of the human face in order to correct an error caused by a difference in the positional relationship between the subject S and the imaging unit 1 in each of a plurality of images captured of the subject S. The calculation unit 12-4 performs error correction for each of the left eye aspect ratio and the right eye aspect ratio based on the estimation result of the movement of the head of the subject S with respect to each of the X-axis, Y-axis, and Z-axis. The calculation unit 12-4 records the result of performing error correction for each of the left eye aspect ratio and the right eye aspect ratio in the recording device 14. Hereinafter, the corrected left eye aspect ratio and right eye aspect ratio will be used.
[0023] Fig. 4 is a diagram showing an example 1 of statistical data of the aspect ratio calculated by the eye fatigue evaluation device according to the present embodiment. In Fig. 4, the calculated aspect ratio data is shown in a histogram that visualizes the relative frequency. The same applies to the histograms shown below. In reality, the calculated aspect ratio data is considered to vary infinitely and continuously within a certain interval, and can be expressed as a graph of the probability density distribution of the aspect ratio, as shown by the solid line. Methods for estimating the probability density distribution of the aspect ratio include histogram density estimation and kernel density estimation, and the estimation method in FIG. 4 was performed using kernel density estimation. The mathematical formula for kernel density estimation is given by Equation (2).
number
number
number
[0024] 4, (1) shows an example of the probability density distribution of the left eye aspect ratio. According to this figure, the area surrounded by the probability density distribution of the left eye aspect ratio and the X-axis is 1. In FIG. 4, (2) shows another example of the probability density distribution of the left-eye aspect ratio. The determination unit 12-5 acquires information specifying each of the left eye aspect ratios calculated by the calculation unit 12-4. The determination unit 12-5 calculates an index indicating eye fatigue of the left eye (hereinafter referred to as a "left eye eye fatigue index") based on the acquired information specifying each of the left eye aspect ratios. Specifically, the determination unit 12-5 calculates the ratio between the peak and the range of aspect ratios corresponding to a predetermined ratio in the probability density distribution of the left-eye aspect ratio. In this embodiment, the description will continue with a case where 95% (probability distribution is from 2.5% to 97.5%) is applied as an example of the predetermined ratio.
[0025] When 95% is applied as the specified rate, as shown in equation (2), judgment unit 12-5 defines the range of the left eye aspect ratio where the probability distribution is 2.5% to 97.5% as xl, the range of 2.5% to 97.5% of the maximum value in the probability density function of the left eye aspect ratio as yl, and determines the ratio of xl to yl as an index of left eye fatigue. Specifically, since the peak of the probability density distribution of the left eye aspect ratio is 18.4195 and the range of aspect ratios corresponding to 95% is 0.0789, determining unit 12-5 calculates 233.3 as the left eye fatigue index. Left eye fatigue index = (value at which the probability density function of the aspect ratio of the left eye is maximized (yl value)) / (range of 2.5% or more in the probability distribution of the aspect ratio (xl value)) (2)
[0026] Fig. 5 is a diagram showing Example 2 of statistical data of aspect ratios calculated by the eye fatigue evaluation device according to this embodiment. Fig. 5 shows, as an example, a probability density distribution of the right eye aspect ratio before starting VDT work. In Fig. 5, the horizontal axis represents the random variable, and the vertical axis represents the probability density at that time. To obtain statistical data on the right eye aspect ratio, subject S was imaged for 60 seconds. Of the image data obtained during the 60 seconds of imaging, 1,000 points of data from 20 seconds after the start of imaging, when eye and head movements are assumed to be relatively stable, were used to calculate a probability density function.
[0027] 5, (1) shows an example of the probability density distribution of the right-eye aspect ratio. According to this figure, the area surrounded by the probability density distribution of the right-eye aspect ratio and the X-axis is 1. In FIG. 5, (2) shows another example of the probability density distribution of the right-eye aspect ratio. The determination unit 12-5 acquires information specifying each of the right-eye aspect ratios calculated by the calculation unit 12-4. The determination unit 12-5 calculates an index indicating right-eye fatigue (hereinafter referred to as a "right-eye fatigue index") based on the acquired information specifying each of the right-eye aspect ratios. Specifically, the determination unit 12-5 calculates the ratio between the peak and the range of aspect ratios corresponding to a predetermined ratio in the probability density distribution of the right-eye aspect ratio. In this embodiment, the description will continue with a case where 95% (probability distribution is from 2.5% to 97.5%) is applied as an example of the predetermined ratio.
[0028] When 95% is applied as the specified ratio, as shown in equation (3), the judgment unit 12-5 defines the range of the right eye aspect ratio where the probability distribution is 2.5% to 97.5% as xr, the range of 2.5% to 97.5% of the maximum value in the probability density function of the right eye aspect ratio as yr, and determines the ratio of xr to yr as an index of right eye fatigue. Specifically, since the peak of the probability density distribution of the right eye aspect ratio is 19.0934 and the range of aspect ratios corresponding to 95% is 0.0807, determining unit 12-5 calculates 236.5 as the right eye fatigue index. Right eye eye fatigue index = (value at which the probability density function of the aspect ratio of the right eye is maximized (yr value)) / (range of 2.5% or more in the probability distribution of the aspect ratio (xr value)) (3) Returning to Figure 1, we will continue with the explanation.
[0029] The determining unit 12-5 performs evaluation by determining eye fatigue of the subject S based on the calculated right eye eye fatigue index and left eye eye fatigue index. FIG. 6 is a diagram showing an example of a change over time in the probability density distribution of the right eye aspect ratio calculated by the eye fatigue evaluation device according to the present embodiment. As an example, Fig. 6 shows the probability density distribution of the right eye aspect ratio acquired every hour from before the start of VDT work until four hours have passed since the start of VDT work. In Fig. 6, the horizontal axis represents the random variable, and the vertical axis represents the probability density at that time. Furthermore, FIG. 6 also shows the change over time in the right eye eye fatigue index calculated based on the probability density distribution of the right eye aspect ratio obtained every hour.
[0030] According to Fig. 6, it can be seen that the maximum value (peak) (yr value) of the probability density function for the right eye aspect ratio is stable and large at a certain aspect ratio before VDT work begins (when eye fatigue is small) compared to after VDT work begins. In other words, it can be seen that as VDT work progresses, the maximum value of the probability density function for the right eye aspect ratio decreases, resulting in a broader probability density distribution (probability density at the most frequent value decreases). In other words, the degree of right eye fatigue can be determined based on the maximum value of the probability density function for the right eye aspect ratio.
[0031] An example of the change over time in the probability density distribution of the left eye aspect ratio is omitted because it shows the same tendency as the right eye aspect ratio described above. As with the maximum value (peak) of the probability density function for the left eye aspect ratio (yl value), the maximum value (peak) of the probability density function for the left eye aspect ratio also stably takes a large value at a certain aspect ratio before the start of VDT work (when eye fatigue is small) compared to after the start of VDT work. In other words, it can be seen that as the time of VDT work progresses, the maximum value of the probability density function for the left eye aspect ratio becomes lower, resulting in a broad probability density distribution (probability density at the most frequent value decreases). In other words, the degree of fatigue of the left eye can be known based on the maximum value of the probability density function for the left eye aspect ratio.
[0032] FIG. 7 is a diagram showing an example of aspect ratio intervals, probability densities, and eye fatigue indexes calculated by the eye fatigue evaluation device according to the present embodiment. 7, the right eye fatigue index is 236.6 before the start of VDT work, 171.6 one hour after the start of VDT work, 141.3 two hours after the start of VDT work, 122.8 three hours after the start of VDT work, and 90.6 four hours after the start of VDT work. It can be seen that the right eye fatigue index decreases according to the time that has passed since the start of VDT work.
[0033] The left eye fatigue index was 233.5 before the start of VDT work, 183.1 one hour after the start of VDT work, 138.4 two hours after the start of VDT work, 125.5 three hours after the start of VDT work, and 74.4 four hours after the start of VDT work. It can be seen that the left eye fatigue index decreases according to the time that has elapsed since the start of VDT work.
[0034] The decrease in the right eye fatigue index and left eye fatigue index is assumed to be due to eye fatigue caused by long-term use of VDT devices such as smartphones and computers. When concentrating on staring at a screen and the state of eyeballs not moving continues, the seven muscles that move the eyeballs and the muscles that adjust the thickness of the "crystalline lens" to focus become fatigued. In addition, the number of times the eyes blink decreases during VDT work. Furthermore, the decrease in the right eye fatigue index and left eye fatigue index is thought to be due to dry eyes, which make tears evaporate more easily when wearing contact lenses or in an air-conditioned environment. This causes an increase in the probability of the aspect ratio becoming smaller, which increases the xr value in the denominator of the right eye fatigue index and the xl value in the denominator of the left eye fatigue index, causing a decrease in the right eye fatigue index and left eye fatigue index, which indicate an increase in eye fatigue.
[0035] In addition, the determination unit 12-5 performs statistical processing based on the right eye fatigue index and the left eye fatigue index to calculate an eye fatigue index that is an index of eye fatigue of both the right eye and the left eye combined. In this embodiment, the explanation will continue with the case where an average value is used as an example of statistical processing. 7, the eye fatigue index (average of left and right eyes) is 235.0 before the start of VDT work, 177.3 one hour after the start of VDT work, 139.8 two hours after the start of VDT work, 124.1 three hours after the start of VDT work, and 82.4 four hours after the start of VDT work. It can be seen that the fatigue index decreases with the time that has elapsed since the start of VDT work, just like the right eye fatigue index and left eye fatigue index.
[0036] 8 is a diagram showing an example of a change over time in the eye fatigue index calculated by the eye fatigue evaluation device according to the present embodiment, in which the horizontal axis represents the elapsed time from the start of the VDT work, and the vertical axis represents the eye fatigue index. According to FIG. 8, it can be seen that the eye fatigue index decreases by about 25% after one hour has elapsed since the start of the VDT work. For this reason, it is preferable that the eye fatigue assessment device 10 has the subject S stop the VDT work and take a rest for a certain period of time when the eye fatigue index decreases by 25% or more from the initial value. Before resuming the work, the eye fatigue assessment device 10 calculates the right eye fatigue index and the left eye fatigue index of the subject S again, and judges whether the calculated fatigue index has decreased by 25% or more based on the calculated right eye fatigue index and the left eye fatigue index. The eye fatigue assessment device 10 performs the same process when the fatigue index has decreased by 25% or more based on the calculated fatigue index thereafter. It is assumed that by configuring in this way, the subject S can prevent symptoms such as eye pain, decreased eyesight, stiff shoulders, and headaches. Returning to FIG. 1, the explanation will be continued.
[0037] The warning unit 12-6 acquires the eye fatigue index calculated by the determination unit 12-5. The warning unit 12-6 judges whether the acquired eye fatigue index is less than the eye fatigue index threshold. An example of the eye fatigue index threshold is a value 25% lower than the initial value of the eye fatigue index. The warning unit 12-6 issues a warning when the eye fatigue index is less than the eye fatigue index threshold. Specifically, the warning unit 12-6 issues a warning to the subject S or the administrator of the subject S.
[0038] (Operation of eye fatigue evaluation system 100) FIG. 9 is a flow chart showing an example of the operation of the eye fatigue evaluation system according to the present embodiment. (S1-1) The eye fatigue assessment device 10 displays an image captured by the imaging unit 1 on the display device 2. The eye fatigue assessment device 10 displays a gaze point on the display device 2 and has the subject S gaze at the gaze point. (S2-1) The imaging unit 1 creates image data of the subject S by imaging the subject S for a predetermined period of time. The data reading unit 15 acquires the image data of the subject S from the imaging unit 1 via the bus 16, and records the acquired image data in the recording device 14 in association with the identification information of the subject S. (S3-1) In the eye fatigue evaluation device 10, the processing unit 12-1 calls up image data from the recording device 14 to the memory 13. The processing unit 12-1 acquires an image of the subject S by processing the image data called up to the memory 13. The first detection unit 12-2 acquires the image of the subject S from the processing unit 12-1. The first detection unit 12-2 detects the face of the subject S from the acquired image of the subject S.
[0039] (S4-1) In the eye fatigue evaluation device 10, the first detection unit 12-2 changes the size of the image of the subject S. (S5-1) In the eye fatigue evaluation device 10, the second detection unit 12-3 acquires the image and coordinates VC of the subject S from the first detection unit 12-2. The second detection unit 12-3 detects the face of the subject S based on the acquired image and coordinates VC of the subject S, and detects feature points of the subject S's face from the detected image of the subject S's face. (S6-1) In the eye fatigue evaluation device 10, the second detection unit 12-3 acquires the coordinates of the feature points corresponding to the left eye portion and the coordinates of the feature points corresponding to the right eye portion from the coordinates VC of each of the acquired feature points. The calculation unit 12-4 acquires the coordinates of the feature points corresponding to the left eye portion and the coordinates of the feature points corresponding to the right eye portion acquired by the second detection unit 12-3. The calculation unit 12-4 calculates the left eye aspect ratio and the right eye aspect ratio of the subject S based on the acquired coordinates of the feature points corresponding to the left eye portion and the coordinates of the feature points corresponding to the right eye portion.
[0040] (S7-1) In the eye fatigue evaluation device 10, the calculation unit 12-4 estimates the movement (orientation) of the head of the subject S with respect to each of the X-axis, Y-axis, and Z-axis by using the coordinates of certain feature points among feature points FP01 to FP68 on a human face. (S8-1) In the eye fatigue evaluation device 10, the calculation unit 12-4 performs error correction for each of the left eye aspect ratio and the right eye aspect ratio based on the estimated results of the movement (direction) of the head of the subject S with respect to each of the X-axis, Y-axis, and Z-axis. (S9-1) In the eye fatigue evaluation device 10, the determination unit 12-5 calculates a right eye eye fatigue index based on the probability density distribution of the right eye aspect ratio. The calculation unit 12-4 calculates a left eye eye fatigue index based on the probability density distribution of the left eye aspect ratio. The determination unit 12-5 calculates the eye fatigue index by performing statistical processing based on the calculated right eye eye fatigue index and left eye eye fatigue index. (S10-1) In the eye fatigue evaluation device 10, the judgment unit 12-5 judges the degree of eye fatigue based on the calculated eye fatigue index. 9, the order of steps S6-1 and S7-1 may be interchanged, i.e., steps S1-1 to S5-1, S7-1, S6-1, S8-1 to S10-1 may be executed in this order.
[0041] In the above embodiment, the first detection unit 12-2 detects the face of the subject S from the image of the subject S using OpenCV Haar Cascades, but the present invention is not limited to this example. For example, the first detection unit 12-2 may detect the face of the subject S from the image of the subject S using an Active Appearance Model. The Active Appearance Model performs object detection based on a statistical model learned from the shape and appearance of an object. For example, the first detection unit 12-2 may detect the face of the subject S from the image of the subject S using Local Binary Features. The Local Binary Features detect features very quickly by regression learning. In the above-described embodiment, the first detection unit 12-2 detects the face of one subject S from the image of the subject S, but the present invention is not limited to this example. For example, the first detection unit 12-2 may detect the faces of each of multiple subjects S from the image of the subject S. In this case, the eye fatigue evaluation device 10 may determine eye fatigue for each of the multiple subjects S.
[0042] In the above-described embodiment, the determination unit 12-5 calculates the right eye eye fatigue index based on the probability density distribution of the right eye aspect ratio, and calculates the left eye eye fatigue index based on the probability density distribution of the left eye aspect ratio. The determination unit 12-5 has been described as calculating the eye fatigue index by performing statistical processing based on the calculated right eye eye fatigue index and left eye eye fatigue index, but the present invention is not limited to this example. For example, the determination unit 12-5 may calculate the right eye eye fatigue index based on the probability density distribution of the right eye aspect ratio, and calculate the eye fatigue index by performing statistical processing based on the calculated right eye eye fatigue index. For example, the determination unit 12-5 may calculate the left eye eye fatigue index based on the probability density distribution of the left eye aspect ratio, and calculate the eye fatigue index by performing statistical processing based on the calculated left eye eye fatigue index.
[0043] In the above embodiment, the warning unit 12-6 acquires the eye fatigue index calculated by the determination unit 12-5, and issues a warning when the acquired eye fatigue index is less than the eye fatigue index threshold value. However, the present invention is not limited to this example. For example, the warning unit 12-6 may determine whether the acquired eye fatigue index corresponds to any of the attention levels based on a plurality of eye fatigue index threshold values. Furthermore, the warning unit 12-6 may display on the display device 2 the determination result of whether the acquired eye fatigue index corresponds to any of the attention levels.
[0044] Fig. 10 is a diagram showing an example of a display of the evaluation result of the eye fatigue evaluation system according to the present embodiment. In the example shown in Fig. 10, the judgment result of the eye fatigue index is displayed using a pie chart. Specifically, as examples of eye fatigue index thresholds, 177, 140, 124, and 82 are set. In the pie chart, the following ranges are set: a level where the eye fatigue index is 177 or more, corresponding to no caution; a caution level 1 where the eye fatigue index is 140 or more and less than 177; a caution level 2 where the eye fatigue index is 124 or more and less than 140; a caution level 3 where the eye fatigue index is 82 or more and less than 140; and a caution level 4 where the eye fatigue index is less than 82; and each range is displayed in a different color. The warning unit 12-6 displays the eye fatigue index with an arrow AR. With this configuration, the subject S can refer to the result of the eye fatigue index judgment and know the degree of his / her eye fatigue. Here, as an example, a case where the number of eye fatigue index thresholds is four is described, but the number of eye fatigue index thresholds may be two or three, or five or more.
[0045] Furthermore, the eye fatigue assessment device 10 may predict how long the state of eye fatigue will continue under the same environment based on the calculated eye fatigue index. By configuring in this way, the timing when the subject S should take a break can be derived based on the prediction result. Also, the measurement program may be automatically started on a terminal device such as a smartphone, tablet, or notebook personal computer by setting the device to automatically calculate the eye fatigue index for a user performing VDT work. The terminal device may capture an image of the user with an imaging device mounted thereon, and calculate the eye fatigue index by performing the above-mentioned processing based on image data obtained by capturing the image. The terminal device judges the eye fatigue of the user based on the eye fatigue index, and displays the judgment result based on the eye fatigue threshold of the user on the monitor of the device. By configuring in this way, the eye fatigue of the user can be constantly judged and the judgment result of the eye fatigue can be displayed, so that information regarding eye fatigue can be shown to the user.
[0046] According to the eye fatigue assessment system 100 of this embodiment, the eye fatigue assessment device 10 assesses the eye fatigue of a subject. The eye fatigue assessment device 10 includes a calculation unit 12-4 that calculates a feature amount representing an eye shape based on information identifying eye feature points acquired from an image of the subject's face, and a determination unit 12-5 that determines the eye fatigue of the subject based on the feature amount calculated by the calculation unit 12-4. With this configuration, the eye fatigue assessment device 10 can calculate features representing the shape of the eyes based on information identifying eye feature points obtained from an image of the subject's face. Therefore, the eye fatigue assessment device 10 can determine the eye fatigue of the subject based on the calculation results of the features representing the shape of the eyes.
[0047] In the early stages of eye fatigue, since there are no subjective symptoms, images are taken periodically using an imaging device implemented in a terminal device such as a smartphone, tablet, or laptop computer, and the eye fatigue evaluation device 10 calculates an eye fatigue index by carrying out the above-mentioned processing based on image data created by imaging using the imaging device. The eye fatigue evaluation device 10 judges whether or not the subject S has eye fatigue based on the calculated eye fatigue index. If the eye fatigue evaluation device 10 judges that the subject S has eye fatigue, the subject S may stop VDT work and resume work after improving the eye fatigue index. In this way, it is possible to prevent symptoms such as eye pain, decreased eyesight, stiff shoulders, and headaches in advance.
[0048] The "Guidelines for Occupational Health Management in Information Device Work" created by the Ministry of Health, Labor and Welfare states that continuous work time and work rest time should not exceed one hour, and that a 10 to 15 minute work rest period should be provided between the next continuous work. However, currently many people other than VDT workers use devices such as smartphones and tablet terminals for many hours every day. It is possible to determine whether or not eyestrain is occurring in a short time based on images captured by imaging devices mounted on smartphones, tablets, and notebook personal computers, and the frequency of such determinations can be increased, so that such people can be encouraged to prevent symptoms such as eye pain, decreased eyesight, stiff shoulders, and headaches in advance.
[0049] In addition, in the eye fatigue evaluation device 10, the calculation unit 12-4 calculates a feature amount representing the shape of the eyes based on any one of information specifying feature points corresponding to the right eye, information specifying feature points corresponding to the left eye, and information specifying feature points corresponding to both eyes. With this configuration, the calculation unit 12-4 can calculate any one of a feature amount representing the shape of the right eye, a feature amount representing the shape of the left eye, and a feature amount representing the shapes of the right eye and the left eye.
[0050] Moreover, in the eye fatigue evaluation device 10, the judgment unit 12-5 judges the eye fatigue of the subject S based on a statistic derived based on the distribution of the feature amount representing the eye shape calculated by the calculation unit 12-4 at a predetermined time. By configuring in this way, the judgment unit 12-5 can derive a statistic as an eye fatigue index based on the distribution of the feature amount representing the eye shape calculated by the calculation unit 12-4 at a predetermined time, and judge the eye fatigue of the subject S based on the derived statistic. Moreover, in the eye fatigue evaluation device 10, the judgment unit 12-5 judges the eye fatigue of the subject S from the probability density distribution of a predetermined section based on the probability density distribution of the feature amount representing the eye shape. With this configuration, it is possible to derive a statistic as an eye fatigue index from the probability density distribution of a predetermined section based on the probability density distribution of the feature amount representing the eye shape, and therefore it is possible to judge the eye fatigue of the subject S based on the derived statistic.
[0051] Moreover, in the eye fatigue evaluation device 10, the calculation unit 12-4 calculates the aspect ratio as a feature representing the eye shape based on information identifying the feature points of the eye. With this configuration, the aspect ratio can be calculated as a feature representing the eye shape, and therefore eye fatigue of the subject S can be determined based on the calculated aspect ratio. The eye fatigue evaluation device 10 further includes a warning unit 12-6 that issues a warning about eye fatigue to the subject S or the manager of the subject S when the degree of eye fatigue determined by the determination unit 12-5 exceeds a predetermined threshold. With this configuration, a warning about the eye fatigue of the subject S can be issued to the subject S or the manager of the subject S.
[0052] (Modification of the embodiment) Hereinafter, an eye fatigue evaluation system according to a modified embodiment will be described with reference to the drawings. FIG. 1 can be applied to the eye fatigue evaluation system 100a according to the modified embodiment. However, it is different in that an eye fatigue evaluation device 10a is provided instead of the eye fatigue evaluation device 10. The imaging unit 1 and the display device 2 are connected to the eye fatigue evaluation device 10a. The imaging unit 1 captures an image of a subject S. The display device 2 displays the image captured by the imaging unit 1. The eye fatigue evaluation device 10a displays an image captured by the imaging unit 1 on the display device 2. The eye fatigue evaluation device 10a displays a gaze point on the display device 2 and has the subject S gaze at the gaze point. This configuration allows the subject S to keep the line of sight constant. The eye fatigue evaluation device 10a evaluates the eye fatigue of the subject S based on the image captured by the imaging unit 1.
[0053] In one example of the eye fatigue evaluation system 100a, the imaging unit 1 is fixed on a desk. The subject S sits on a chair and gazes at the gaze point displayed on the display device 2. The imaging unit 1 creates multiple image data of the subject S by capturing an image of the face of the subject S for a predetermined time. In this embodiment, the explanation will be continued assuming that the predetermined time is 60 seconds as an example. The eye fatigue evaluation device 10a acquires multiple image data of the subject S created by the imaging unit 1. If the imaging unit 1 can capture 30 frames per second, 1800 frames can be captured in 60 seconds, so the eye fatigue evaluation device 10a acquires approximately 1800 pieces of image data. The eye fatigue evaluation device 10a records the multiple acquired image data of the subject S.
[0054] The eye fatigue evaluation device 10a calls up the recorded image data into a memory and processes the image data called up into the memory to detect a face from each of the images of the subject S. For example, the eye fatigue evaluation device 10a obtains information specifying a rectangle (quadrangle) enclosing the face of the subject S as a result of detecting the face from the image of the subject S. The eye fatigue evaluation device 10a detects facial feature points of the subject S from each of the multiple images of the subject S based on information identifying a rectangle enclosing the face of the subject S obtained from each of the multiple images of the subject S. The eye fatigue evaluation device 10a detects eye feature points of the subject S from each of the multiple images of the subject S, based on the detection results of the facial feature points of the subject S obtained from each of the multiple images of the subject S. The eye fatigue evaluation device 10a calculates features representing the shape of the eyes of the subject S for each of the multiple images of the subject S based on the detection results of the feature points of the eyes of the subject S obtained from each of the multiple images of the subject S. The eye fatigue evaluation device 10a judges the eye fatigue of the subject S based on feature amounts representing the eye shape of the subject S calculated for each of a plurality of images of the subject S.
[0055] The eye fatigue evaluation device 10a will be described in detail below. The eye fatigue evaluation device 10a includes an input / output I / F 11, a CPU 12, a memory 13, a recording device 14, a data reading unit 15, and a bus 16. The bus 16 connects the input / output I / F 11, the CPU 12, the memory 13, the recording device 14, and the data reading unit 15 to one another. The input / output I / F 11 is an interface for inputting and outputting information between the image capturing unit 1 and the display device 2, and the eye fatigue evaluation device 10a.
[0056] The CPU 12 executes a computer program (software) stored in the recording device 14 to function as a processing unit 12-1, a first detection unit 12-2, a second detection unit 12-3, a calculation unit 12a-4, a determination unit 12a-5, and a warning unit 12a-6.
[0057] The calculation unit 12a-4 acquires the array data created by the second detection unit 12-3. The calculation unit 12a-4 acquires the coordinates of each of the multiple feature points corresponding to the left eye portion and the coordinates of each of the multiple feature points corresponding to the right eye portion included in the acquired array data. The calculation unit 12a-4 calculates a feature amount representing the shape of the left eye and a feature amount representing the shape of the right eye based on the coordinates of each of the multiple feature points corresponding to the left eye portion and the coordinates of each of the multiple feature points corresponding to the right eye portion acquired. In a modified example of the embodiment, the explanation will continue with the case where the angle of the opening at the inner corner of the eye in the image of the left eye (hereinafter referred to as the "left eye inner corner opening angle") and the angle of the opening at the outer corner of the eye (hereinafter referred to as the "left eye outer corner opening angle") are used as an example of a feature representing the shape of the left eye, and the angle of the opening at the inner corner of the eye in the image of the right eye (hereinafter referred to as the "right eye inner corner opening angle") and the angle of the opening at the outer corner of the eye (hereinafter referred to as the "right eye outer corner opening angle") are used as an example of a feature representing the shape of the right eye.
[0058] 11 is a diagram showing an example of the process of the eye fatigue assessment device according to the modified example of the embodiment, which shows, as an example, characteristic points P1 to P6 corresponding to the right eye. As shown in Fig. 11, an example of the feature point P1 (corresponding to FP37 in Fig. 2) corresponds to the outer corner of the right eye, an example of the feature point P2 (corresponding to FP38 in Fig. 2) corresponds to the intersection of the right boundary between the white and black of the eye and the upper eyelid in the right eye, and an example of the feature point P3 (corresponding to FP39 in Fig. 2) corresponds to the intersection of the left boundary between the white and black of the eye and the upper eyelid in the right eye. As shown in Fig. 11, an example of the feature point P4 (corresponding to FP40 in Fig. 2) corresponds to the inner corner of the right eye, an example of the feature point P5 (corresponding to FP41 in Fig. 2) corresponds to the intersection of the left boundary between the white and black of the eye and the lower eyelid in the right eye, and an example of the feature point P6 (corresponding to FP42 in Fig. 2) corresponds to the intersection of the right boundary between the white and black of the eye and the lower eyelid in the right eye.
[0059] The calculation unit 12a-4 calculates the right eye inner corner opening angle and the right eye outer corner opening angle based on the coordinates of each of the characteristic points P1 to P6 corresponding to the right eye. Specifically, the calculation unit 12a-4 calculates the right eye outer corner opening angle (the angle between the line connecting the feature points P1 and P2 and the line connecting the feature points P1 and P6) using the law of cosines for a triangle formed by connecting the feature points P1, P2, and P6 with lines. The calculation unit 12a-4 calculates the right eye inner corner opening angle (the angle between the line connecting the feature points P4 and P3 and the line connecting the feature points P4 and P5) using the law of cosines for a triangle formed by connecting the feature points P3, P4, and P5 with lines. The calculation unit 12a-4 calculates the right eye opening angle, which is the opening angle of both the right eye inner corner and the right eye outer corner, by performing statistical processing based on the calculated right eye inner corner opening angle and right eye outer corner opening angle. In this embodiment, the explanation will continue with the case where an average value is used as an example of statistical processing.
[0060] The calculation unit 12a-4 calculates the left eye inner corner opening angle and the left eye outer corner opening angle based on the coordinates of each of the six feature points corresponding to the left eye. Details of the process in which the calculation unit 12a-4 calculates the left eye inner corner opening angle and the left eye outer corner opening angle are omitted here because the process of calculating the right eye inner corner opening angle and the right eye outer corner opening angle described above can be applied. The calculation unit 12a-4 calculates the left opening angle, which is the opening angle of both the left eye inner corner and the left eye outer corner, by performing statistical processing based on the calculated left eye inner corner opening angle and left eye outer corner opening angle. In this embodiment, the explanation will continue with the case where the average value is used as an example of statistical processing. The calculation unit 12a-4 estimates the movement (orientation) of the subject S's head along each of the X-axis, Y-axis, and Z-axis using the coordinates of predetermined feature points among the feature points FP01 to FP68 of the human face in order to correct errors that occur due to differences in the positional relationship between the subject S and the imaging unit 1 in each of a plurality of images captured of the subject S. The calculation unit 12a-4 records the results of error correction performed on the calculated right opening angle and left opening angle in the recording device 14 based on the estimated results of the movement of the subject S's head along each of the X-axis, Y-axis, and Z-axis.
[0061] Fig. 12 is a diagram showing Example 1 of statistical data of the right opening angle calculated by the eye fatigue evaluation device according to the modified example of the embodiment. Fig. 12 shows, as an example, a probability density distribution of the right opening angle acquired every hour from before the start of VDT work until four hours have passed since the start of VDT work. In Fig. 12, the horizontal axis shows rad, and the vertical axis shows the probability density at that time. To obtain statistical data on the right opening angle, subject S was imaged for 60 seconds. Of the image data obtained during the 60 seconds of imaging, 1000 points of data from 20 seconds after the start of imaging, when eye and head movements are assumed to be relatively stable, were used to calculate a probability density function.
[0062] The determination unit 12a-5 acquires information for identifying each of the right opening angles calculated by the calculation unit 12a-4. The determination unit 12a-5 calculates an index indicating eye fatigue of the right eye (hereinafter referred to as a "right eye eye fatigue index") based on the acquired information for identifying each of the right opening angles. Specifically, the determination unit 12a-5 calculates the ratio of the peak to the range of the right opening angle corresponding to a predetermined ratio in the probability density distribution of the right opening angle. In this embodiment, the description will continue with a case where 95% (probability distribution is from 2.5% to 97.5%) is applied as an example of the predetermined ratio.
[0063] When 95% is applied as the specified ratio, the judgment unit 12a-5 determines the range of the right opening angle where the probability distribution is 2.5% to 97.5% as shown in equation (4), the range of 2.5% to 97.5% of the maximum value in the probability density function of the right opening angle as xra, and the maximum value (peak) of the probability density function of the right opening angle of the right eye as yra, and determines the ratio of xra to yra as an index of right eye fatigue, as shown in equation (4). Specifically, before starting VDT work, the peak of the probability density distribution of the right opening angle is 7.1951, and the range of the right opening angle corresponding to 95% is 0.29, so the judgment unit 12-5 calculates the right eye fatigue index to be 24.8. Right eye eye fatigue index = (the value at which the probability density function of the right eye's right opening angle is maximized (yra value)) / (the range of 2.5% or more in the probability distribution of the right opening angle (xra value)) (4) FIG. 12 also shows the change over time in the right eye fatigue index calculated based on the probability density distribution of the right opening angle obtained every hour.
[0064] According to Fig. 12, it can be seen that the maximum value (peak) (yra value) of the probability density function at the right opening angle is stable and takes a large value at a certain right opening angle before the start of VDT work (when eye fatigue is small) compared to after the start of VDT work. In other words, it can be seen that as the time of VDT work progresses, the maximum value of the probability density function at the right opening angle becomes lower, resulting in a broad probability density distribution (probability density at the mode decreases). In other words, the degree of fatigue of the right eye can be known based on the maximum value of the probability density function at the right opening angle.
[0065] Fig. 13 is a diagram showing Example 2 of statistical data of the left opening angle calculated by the eye fatigue evaluation device according to the modified example of the embodiment. Fig. 13 shows, as an example, a probability density distribution of the left opening angle acquired every hour from before the start of VDT work until four hours have passed since the start of VDT work. In Fig. 13, the horizontal axis represents rad, and the vertical axis represents the probability density at that time. To obtain statistical data on the left opening angle, subject S was imaged for 60 seconds. Of the image data obtained during the 60 seconds of imaging, 1000 points of data from 20 seconds after the start of imaging, when eye and head movements are assumed to be relatively stable, were used to calculate a probability density function.
[0066] The determination unit 12a-5 acquires information for identifying each of the left opening angles calculated by the calculation unit 12a-4. The determination unit 12a-5 calculates an index indicating eye fatigue of the left eye (hereinafter referred to as a "left eye eye fatigue index") based on the acquired information for identifying each of the left opening angles. Specifically, the determination unit 12a-5 calculates the ratio of the peak to the range of the left opening angle corresponding to a predetermined ratio in the probability density distribution of the left opening angle. In this embodiment, the description will continue with a case where 95% (probability distribution is from 2.5% to 97.5%) is applied as an example of the predetermined ratio.
[0067] When 95% is applied as the specified ratio, the judgment unit 12a-5 determines the range of the left opening angle where the probability distribution is 2.5% to 97.5% as shown in equation (5), the range of 2.5% to 97.5% of the maximum value in the probability density function of the left opening angle as xla, and the maximum value (peak) of the probability density function of the left opening angle of the left eye as yla, and determines the ratio of xla to yla as an index of left eye fatigue, as shown in equation (5). Specifically, before starting VDT work, the judgment unit 12a-5 calculates the left eye fatigue index to be 22.5 because the peak of the probability density distribution of the left opening angle is 6.9738 and the range of the left opening angle corresponding to 95% is 0.31. Left eye fatigue index = (the value at which the probability density function of the left eye's left opening angle is maximized (YLA value)) / (the range of 2.5% or more in the probability distribution of the left opening angle (XLA value)) (5) FIG. 13 also shows the time change in the left eye fatigue index calculated based on the probability density distribution of the left opening angle obtained every hour.
[0068] According to Fig. 13, it can be seen that the maximum value (peak) (yla value) of the probability density function at the left opening angle is stable and large at a certain left opening angle before the start of VDT work (when eye fatigue is small) compared to after the start of VDT work. In other words, it can be seen that as the time of VDT work progresses, the maximum value of the probability density function at the left opening angle becomes lower, resulting in a broader probability density distribution (probability density at the mode decreases). In other words, the degree of fatigue of the left eye can be known based on the maximum value of the probability density function at the left opening angle.
[0069] FIG. 14 is a diagram showing an example of intervals, probability densities, and eye fatigue indexes of opening angles calculated by the eye fatigue evaluation device according to the modified example of the embodiment. 14, the right eye fatigue index is 24.83 before the start of VDT work, 16.22 one hour after the start of VDT work, 13.38 two hours after the start of VDT work, 11.18 three hours after the start of VDT work, and 9.38 four hours after the start of VDT work. It can be seen that the right eye fatigue index decreases exponentially according to the time that has elapsed since the start of VDT work.
[0070] The left eye fatigue index was 22.58 before the start of VDT work, 16.94 one hour after the start of VDT work, 11.74 two hours after the start of VDT work, 8.35 three hours after the start of VDT work, and 7.95 four hours after the start of VDT work. It can be seen that the left eye fatigue index decreases according to the time that has elapsed since the start of VDT work.
[0071] The decrease in the right eye fatigue index and the left eye fatigue index is assumed to be due to eye fatigue caused by long-term use of VDT devices such as smartphones and personal computers. When concentrating on staring at a screen and the eyeballs are not moving for a long time, the seven muscles that move the eyeballs and the muscles that adjust the thickness of the "crystalline lens" to focus become fatigued. In addition, the number of times the eyes blink is often reduced during VDT work. Furthermore, the decrease in the right eye fatigue index and the left eye fatigue index is also thought to be due to dry eyes, which make tears evaporate easily when wearing contact lenses or in an environment exposed to air conditioning. Due to these causes, the probability of the opening angle becoming smaller increases, and the xra value in the denominator of the right eye fatigue index and the xla value in the denominator of the left eye fatigue index, which means an increase in eye fatigue, decrease the right eye fatigue index and the left eye fatigue index, respectively.
[0072] The determination unit 12a-5 also performs statistical processing based on the right eye fatigue index and the left eye fatigue index to calculate an eye fatigue index that is an index of eye fatigue of both the right eye and the left eye combined. In this embodiment, the explanation will continue with the case where an average value is used as an example of statistical processing. 14, the fatigue index (average of left and right eyes) is 23.7 before the start of VDT work, 16.58 one hour after the start of VDT work, 12.56 two hours after the start of VDT work, 9.76 three hours after the start of VDT work, and 8.67 four hours after the start of VDT work. It can be seen that the fatigue index decreases with the time that has elapsed since the start of VDT work, just like the right eye fatigue index and left eye fatigue index.
[0073] 15 is a diagram showing an example of a change over time in the eye fatigue index calculated by the eye fatigue evaluation device according to the modified example of the embodiment, in which the horizontal axis represents the elapsed time from the start of the VDT work, and the vertical axis represents the eye fatigue index. According to FIG. 15, it can be seen that the eye fatigue index decreases by about 25% one hour after the start of the VDT work. Therefore, when the eye fatigue index decreases by 25% or more from the initial value, it is preferable that the eye fatigue evaluation device 10a has the subject S stop the VDT work and take a rest for a certain period of time. Before resuming the work, the eye fatigue evaluation device 10a calculates the right eye fatigue index and the left eye fatigue index of the subject S again, and judges whether the calculated fatigue index has decreased by 25% or more based on the calculated right eye fatigue index and the left eye fatigue index. The eye fatigue evaluation device 10a performs the same process when the fatigue index has decreased by 25% or more based on the calculated fatigue index thereafter. It is assumed that by configuring in this way, the subject S can prevent symptoms such as eye pain, decreased eyesight, stiff shoulders, and headaches.
[0074] The warning unit 12a-6 acquires the eye fatigue index calculated by the determination unit 12a-5. The warning unit 12a-6 judges whether the acquired eye fatigue index is less than the eye fatigue index threshold. An example of the eye fatigue index threshold is a value 25% lower than the initial value of the eye fatigue index. The warning unit 12a-6 issues a warning when the eye fatigue index is less than the eye fatigue index threshold. Specifically, the warning unit 12a-6 issues a warning to the subject S or the administrator of the subject S.
[0075] (Operation of eye fatigue evaluation system 100a) FIG. 16 is a flowchart showing an example of the operation of the eye fatigue evaluation system according to the modified example of the embodiment. Steps S1-2 to S5-2 can be implemented by steps S1-1 to S5-1 in FIG. 9, so a description thereof will be omitted here. (S6-2) In the eye fatigue evaluation device 10a, the calculation unit 12a-4 estimates the movement (direction) of the head of the subject S with respect to each of the X-axis, Y-axis, and Z-axis by using the coordinates of predetermined feature points among feature points FP01 to FP68 on a human face. (S7-2) In the eye fatigue evaluation device 10a, the calculation unit 12a-4 calculates the right eye inner corner opening angle and the right eye outer corner opening angle based on the coordinates of each of the characteristic points P1 to P6 corresponding to the right eye.
[0076] (S8-2) In the eye fatigue evaluation device 10a, the calculation unit 12a-4 estimates the movement (direction) of the head of the subject S with respect to each of the X-axis, Y-axis, and Z-axis by using the coordinates of predetermined feature points among the feature points FP01 to FP68 of the human face. The calculation unit 12a-4 records in the recording device 14 the results of performing error correction on the calculated right opening angle and left opening angle based on the estimation results of the movement of the head of the subject S with respect to each of the X-axis, Y-axis, and Z-axis. (S9-2) In the eye fatigue evaluation device 10a, the determination unit 12a-5 calculates a right eye eye fatigue index based on the probability density distribution of the right eye inner corner opening angle. The calculation unit 12a-4 calculates a left eye eye fatigue index based on the probability density distribution of the left eye inner corner opening angle. The determination unit 12a-5 calculates the eye fatigue index by performing statistical processing based on the calculated right eye eye fatigue index and left eye eye fatigue index. (S10-2) In the eye fatigue evaluation device 10a, the judgment unit 12a-5 judges the degree of eye fatigue on the basis of the calculated eye fatigue index. 16, the order of steps S6-2 and S7-2 may be interchanged, i.e., steps S1-2 to S5-2, S7-2, S6-2, S8-2 to S10-2 may be executed in this order.
[0077] In the modified example of the embodiment described above, the determination unit 12a-5 calculates the right eye eye fatigue index based on the probability density distribution of the right eye opening angle, and calculates the left eye eye fatigue index based on the probability density distribution of the left eye opening angle. The determination unit 12a-5 has been described as calculating the eye fatigue index by performing statistical processing based on the calculated right eye eye fatigue index and left eye eye fatigue index, but this is not limited to the example. For example, the determination unit 12a-5 may calculate the right eye eye fatigue index based on the probability density distribution of the right eye opening angle, and calculate the eye fatigue index by performing statistical processing based on the calculated right eye eye fatigue index. For example, the determination unit 12a-5 may calculate the left eye eye fatigue index based on the probability density distribution of the left eye opening angle, and calculate the eye fatigue index by performing statistical processing based on the calculated left eye eye fatigue index.
[0078] In the modified example of the embodiment described above, the calculation unit 12a-4 calculates the right eye outer corner opening angle and the right eye inner corner opening angle, and calculates the right opening angle based on the calculated right eye outer corner opening angle and right eye inner corner opening angle, but this is not limited to the example. For example, the calculation unit 12a-4 may calculate the right eye outer corner opening angle and calculate the right opening angle based on the calculated right eye outer corner opening angle, or may calculate the right eye inner corner opening angle and calculate the right opening angle based on the calculated right eye inner corner opening angle. In the modified example of the embodiment described above, the calculation unit 12a-4 calculates the left eye outer corner opening angle and the left eye inner corner opening angle, and calculates the left opening angle based on the calculated left eye outer corner opening angle and the left eye inner corner opening angle, but this is not limited to the example. For example, the calculation unit 12a-4 may calculate the left eye outer corner opening angle and calculate the left opening angle based on the calculated left eye outer corner opening angle, or may calculate the left eye inner corner opening angle and calculate the left opening angle based on the calculated left eye inner corner opening angle.
[0079] In the modification of the embodiment described above, the warning unit 12a-6 acquires the eye fatigue index calculated by the determination unit 12a-5, and issues a warning when the acquired eye fatigue index is less than the eye fatigue index threshold value. However, the present invention is not limited to this example. For example, the warning unit 12a-6 may determine whether the acquired eye fatigue index corresponds to any of the attention levels based on a plurality of eye fatigue index threshold values. Furthermore, the warning unit 12a-6 may display the determination result of whether the acquired eye fatigue index corresponds to any of the attention levels on the display device 2.
[0080] FIG. 17 is a diagram showing an example of the display of the evaluation result of the eye fatigue evaluation system according to the modified embodiment. In the example shown in FIG. 17, the judgment result of the eye fatigue index is displayed using a bar chart. Specifically, 16.5, 12.6, 9.8, and 8.7 are set as examples of eye fatigue index thresholds. In the bar chart, the following ranges are set: a level corresponding to no attention where the eye fatigue index is 16.5 or more, a caution level 1 where the eye fatigue index is 12.6 or more and less than 16.5, a caution level 2 where the eye fatigue index is 9.8 or more and less than 12.6, a caution level 3 where the eye fatigue index is 8.7 or more and less than 9.8, and a caution level 4 where the eye fatigue index is less than 8.7, and each range is displayed in a different color. The warning unit 12a-6 displays the eye fatigue index with a character. By configuring in this way, the subject S who refers to the judgment result of the eye fatigue index can know the degree of his or her eye fatigue state. Here, as an example, a case where the number of eye fatigue index thresholds is four has been described, but the number of eye fatigue index thresholds may be 2-3, or may be 5 or more.
[0081] According to the eye fatigue evaluation system 100a according to the modified embodiment, in the eye fatigue evaluation device 10 according to the above-mentioned embodiment, the calculation unit 12a-4 calculates either or both of the angle of the inner corner of the eye and the angle of the outer corner of the eye as a feature representing the shape of the eye based on information specifying the feature points of the eye. By configuring in this way, either or both of the angle of the inner corner of the eye and the angle of the outer corner of the eye can be calculated as a feature representing the shape of the eye, so that the eye fatigue of the subject S can be determined based on either or both of the calculated angle of the inner corner of the eye and the angle of the outer corner of the eye.
[0082] Although the embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs within the scope of the gist of the present invention are also included. [Explanation of symbols]
[0083] 1...imaging unit, 2...display device, 10...eye fatigue evaluation device, 11...input / output I / F, 12...CPU, 12-1...processing unit, 12-2...first detection unit, 12-3...second detection unit, 12-4, 12a-4...calculation unit, 12-5, 12a-5...determination unit, 12-6, 12a-6...warning unit, 13...memory, 14...recording device, 15...data reading unit
Claims
1. An eye fatigue evaluation device for evaluating eye fatigue of a subject, comprising: a calculation unit that calculates a feature amount representing an eye shape based on information identifying eye feature points obtained from an image of the face of the subject; a determination unit that determines eye fatigue of the subject based on the feature amount calculated by the calculation unit; Equipped with The determination unit is an eye fatigue evaluation device that determines eye fatigue of the subject based on a statistical quantity derived based on the distribution of features representing the eye shape calculated by the calculation unit at a specified time.
2. 2. The eye fatigue assessment device of claim 1, wherein the calculation unit calculates a feature quantity representing an eye shape based on any one of information specifying feature points corresponding to a right eye, information specifying feature points corresponding to a left eye, and information specifying feature points corresponding to both eyes.
3. The eye fatigue evaluation device according to claim 1 , wherein the determining unit determines eye fatigue of the subject from a probability density distribution of a predetermined section based on a probability density distribution of a feature amount representing a shape of the eye.
4. The eye fatigue evaluation device according to claim 1 , wherein the calculation unit calculates an aspect ratio as a feature quantity representing a shape of the eye based on the information that identifies feature points of the eye.
5. The eye fatigue evaluation device according to claim 1 , wherein the calculation unit calculates either or both of an angle of an opening at the inner corner of the eye and an angle of an opening at the outer corner of the eye as features representing an eye shape based on the information identifying feature points of the eye.
6. a warning unit that issues a warning regarding eye fatigue to the subject or a manager of the subject when the degree of eye fatigue determined by the determination unit exceeds a predetermined threshold value. The eye fatigue evaluation device according to claim 1 , further comprising:
7. A computer-implemented eye fatigue assessment method, comprising: calculating a feature quantity representing an eye shape based on information identifying eye feature points obtained from an image of the subject's face; determining eye fatigue of the subject based on the feature amount calculated in the calculating step; having In the determining step, the eye fatigue of the subject is determined based on a statistical quantity derived based on the distribution of features representing the eye shape calculated in the calculating step at a predetermined time.
8. On the computer, calculating a feature quantity representing an eye shape based on information identifying eye feature points obtained from an image of the subject's face; determining eye fatigue of the subject based on the feature amount calculated in the calculating step; Run the command, In the determining step, the program determines eye fatigue of the subject based on a statistical quantity derived based on the distribution of features representing the eye shape calculated in the calculating step at a predetermined time.
Citation Information
Patent Citations
Head-mounted display
JP2003076353A
Mobile terminal, method and program of measuring eye strain, and support system
JP2010162124A
Blinking measuring apparatus and blinking measuring method
JP2010273954A
Image analyzer
JP2012084068A
Eye fatigue prediction based on calculated blood vessel density score
US20190038129A1
