Driver's condition estimation device

Abstract

To provide a driver's condition estimation device capable of estimating the possibility of a dry eye of a driver.SOLUTION: A driver's condition estimation device 1 comprises: an acquisition unit 11 which acquires an opening degree of the eyes of a vehicle driver; a detection unit 12 which detects a winking pattern of the driver based on the progress of the opening degree of the eyes; and an estimation unit 13 which estimates the possibility of a dry eye of the driver based on the winking pattern. The estimation unit estimates the possibility of the dry eye of the driver based on a winking pattern detected while the driver drives the vehicle and a winking pattern detected while the driver does not drive the vehicle.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a driver state estimating device. [Background technology]

[0002] Conventionally, there are devices for monitoring drivers. Patent Document 1 discloses a driver monitoring device that includes a dozing determination unit that detects whether the driver's eyes are open or closed based on a facial image showing the face of the driver of a vehicle, and determines whether the driver is dozing or not based on the eye opening or closing state, and a conversation determination unit that determines whether the driver is talking or not. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-149287 A Summary of the Invention [Problem to be solved by the invention]

[0004] It is desirable to be able to estimate the possibility of a driver having dry eye in a vehicle. For example, encouraging the driver to take a break when it is estimated that the driver has dry eyes can contribute to safe driving.

[0005] An object of the present invention is to provide a driver condition estimating device that can estimate the possibility of dry eye in a driver. [Means for solving the problem]

[0006] The driver state estimation device of the present invention comprises an acquisition unit that acquires the degree of eye opening of a driver of a vehicle, a detection unit that detects the blinking pattern of the driver based on a change in the degree of eye opening, and an estimation unit that estimates the possibility of dry eye in the driver based on the blinking pattern, wherein the estimation unit estimates the possibility of dry eye in the driver based on the blinking pattern detected when the driver is driving the vehicle and the blinking pattern detected when the driver is not driving the vehicle. Effect of the Invention

[0007] The estimation unit of the driver state estimation device according to the present invention estimates the possibility of dry eye in the driver based on a blinking pattern detected when the driver is driving a vehicle and a blinking pattern detected when the driver is not driving a vehicle. The driver state estimation device according to the present invention has the effect of being able to estimate the possibility of dry eye in the driver. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram of a vehicle equipped with a driver state estimating device according to an embodiment. [Diagram 2] FIG. 2 is a block diagram of a driver state estimating device according to the embodiment. [Diagram 3] FIG. 3 is a diagram showing a first pattern of blinking. [Figure 4] FIG. 4 is a diagram showing a second pattern of blinking. [Diagram 5] FIG. 5 is a diagram showing a third pattern of blinking. [Figure 6] FIG. 6 is a diagram showing a fourth pattern of blinking. [Figure 7] FIG. 7 is a diagram showing an example of blink data of a person without dry eye. [Figure 8] FIG. 8 is a diagram showing an example of blink data of a person with dry eye. [Figure 9]FIG. 9 is a diagram showing blink data of people without dry eyes and the occurrence rate of each pattern. [Figure 10] FIG. 10 is a diagram showing blink data of people prone to dry eye and the occurrence rate of each pattern. [Figure 11] FIG. 11 is a diagram showing blink data of people with dry eye and the occurrence rate of each pattern. [Figure 12] FIG. 12 is a flowchart related to the driver state estimating device of the embodiment. [Figure 13] FIG. 13 is a flowchart showing a process for setting the flag during driving. [Figure 14] FIG. 14 is a list of flag setting values. [Figure 15] FIG. 15 is a flowchart showing a process for setting the flag when the vehicle is not driving. [Figure 16] FIG. 16 is a list of determinations according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] A driver state estimating device according to an embodiment of the present invention will be described in detail below with reference to the drawings. Note that the present invention is not limited to the embodiment. In addition, the components in the following embodiment include those that a person skilled in the art can easily imagine or that are substantially the same.

[0010] [Embodiment] An embodiment will be described with reference to Fig. 1 to Fig. 16. The embodiment relates to a driver state estimation device. Fig. 1 is a diagram of a vehicle equipped with a driver state estimation device of the embodiment, Fig. 2 is a block diagram of the driver state estimation device of the embodiment, Fig. 3 is a diagram showing a first pattern of blinking, Fig. 4 is a diagram showing a second pattern of blinking, Fig. 5 is a diagram showing a third pattern of blinking, Fig. 6 is a diagram showing a fourth pattern of blinking, Fig. 7 is a diagram showing an example of blink data of a person without dry eye, Fig. 8 is a diagram showing an example of blink data of a person with dry eye, Fig. 9 is a diagram showing blink data of a person without dry eye and the occurrence ratio of each pattern, Fig. 10 is a diagram showing blink data of a person prone to dry eye and the occurrence ratio of each pattern, Fig. 11 is a diagram showing blink data of a person with dry eye and the occurrence ratio of each pattern, and Fig. 12 is a flowchart related to the driver state estimation device of the embodiment.

[0011] FIG. 13 is a flowchart for setting flags when driving, FIG. 14 is a list of flag setting values, FIG. 15 is a flowchart for setting flags when not driving, and FIG. 16 is a list of judgments according to the embodiment.

[0012] As shown in Fig. 1, a driver state estimation device 1 of this embodiment is mounted on a vehicle 100. The vehicle 100 is an automobile, and is driven by a driver 200. The vehicle 100 has a camera 20 that captures an image of the face of the driver 200. The camera 20 is disposed in front of the driver 200, and is disposed so as to capture an image of the eyes 210 of the driver 200. The camera 20 is configured to continuously capture images of the driver 200 at a predetermined frame rate.

[0013] The driver state estimation device 1 is a device that estimates the possibility of dry eye in a driver 200. As shown in Fig. 2, the driver state estimation device 1 has an acquisition unit 11, a detection unit 12, an estimation unit 13, and a warning unit 14. The driver state estimation device 1 is, for example, a computer having an arithmetic circuit, a memory, an input / output interface, etc. The driver state estimation device 1 is configured as, for example, an electronic control unit (ECU).

[0014] The acquisition unit 11 acquires the eye opening degree of the driver 200. The acquisition unit 11 of this embodiment acquires the eye opening degree of the driver 200 based on the image acquired from the camera 20. The acquisition unit 11 calculates the eye opening degree EO [%] of the driver 200 in each image, for example, by image recognition. When the eyes 210 of the driver 200 are completely closed, the value of the eye opening degree EO is 0%. The acquisition unit 11 may calculate the eye opening degree EO for each of the left eye and the right eye of the driver 200. The eye opening degree EO may be an average value of the eye opening degree of the left eye and the eye opening degree of the right eye of the driver 200. The acquisition unit 11 may calculate only the eye opening degree EO of either one of the eyes 210 of the driver 200. The acquisition unit 11 calculates the eye opening degree EO for each of the images acquired continuously from the camera 20.

[0015] The detection unit 12 detects the blinking pattern of the driver 200 based on the transition of the eye opening degree EO. The detection unit 12 of this embodiment determines which of the four patterns described below the blink corresponds to. The four patterns used in this embodiment are the first pattern shown in FIG. 3 to the fourth pattern shown in FIG. 6.

[0016] (a) First pattern The first pattern shown in Fig. 3 is a blinking pattern in which the eyes are not completely closed. In the graphs of Fig. 3 and the like, the horizontal axis is time [msec] and the vertical axis is eye opening degree EO [%]. The graph plots the transition of the value of eye opening degree EO acquired by acquisition unit 11. In the blink of Fig. 3, the minimum value of eye opening degree EO is about 40 [%], and the eyelids are opened before the eyes are fully closed. Such a pattern occurs, for example, when the eyes 210 of driver 200 are slightly dry.

[0017] (b) Second pattern The second pattern shown in Fig. 4 is a blinking pattern in which the eyes are repeatedly opened and closed without being completely opened. In the blinking of Fig. 4, after the blinking starts, the eyes are closed again before the eye opening degree EO returns to 100[%]. In this embodiment, one blink is an eye opening and closing action from the start of the blinking to the eye being completely opened. In other words, one blink may include multiple eye opening and closing actions. The second pattern is one of the typical blinking patterns of people with dry eye.

[0018] (c) Third pattern The third pattern shown in FIG. 5 is a blinking pattern in which the eyes are closed for a long time. Here, a long time means a long time compared to the average blinking pattern of people who do not have dry eye. In the blink of FIG. 5, the eyes remain closed for a while before they begin to open. In other words, the eyes are closed for a long time compared to normal blinking. The third pattern is one of the typical blinking patterns of people with dry eye.

[0019] (d) Fourth pattern The fourth pattern shown in FIG. 6 is a typical blink pattern of a person without dry eye. In other words, the fourth pattern is a standard blink pattern or a normal blink pattern. In the blink shown in FIG. 6, the blink is completed with one opening and closing of the eyes. In addition, the time from when the eyes are fully closed to when they are opened, and from when the eyes are closed to when they start to open, is short. Furthermore, the time from when the blink starts to when the eyes are completely closed, and the time from when the eyes start to open to when the eyes are completely opened, are both short.

[0020] The detection unit 12 extracts one blink from the data of the transition of the eye opening degree EO. The data extracted as one blink is, for example, the value of the eye opening degree EO from the time when it is determined that the blink has started to the time when it is determined that the blink has ended. The detection unit 12 determines which of the four blink patterns the extracted one blink corresponds to. The detection unit 12 selects the pattern with the highest degree of match, for example, by pattern matching.

[0021] FIG. 7 is an example of blink data of a person without dry eye. FIG. 8 is an example of blink data of a person with dry eye. In the case of a person without dry eye, as shown in FIG. 7, blinking of a fourth pattern is repeated at a substantially regular cycle. In the case of a person with dry eye, as shown in FIG. 8, the interval between blinks is shorter than that of a person without dry eye. In addition, the blinking of a person with dry eye includes a plurality of patterns. In the data of FIG. 8, (a) a first pattern, (b) a second pattern, and (c) a third pattern appear.

[0022] The estimation unit 13 calculates the occurrence ratio of a plurality of patterns in a predetermined period of time. The predetermined period of time may be, for example, a period shorter than one minute, about one minute, several minutes, or any other length of time.

[0023] The estimation unit 13 calculates the number of blinks in a first pattern B1, the number of blinks in a second pattern B2, the number of blinks in a third pattern B3, and the number of blinks in a fourth pattern B4 in a predetermined period. The occurrence ratio Ri of the blinks in the i-th pattern (i=1, 2, 3, 4) is calculated by the following formula (1). Ri = Bi / (B1 + B2 + B3 + B4) (1)

[0024] 9 shows blink data of a person without dry eye and the occurrence rate Ri [%] of each pattern calculated from this data. For a person without dry eye, the occurrence rate R4 of the fourth pattern is high, and the occurrence rates R1, R2, and R3 of the first to third patterns are all low.

[0025] Fig. 10 shows blink data of a person prone to dry eye, and the occurrence rate Ri [%] of each pattern calculated from this data. The data in Fig. 10 is data on the eye opening degree EO of the driver 200 while driving. When the workload of the eyes 210 is high, such as when driving the vehicle 100, the number of blinks is reduced compared to when the workload is low. For people with dry eye or prone to dry eye, the occurrence rate R1 of the first pattern of blinks increases as the number of blinks decreases.

[0026] Figure 11 shows blink data of a person with dry eye, and the occurrence rate Ri [%] of each pattern calculated from this data. The data in Figure 11 is data on the eye opening degree EO of driver 200 while driving. For a person with dry eye, the occurrence rates R1, R2, and R3 of the first to third patterns are large. In the data in Figure 11, the occurrence rate R4 of the fourth pattern is the smallest.

[0027] As described above, the three blinking patterns from the first pattern to the third pattern are abnormal blinking patterns that indicate dry eyes and are blinking patterns that indicate the possibility of dry eye. Among the abnormal blinking patterns, the blinking of the first pattern indicates a tendency for dry eye. The blinking of the second and third patterns indicates the possibility of dry eye.

[0028] 2, the warning unit 14 controls a warning device 40. The warning device 40 is a device that outputs a caution or a warning to the driver 200. The warning device 40 is, for example, a warning lamp, a speaker, an image display device, etc. provided in the vehicle 100.

[0029] The driver state estimating device 1 acquires various information from the vehicle ECU 30. The acquired information includes, for example, the traveling speed of the vehicle 100, the shift position, the operating state of the brake hold function, and whether or not the steering is held.

[0030] The flowchart shown in Fig. 12 is executed when the power of the vehicle 100 is ON. This flowchart is executed regardless of whether the vehicle 100 is moving or not. In step S310, the eye opening degree EO is detected. The acquisition unit 11 acquires the eye opening degree of the driver 200 based on the image acquired from the camera 20. After step S310 is executed, the process proceeds to step S320.

[0031] In step S320, blink analysis is performed. The detection unit 12 detects a blink pattern based on the data of the eye opening degree EO acquired in step S310. The detection unit 12 determines which of the first to fourth patterns each detected blink corresponds to. After step S320 is executed, the process proceeds to step S330.

[0032] In step S330, it is determined whether the vehicle 100 is moving. The determination in step S330 is made based on, for example, the traveling speed of the vehicle 100. If the determination in step S330 is affirmative, the process proceeds to step S340, and if the determination is negative, the process proceeds to step S350.

[0033] In step S340, the driving flag FG2 is set. The driver state estimation device 1 of this embodiment performs a comprehensive determination of the possibility of dry eye based on the non-driving flag FG1 and the driving flag FG2. Setting of the driving flag FG2 will be described with reference to Fig. 13. The process of step S340 includes steps S341 to S345.

[0034] In step S341, the blink pattern is determined. The estimation unit 13 performs the determination in step S341 based on the appearance ratio Ri (i=1, 2, 3, 4) of each pattern acquired by the blink analysis. The estimation unit 13 of this embodiment classifies the blink patterns as shown in FIG.

[0035] In FIG. 14, the column for while driving (FG2) shows classification of blink data while driving. The value of the flag FG2 while driving is 0, 1, 2, or 3. A value of 0 indicates that it is impossible to determine. The initial value of the flag FG2 while driving is 0. A value of 1 indicates that there is no dry eye. A value of 2 indicates that there is a possibility of dry eye. A value of 3 indicates that there is a high possibility of dry eye.

[0036] If only shallow blinks of the first pattern are frequently observed while driving, it is determined that there is a tendency for dry eye and the value of the flag FG2 during driving is set to 2. If blinks of the second pattern and / or the third pattern are observed in addition to shallow blinks of the first pattern while driving, it is determined that there is a high possibility of dry eye and the value of the flag FG2 during driving is set to 3. If normal blinks of the fourth pattern are mostly observed while driving, it is determined that there is no dry eye and the value of the flag FG2 during driving is set to 1. Note that if a clear determination cannot be made, the value of the flag FG2 during driving is set to 0.

[0037] If various blinking patterns are observed as shown in Fig. 11, the process proceeds to step S342, where the value of the flag FG2 for when driving is set to 3. If shallow eye closing patterns are frequently observed as shown in Fig. 10, the process proceeds to step S343, where the value of the flag FG2 for when driving is set to 2. If most blinks are of the fourth pattern as shown in Fig. 9, the process proceeds to step S344, where the value of the flag FG2 for when driving is set to 1. If a clear determination cannot be made, the process proceeds to step S345, where the value of the flag FG2 for when driving is set to 0.

[0038] In step S350 in Fig. 12, a non-driving flag FG1 is set. As shown in Fig. 14, the value of the non-driving flag FG1 is any one of 0, 1, 2, and 3. A value of 0 indicates that a determination cannot be made. The initial value of the non-driving flag FG1 is 0. A value of 1 indicates that there is no dry eye. A value of 2 indicates that there is a possibility of dry eye. A value of 3 indicates that there is a high possibility of dry eye.

[0039] When only shallow blinks of the first pattern are frequently observed while not driving, the value of the non-driving flag FG1 is set to 2 as a tendency to dry eye. Shallow blinks may occur while not driving because the driver 200 is still tense due to driving, or the driver 200 is operating an electronic device. When various blink patterns are observed while not driving, the value of the non-driving flag FG1 is set to 3 as a high possibility of dry eye. When most of the blinks are normal blinks of the fourth pattern while not driving, the value of the non-driving flag FG1 is set to 1 as a non-dry eye. If a clear determination cannot be made, the value of the non-driving flag FG1 is set to 0.

[0040] Setting of the flag FG1 when not driving will be described with reference to Fig. 15. The process of step S350 includes steps S351 to S355. In step S351, a blinking pattern is determined. The estimation unit 13 performs the determination of step S351 based on the occurrence rate Ri (i = 1, 2, 3, 4) of each pattern acquired by the blink analysis.

[0041] When not driving, if the second pattern blink or the third pattern blink appears in addition to the shallow blink of the first pattern, the process proceeds to step S352, where the value of the non-driving flag FG1 is set to 3 as a high possibility of dry eye. When not driving, if only the shallow blink of the first pattern appears frequently, the process proceeds to step S353, where the value of the non-driving flag FG1 is set to 2 as a tendency of dry eye. When not driving, if the normal blink of the fourth pattern is mostly observed, the process proceeds to step S354, where the value of the non-driving flag FG1 is set to 1 as a non-dry eye. If a clear determination cannot be made, the process proceeds to step S355, where the value of the non-driving flag FG1 is set to 0. When the setting of the non-driving flag FG1 is completed, the control flow ends once and proceeds to step S310.

[0042] In this embodiment, as shown in Fig. 16, a comprehensive judgment is made according to a combination of the value of the flag FG1 when not driving and the value of the flag FG2 when driving. For example, if the value of the flag FG1 when not driving is 2 and the value of the flag FG2 when driving is also 2, it is estimated that there is a tendency for dry eye. In other words, if shallow blinking occurs both when driving and when not driving, it is estimated that there is a tendency for dry eye. In the case of such a combination, the dryness of the eyes 210 of the driver 200 is considered to be mild.

[0043] When the value of the flag FG1 when not driving is 2 and the value of the flag FG2 when driving is 3, it is estimated that the driver 200 has mild dry eye. It is considered that the dryness of the eyes 210 of the driver 200 is progressing.

[0044] If the value of the flag FG1 when not driving is 2 and the value of the flag FG2 when driving is 1, the dry eye estimation result is indefinite because such a combination of the flags FG1 and FG2 is considered to be contradictory.

[0045] When the value of the flag FG1 when not driving is 3 and the value of the flag FG2 when driving is 2, it is estimated that the dry eye is severe but not yet dry. Since blinking of the second or third pattern occurs when not driving, it is estimated that the degree of dry eye is severe. The reason why blinking of the second or third pattern does not occur when driving may be that the dryness of the eye 210 is mild or that the driver 200 blinks in a short time to gaze ahead.

[0046] When both the non-driving flag FG1 and the driving flag FG2 have a value of 3, it is estimated that the patient has severe dry eyes. It is also estimated that the dryness of the eyes is progressing due to the high workload of the eyes 210 caused by driving.

[0047] If the value of the flag FG1 when not driving is 3 and the value of the flag FG2 when driving is 1, the dry eye estimation result is indeterminate because such a combination of the flags FG1 and FG2 is considered to be contradictory.

[0048] When the value of the flag FG1 when not driving is 1 and the value of the flag FG2 when driving is 2, it is estimated that the patient has mild dry eye and the eyes are not yet dry. It is considered that shallow blinking occurs due to the increased workload of the eyes 210 caused by driving.

[0049] When the value of the flag FG1 when not driving is 1 and the value of the flag FG2 when driving is 3, it is estimated that the patient has mild dry eye and the dryness of the eyes is progressing. For example, it is assumed that the dryness of the eyes is progressing due to long hours of driving.

[0050] It is estimated that the driver does not have dry eye when both the flag FG1 when not driving and the flag FG2 when driving are 1. The driver state estimating device 1 of the present embodiment issues a warning or alert to the driver 200 based on such an estimation.

[0051] Returning to Fig. 12, in step S360, Perclos analysis is performed. Perclos is a value indicating the proportion of time the eyes are closed, and is an index relating to the degree of fatigue of the driver 200. The Perclos value is calculated, for example, by the detection unit 12. After the Perclos value is calculated, the process proceeds to step S370.

[0052] In step S370, it is determined whether the value of the flag FG2 during driving is 1. If the determination in step S370 is affirmative, the process proceeds to step S390, and if the determination is negative, the process proceeds to step S380.

[0053] In step S380, it is determined whether the value of flag FG2 during driving is 0. If the determination in step S380 is affirmative, the process proceeds to step S390, and if the determination is negative, the process proceeds to step S450.

[0054] In step S390, it is determined whether the vehicle has traveled for a specified time ΔTa or more. The specified time ΔTa is a threshold value for continuous driving time for prompting the driver 200 to take a break. The length of the specified time ΔTa is set according to the estimation result of dry eye. The value of the specified time ΔTa is set to any one of T0, T1, and T2. T0 is set in the case of no dry eye and is the longest time. T1 is set when it is determined that there is a possibility of dry eye. T2 is set when it is determined that there is a high possibility of dry eye and is the shortest time. The initial value of the specified time ΔTa is T0. If the determination in step S390 is affirmative, the process proceeds to step S400, and if the determination is negative, the process proceeds to step S420.

[0055] In step S400, the driving time is reset, and the process proceeds to step S410. In step S410, the driver 200 is prompted to take a rest, and the process proceeds to step S420.

[0056] In step S420, it is determined whether drowsiness is equal to or greater than a predetermined value. If the determination in step S420 is affirmative, the process proceeds to step S430, and if the determination is negative, the control flow ends and proceeds to step S310.

[0057] In step S430, the warning device calls the driver 200's attention to wake up from drowsiness, for example, by sound, a warning lamp, image display, etc. After step S430 is executed, the process proceeds to step S440.

[0058] In step S440, it is determined whether a warning release signal is present. If the determination in step S440 is affirmative, the control flow ends and proceeds to step S310, whereas if the determination is negative, the flow proceeds to step S430 and the output of the warning continues.

[0059] In step S450, it is determined whether the value of the non-driving flag FG1 is 0. If the determination in step S450 is affirmative, the process proceeds to step S460, and if the determination is negative, the process proceeds to step S470.

[0060] In step S460, the estimation unit 13 determines that there is a possibility of dry eye. The process proceeds to step S460 when the value of the flag FG1 when not driving is 0 and the value of the flag FG2 when driving is 2 or 3. In other words, it is impossible to determine or has not been determined when not driving, and it is estimated that there is a possibility of dry eye when driving. In this case, since it is considered that there is a possibility of dry eye, the value of the designated time ΔTa is set to T1. After step S460 is executed, the process proceeds to step S390.

[0061] In step S470, it is determined whether the value of the non-driving flag FG1 is 1. If the determination in step S470 is affirmative, the process proceeds to step S480, and if the determination is negative, the process proceeds to step S520.

[0062] In step S480, the estimation unit 13 determines that the patient has mild dry eye. The process proceeds to step S480 when the non-driving flag FG1 is 1 and the driving flag FG2 is 2 or 3. In other words, blinking is normal when the patient is not driving, but blinking while driving indicates a tendency toward dry eye. Since it is estimated that the patient has dry eyes, it is preferable to encourage the patient to take a break early. In step S480, the value of the designated time ΔTa is set to T1. After step S480 is executed, the process proceeds to step S490.

[0063] In step S490, it is determined whether the value of the driving flag FG2 is 2. If the determination in step S490 is affirmative, the process proceeds to step S390, and if the determination is negative, the process proceeds to step S500.

[0064] In step S500, it is determined whether the vehicle has been traveling for a specified time ΔTb or more. The specified time ΔTb is a time different from the specified time ΔTa. The specified time ΔTb may be a time shorter than the specified time ΔTa. If the determination in step S500 is affirmative, the process proceeds to step S510, and if the determination is negative, the process proceeds to step S390.

[0065] In step S510, a warning about dry eyes is output. This warning informs driver 200 that a tendency toward dry eyes is observed. When proceeding to step S510, the value of flag FG2 during driving is 3, and a blinking pattern that is highly likely to be a dry eye has appeared. Warning device 40 informs driver 200 to be careful about dry eyes, for example, by sound, a warning lamp, image display, etc. After step S510 is executed, the process proceeds to step S390.

[0066] In step S520, it is determined whether the value of the non-driving flag FG1 is 2. If the determination in step S520 is affirmative, the process proceeds to step S530, and if the determination is negative, the process proceeds to step S540.

[0067] In step S530, the estimation unit 13 determines that the patient has mild dry eye. The process proceeds to step S530 when the non-driving flag FG1 is 2 and the driving flag FG2 is 2 or 3. That is, a tendency to have dry eye is observed both when not driving and when driving. In step S530, the value of the designated time ΔTa is set to T1. After step S530 is executed, the process proceeds to step S490.

[0068] In step S540, the estimation unit 13 determines that the patient has severe dry eye. The process proceeds to step S540 when the non-driving flag FG1 is 3 and the driving flag FG2 is 2 or 3. That is, the patient shows a tendency toward dry eye when not driving and the eye workload is not high, and also when driving and the eye workload is high. In this case, it is estimated that the patient has a high possibility of having dry eye and that the degree of dry eye is severe. In step S540, the value of the designated time ΔTa is set to T2. After step S540 is executed, the process proceeds to step S550.

[0069] In step S550, it is determined whether the vehicle has been traveling for a specified time ΔTb or more. If the determination in step S550 is affirmative, the process proceeds to step S390, and if the determination is negative, the process proceeds to step S560.

[0070] In step S560, it is determined whether the vehicle has been traveling for a specified time ΔTb or more. If the determination in step S560 is affirmative, the process proceeds to step S570, and if the determination is negative, the process proceeds to step S390.

[0071] In step S570, a dry eye warning is output. This warning informs driver 200 that the dry eye is estimated to be severe. Warning device 40 informs driver 200 of the dry eye warning, for example, by sound, a warning lamp, or an image display. The warning in step S570 is output differently from the caution in step S510, and calls for greater attention. After step S570 is executed, the process proceeds to step S390.

[0072] As described above, the estimation unit 13 of this embodiment sets the driving flag FG2 based on the blinking pattern detected when the driver 200 is driving the vehicle 100, and sets the non-driving flag FG1 based on the blinking pattern detected when the driver 200 is not driving the vehicle 100. The estimation unit 13 estimates the possibility of dry eye in the driver 200 based on the non-driving flag FG1 and the driving flag FG2. That is, the estimation unit 13 estimates the possibility of dry eye based on both the blinking pattern when the eye workload is high and the blinking pattern when the eye workload is not high. Therefore, the driver state estimation device 1 of this embodiment can improve the estimation accuracy of dry eye.

[0073] As described above, the driver state estimation device 1 of this embodiment has the acquisition unit 11, the detection unit 12, and the estimation unit 13. The acquisition unit 11 acquires the eye opening degree EO of the driver 200 of the vehicle 100. The detection unit 12 detects the blinking pattern of the driver 200 based on the transition of the eye opening degree EO. The estimation unit 13 estimates the possibility of dry eye in the driver 200 based on the blinking pattern.

[0074] The estimation unit 13 of this embodiment estimates the possibility of dry eye in the driver 200 based on a blinking pattern detected when the driver 200 is driving the vehicle 100 and a blinking pattern detected when the driver 200 is not driving the vehicle 100. Therefore, the driver state estimation device 1 of this embodiment can improve the accuracy of estimating dry eye.

[0075] The detection unit 12 of this embodiment determines which of a plurality of predetermined patterns each blink of the driver 200 corresponds to. The estimation unit 13 estimates the possibility of dry eye in the driver 200 based on the occurrence rate of the plurality of patterns when the driver 200 is driving the vehicle 100 and the occurrence rate of the plurality of patterns when the driver 200 is not driving the vehicle 100. By using the occurrence rate of the plurality of patterns, it is possible to improve the accuracy of estimating dry eye.

[0076] In this embodiment, the multiple patterns include normal blinking patterns and abnormal blinking patterns indicating dry eyes. When an abnormal blinking pattern appears while the driver 200 is driving the vehicle 100, the estimation unit 13 estimates that there is a possibility of dry eye. The interval between blinks becomes longer while driving. For this reason, it is considered that the occurrence rate of abnormal blinking patterns becomes significant when the driver 200 has dry eye or is prone to dry eye. Therefore, the driver state estimation device 1 of this embodiment can improve the estimation accuracy of dry eye.

[0077] In this embodiment, the abnormal blinking pattern includes a blinking pattern indicating a tendency to dry eye and a blinking pattern indicating a high possibility of dry eye. When the following two conditions are met, the estimation unit 13 estimates that the driver 200 is highly likely to have dry eye. The first condition is that the abnormal blinking pattern appears when the driver 200 is driving the vehicle 100. The first condition is met when the value of the flag FG2 during driving is 2 or 3. The second condition is that the blinking pattern indicating a high possibility of dry eye appears when the driver 200 is not driving the vehicle 100. The second condition is met when the value of the flag FG1 during non-driving is 3. Since a blinking pattern indicating a high possibility of dry eye appears when not driving, it is possible to determine that the driver 200 is highly likely to have dry eye.

[0078] In this embodiment, the estimation unit 13 estimates that the eyes 210 of the driver 200 are becoming increasingly dry when the value of the flag FG2 during driving is 3. When the eyes 210 are becoming increasingly dry, the warning unit 14 executes a warning about dry eyes (step S510) or a dry eye warning (step S570). This allows the driver 200 to recognize that the eyes are becoming increasingly dry, and supports safe driving.

[0079] The estimation unit 13 may issue a caution or warning to the driver 200 when the value of the flag FG2 during driving changes to 3 while driving. For example, when the value of the flag FG2 during driving changes from 2 to 3, it is considered that the dryness of the eyes has progressed due to fatigue of the eyes 210. By calling the driver 200's attention in such a case, it is possible to support safe driving.

[0080] [First Modification of the Embodiment] A first modified example of the embodiment will be described. The first modified example of the embodiment is different from the above-mentioned embodiment in that, for example, vehicle control is performed to suppress dry eyes. The driver state estimation device 1 according to the first modified example of the embodiment executes vehicle control to suppress dry eyes when it is estimated that the driver 200 is highly likely to have dry eyes.

[0081] An example of vehicle control for suppressing dry eyes is air conditioning control of vehicle 100. For example, air conditioning control is switching an air conditioner from an inside air circulation state to an outside air introduction state. Air conditioning control may include changing the air direction of the air conditioner. In this case, it is preferable to make the air blowing direction different from the direction toward driver 200's face.

[0082] The vehicle control for suppressing dry eyes may include control for introducing outside air by opening the windows of the vehicle 100. The vehicle control for suppressing dry eyes may include control for cutting blue light from a display.

[0083] The contents disclosed in the above-mentioned embodiments and modifications can be implemented in appropriate combinations. [Explanation of symbols]

[0084] 1: Driver state estimation device 11: Acquisition unit, 12: Detection unit, 13: Estimation unit, 14: Warning unit 20: camera; 30: vehicle ECU; 40: warning device 100:Vehicle 200: driver, 210: eye EO: Eye openness FG1: Flag when not in operation, FG2: Flag when in operation R1, R2, R3, R4: Occurrence rate

Claims

1. An acquisition unit that acquires an eye opening degree of a driver of the vehicle; a detection unit that detects a blinking pattern of the driver based on a transition of the eye opening degree; an estimation unit that estimates a possibility of dry eye in the driver based on the blinking pattern; Equipped with The estimation unit estimates a possibility of dry eye in the driver based on the blinking pattern detected when the driver is driving the vehicle and the blinking pattern detected when the driver is not driving the vehicle. Driver state estimation device.

2. The detection unit determines which of a plurality of predetermined patterns each blink of the driver corresponds to, The estimation unit estimates a possibility of dry eye in the driver based on an occurrence rate of the plurality of patterns when the driver is driving the vehicle and an occurrence rate of the plurality of patterns when the driver is not driving the vehicle. The driver state estimating device according to claim 1 .

3. the plurality of patterns includes a normal blinking pattern and an abnormal blinking pattern indicative of dry eyes; The estimation unit estimates that the driver may have dry eye when the abnormal blinking pattern occurs while the driver is driving the vehicle. The driver state estimating device according to claim 2 .

4. The abnormal blinking pattern includes a blinking pattern indicating a tendency for dry eye and a blinking pattern indicating a high possibility of dry eye, The estimation unit estimates that the driver is likely to have dry eye when the abnormal blinking pattern appears while the driver is driving the vehicle and when the blinking pattern indicating the high possibility of dry eye appears while the driver is not driving the vehicle. The driver state estimating device according to claim 2 .

Images