Vehicle driver status detection device

Abstract

To provide a vehicle driver status detection device that can detect the driver's condition without burdening the driver. [Solution] The driver state detection safety device 1 comprises a camera 5 that takes images of the driver's eyes while seated in the driver's seat of a vehicle, a voice guidance device 6 that instructs the driver seated in the driver's seat to open and close their eyes, and a cognitive ability determination unit 10 that obtains parameters related to the driver's reaction speed, such as eye opening and closing speed and pupil diameter, from the image of the driver's eyes obtained by the camera 5 after the instruction to open and close the eyes, and calculates a determination score S based on these parameters to estimate the driver's cognitive ability. Furthermore, if the driver state detection safety device 1 determines that the driver's cognitive ability is impaired, it will implement driving restrictions, strengthen the control of the driver assistance control system 21, and issue a warning to the driver via a warning device 25.

Description

Technical Field

[0001] The present invention relates to a device for detecting the state of a vehicle driver.

Background Art

[0002] In order to prevent accidents caused by drowsy driving, various safety devices have been proposed. For example, in Patent Document 1, a photographing device for photographing the eyes of a vehicle driver is provided, the eyes of the driver are illuminated with light, and the response speed of the pupil to the light is measured from the photographed image. Then, for example, when it is measured that the response speed of the pupil is decreasing, it is estimated that the cognitive ability of the driver is decreasing, and the vehicle startup is suppressed, thereby suppressing the driving of the vehicle in a state where the cognitive ability is decreasing.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, since light is applied to the eyes of the driver to measure the response speed of the pupil, there is a problem that an afterimage of light remains in the eyes and driving cannot be started immediately after the measurement. In addition, for a driver with low tolerance to light, it may be difficult to apply light to the eyes before driving. Although it is conceivable to measure the cognitive ability of the driver other than the responsiveness of the pupil, in order to accurately estimate the cognitive ability of the driver due to drowsiness or the like, it is desirable to measure the cognitive ability of the driver based on the responsiveness of the pupil.

[0005] The present invention has been made in view of such problems, and an object thereof is to provide a driver state detection device that can detect the state of a driver without imposing a burden on the driver.

Means for Solving the Problems

[0006] To achieve the above objective, the vehicle driver state detection device of the present invention is characterized by comprising: an imaging unit that captures an image of the eyes of a driver seated in the driver's seat of a vehicle; an instruction unit that instructs the driver seated in the driver's seat to open and close their eyes; and an estimation unit that obtains parameters related to the reaction speed of the driver's eyes from the image of the driver's eyes acquired by the imaging unit after the instruction to open and close the eyes, and estimates the driver's cognitive ability based on said parameters. [Effects of the Invention]

[0007] The vehicle driver state detection device of the present invention obtains parameters related to the driver's eye reaction speed from images of the eyes taken by instructing the driver to open and close their eyes, and estimates the driver's cognitive ability from these parameters. Therefore, it does not require, for example, shining light into the driver's eyes to detect the reaction speed, and the burden on the driver can be reduced. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the configuration of a driver state detection safety device according to an embodiment of the present invention. [Figure 2] This is a part of a flowchart illustrating an example of the control procedure for driver state detection control in a driver state detection safety device. [Figure 3] This is the remainder of a flowchart showing an example of the control procedure for driver state detection control in a driver state detection device. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 is a block diagram showing the configuration of a driver state detection safety device 1 (driver state detection device) according to one embodiment of the present invention. The driver state detection safety device 1 (driver state detection device) of this disclosure is applicable to various vehicles, such as plug-in hybrid vehicles (PHEVs), hybrid vehicles (HEVs), electric vehicles (EVs), or engine-driven vehicles, which are capable of external charging and external power supply.

[0010] The driver state detection safety device 1 is installed in the vehicle and has the function of detecting when the driver of the vehicle is in a state where their cognitive ability is impaired and they are unsuitable for driving, such as when they are drowsy (a state of impaired cognitive ability), and outputting control signals to various driving systems of the vehicle based on the detected driver state to provide driving restrictions, driving assistance, and warnings.

[0011] As shown in Figure 1, the driver state detection safety device 1 of this disclosure includes a camera 5 (imaging unit), a voice guidance issuing device 6 (instruction unit), and a cognitive ability determination unit 10 (estimation unit). Camera 5 is a video camera that is installed, for example, near the driver's seat of a vehicle and is capable of filming the head of the driver seated in the driver's seat.

[0012] The voice guidance device 6 is a device that generates voice guidance, which will be described later, for the driver. The camera 5 and the voice guidance device 6 are powered by the vehicle's auxiliary battery 11. The cognitive ability determination unit 10 is installed in the vehicle's control unit (ECU), which has input / output devices, memory devices (ROM, RAM, non-volatile RAM, etc.), a central processing unit (CPU), a timer, etc. The cognitive ability determination unit 10 receives input of the timing when voice guidance is issued from the voice guidance issuing device 6, as well as an image of the driver's head taken by the camera 5, and determines the state of the driver's cognitive decline based on the image of the driver's head.

[0013] Furthermore, if the cognitive ability determination unit 10 determines that the driver is in a state of impaired cognitive ability, it controls the driving drive control device 20 (driving restriction control device) that controls the driving power source in the vehicle to suppress the vehicle's movement (preventing engine start, adjusting acceleration). Furthermore, if the cognitive ability determination unit 10 determines that the driver is in a state of cognitive decline, it outputs a control signal to the control unit (ADAS ECU) of the driver assistance control system 21 (driver assistance control device) to perform enhanced driver assistance control. As part of the enhanced driver assistance control, the driver assistance control system 21 performs, for example, enhanced steering assist function in the EPS 22 (electric power steering system) and enhanced sensitivity of the surrounding surveillance camera 23 used in the autonomous driving system.

[0014] Furthermore, if the cognitive ability assessment unit 10 determines that the driver is in a state of cognitive decline, it will issue a warning to the driver using the warning device 25. The warning device 25 includes, for example, a meter 26 (instrument panel), HUD 27 (head-up display), buzzer 28, and voice guidance device 29 located near the driver's seat in the vehicle.

[0015] Figures 2 and 3 are flowcharts showing an example of the control procedure for driver state detection control in the driver state detection safety device 1. The cognitive ability determination unit 10 performs the driver state detection control shown in Figures 2 and 3 at a timing before the vehicle starts moving, such as when the vehicle engine is started or when the vehicle power is turned on.

[0016] The cognitive ability assessment unit 10 first measures the light intensity L inside the vehicle (driver's seat) (step S10 in Figure 2). The light intensity L inside the vehicle may be estimated from images captured by the camera 5, or detected values ​​may be input from a light intensity sensor or the like installed in the vehicle. Next, the cognitive ability determination unit 10 determines from the head image of camera 5 whether the driver's gaze is directed towards the lens of camera 5 (step S20). If the cognitive ability determination unit 10 determines that the driver's gaze is not directed towards camera 5 (NO in step S20), it causes the voice guidance device 6 to provide voice guidance to direct the driver's gaze towards the lens of camera 5 (step S30). Then, it executes the process from step S20 onward again. This causes the driver to direct their gaze towards camera 5.

[0017] When the cognitive ability determination unit 10 determines that the line of sight is directed at the camera 5 (YES in step S20), it reads out image processing conditions corresponding to the light amount L measured in step S10 (step S40). The image processing conditions here are conditions such as the contrast ratio and brightness of the image so that, for example, the pupil diameter and the position of the head can be detected from the head image of the driver taken by the camera 5.

[0018] Next, the cognitive ability determination unit 10 starts shooting by the camera 5 and starts acquiring images of the driver's eyes and head (such as between the eyebrows) (step S50). Then, the cognitive ability determination unit 10 causes the voice guidance device 6 to give voice guidance to the driver to open and close the eyes (step S60). This voice guidance for opening and closing the eyes is, for example, content that guides the driver to first open the eyes, then prompt to close the eyes, and further repeat this opening and closing of the eyes.

[0019] After a predetermined time has elapsed since the voice guidance, the cognitive ability determination unit 10 ends the image acquisition (step S70), inputs the acquired images, and performs image processing according to the image processing conditions read out in step S40 (step S80). The cognitive ability determination unit 10 measures the head movement h of the driver from the acquired head image of the driver (the image after image processing) (step S90 in FIG. 3). The head movement h is, for example, the moving speed of the driver's head.

[0020] Next, the cognitive ability determination unit 10 measures the opening / closing speed H from the acquired image of the driver's eyes (step S100). Further, the cognitive ability determination unit 10 measures the pupil diameter P from the acquired image of the driver's eyes (step S110), and further measures the pupil diameter change speed ΔP (step S120).

[0021] Furthermore, the cognitive ability determination unit 10 calculates the left-right differences (ΔLRH, ΔLRP) for the opening / closing speed H and the pupil diameter P, respectively (step S130). Next, the cognitive ability assessment unit 10 calculates an assessment score S based on the eye opening / closing speed H, pupil diameter P, the left-right difference ΔLRH of the eye opening / closing speed H, the left-right difference ΔLRP of the pupil diameter P, and head movement h, as shown in formula (1) below.

[0022] S=Lha(αH+βP+γΔLRH+εΔLRP)...(1) Note that α, β, γ, and ε in equation (1) are coefficients set as appropriate. Lha in equation (1) is a correction coefficient set based on the light intensity L and head movement h in order to correct for the head movement h so that it does not affect the judgment score S (so that head movement h becomes 0).

[0023] If the cognitive ability determination unit 10 determines that the determination score S calculated by formula (1) is less than the first threshold S1 (safe side) (YES in step S150), it starts the vehicle's drive engine (step S160). If the vehicle is an EV, PHEV, or HEV that can be driven by an electric motor, it may be controlled to allow the operation of the electric motor. The first threshold S1 should be set near the upper limit of the determination score S within a range in which it can be determined that the driver is not in a state of cognitive impairment (no problems driving).

[0024] The cognitive ability determination unit 10 determines whether the determination score S is less than the second threshold S2 (step). The second threshold S2 should be set to a higher value on the dangerous side than the first threshold S1, but within a range where a warning to the driver is sufficient without requiring restrictions on vehicle operation, and should be set near the upper limit of the determination score S.

[0025] If the cognitive ability assessment unit 10 determines in step S170 that the assessment score S is less than the second threshold S2 (YES in step S170), it switches to the driving assistance function enhancement mode (step S180). The enhanced driving assistance mode is a mode in which the driving assistance control system 21 enhances the driving assistance control as described above. Furthermore, the cognitive ability determination unit 10 should warn the driver by various warning devices 25 that it has switched to the enhanced driving assistance mode.

[0026] If the cognitive ability determination unit 10 determines in step S170 that the determination score S is not less than the second threshold S2 (NO in step S170), it causes the vehicle to perform driving interlock control (step S190). As described above, the driving interlock control controls the driving drive control device 20 (driving restriction control device) to restrict the vehicle's movement, including preventing engine starting and restricting the operation of the driving drive motor. Furthermore, the cognitive ability determination unit 10 may warn the driver using various warning devices 25 if the vehicle is in a driving interlock state or if its cognitive ability is impaired.

[0027] In this way, the cognitive ability determination unit 10 acquires an image of the driver's eyes from an image of the driver's head before the vehicle is driven, such as when the vehicle power is turned on, measures the reaction speed of the driver's pupils, and calculates a determination score S corresponding to the driver's cognitive ability based on the measured reaction speed. If the determination score S is less than or equal to the first threshold S1, the cognitive ability determination unit 10 determines that the reaction speed is high and the cognitive ability necessary for driving the vehicle is ensured, and permits driving by starting the vehicle engine, etc.

[0028] If the cognitive ability determination unit 10 determines that the determination score S is between the first threshold S1 and the second threshold S2, it switches to the driving control enhancement mode. This ensures driving safety by enabling vehicle operation while switching to a driving mode that enhances safety (suppresses driving performance).

[0029] The cognitive ability determination unit 10, if it determines that the determination score S is equal to or greater than the second threshold S2, performs driving interlock control to restrict the operation of the vehicle. This improves safety by restricting the operation of the vehicle when the driver's cognitive ability is impaired. As described above, the driver state detection safety device 1 of this disclosure includes a camera 5 that takes images of the head, particularly the eyes, of the driver seated in the driver's seat of the vehicle, and a cognitive ability determination unit 10 that estimates the driver's cognitive ability from the images of the driver's eyes taken by the camera 5. Furthermore, the driver state detection safety device 1 includes a voice guidance device 6 that instructs the driver seated in the driver's seat to open and close their eyes, and the cognitive ability determination unit 10 obtains parameters related to the driver's reaction speed from the images of the driver's eyes taken by the camera 5 after the instruction to open and close the eyes, and estimates the driver's cognitive ability based on these parameters.

[0030] Thus, the driver state detection safety device 1 of this disclosure does not estimate cognitive ability by shining light into the driver's eyes and measuring, for example, the reaction speed of the pupil, but rather estimates the driver's cognitive ability from the image of the driver's eyes after having the driver open and close their eyes. Therefore, it is not necessary to shine light into the driver's eyes, and the driver's cognitive ability can be estimated without putting a strain on the driver's eyes. Consequently, even if cognitive ability is estimated immediately before driving the vehicle, if the cognitive ability is normal, driving can be started safely immediately.

[0031] The cognitive ability assessment unit 10 obtains parameters related to the driver's reaction speed from the driver's eye image, including the driver's eye opening and closing speed, pupil diameter when the eyes are open, and the rate of change of the pupil diameter, and further calculates the left-right difference in each of these parameters. In this way, various parameters related to the driver's eye reaction time are measured from images of the driver's eyes, a judgment score S is calculated from these parameters, and a comprehensive judgment is made based on the judgment score S. Therefore, it becomes possible to accurately estimate the driver's cognitive ability from these parameters related to the driver's eye reaction time.

[0032] The vehicle is equipped with a driving control device 20 that performs driving restriction control and a driver assistance control system 21 that performs driver assistance control. The cognitive ability determination unit 10, when it is estimated that the driver's cognitive ability is impaired, will strengthen the driving restriction control or strengthen the driver assistance control. This allows the vehicle's operation to be suppressed according to the driver's cognitive ability, thereby enhancing safety.

[0033] In particular, the cognitive ability determination unit 10 selectively strengthens driving restriction control and driving assistance control according to the value of the determination score S. Specifically, if the determination score S is less than the first threshold S1, the cognitive ability determination unit 10 does not strengthen driving restriction control or driving assistance control, but instead permits engine starting (permission to operate the driving power source). If the determination score S is 2 or higher than the second threshold S2, that is, if the driver's cognitive ability is not suitable for driving, the operation of the driving power source such as the engine is restricted, thereby restricting driving and improving safety. Furthermore, if the determination score S is 1 or higher than the first threshold S1 and less than the second threshold S2, driving is possible, but safety can be improved by strengthening driving assistance control. Therefore, it is possible to improve the usability of the vehicle by suppressing unnecessary driving restrictions while ensuring safety according to the degree of the driver's cognitive ability.

[0034] Furthermore, the cognitive ability assessment unit 10 acquires the head movement state from the image of the driver's head captured by the camera 5, and corrects the parameters related to the driver's reaction speed based on the head movement state. Therefore, even when the head is moving, information such as pupil dilation and closure speed can be accurately measured from the eye image. As a result, parameters related to the driver's reaction speed can be accurately measured from the image captured by the camera 5.

[0035] Furthermore, if the cognitive ability assessment unit 10 estimates that the driver's cognitive ability is impaired, the warning device 25 will warn the driver that their cognitive ability is impaired. This makes the driver aware of their impaired cognitive ability and encourages them to take action, such as refraining from driving and taking a break, thereby preventing reckless driving.

[0036] Furthermore, the cognitive ability assessment unit 10 notifies the driver via the warning device 25 when it restricts the operation of the driving power source or strengthens the driver assistance control according to the assessment score S. This prevents the driver from mistakenly believing that the vehicle is malfunctioning due to changes in the vehicle's behavior when the driving power source is restricted or the driver assistance control is strengthened.

[0037] This concludes the description of the embodiments, but the embodiments of the present invention are not limited to those described above. For example, the driver state detection device of this disclosure performs driver state detection control before the start of vehicle operation, but driver state detection control may also be performed when the vehicle is stopped, for example, due to a traffic light, while driving. In this case, if it is determined that the driver's cognitive ability is impaired, driving restriction control should not be performed, and only a warning or, furthermore, only enhanced driving assistance control should be provided. This makes it possible to make the driver aware of the situation even when their cognitive ability is impaired while driving, and to encourage them to take action, for example, a break. [Explanation of Symbols]

[0038] 1. Driver status detection safety device (driver status detection device) 5. Camera (imaging unit) 6. Voice guidance device (indicator unit) 10 Cognitive ability determination section (estimation section) 20. Driving control device (driving restriction control device) 21. Driver assistance control system (driver assistance control device) 25 Warning device

Claims

1. An imaging unit that captures images of the driver's eyes while seated in the driver's seat of the vehicle, An indicator unit that instructs the driver seated in the driver's seat to open and close their eyes, An estimation unit obtains parameters related to the driver's eye reaction speed from the image of the driver's eye acquired by the imaging unit after the instruction to open and close the eyes, and estimates the driver's cognitive ability based on said parameters. A vehicle driver status detection device characterized by comprising the following:

2. A vehicle driver state detection device according to claim 1, applicable to a vehicle equipped with a driving restriction control device that performs driving restriction control and a driver assistance control device that performs driver assistance control, The estimation unit, when it is estimated that the driver's cognitive ability has decreased below a predetermined level, will perform at least one of the following: strengthening of the driving restriction control and strengthening of the driving assistance control. A vehicle driver status detection device characterized by the following features.

3. The estimation unit obtains the driver's eye opening and closing speeds, pupil diameter when eyes are open, the rate of change of the pupil diameter, and the difference between the left and right pupil diameters from the driver's eye image, and calculates parameters related to the driver's reaction speed from the eye opening and closing speeds, pupil diameter, rate of change of pupil diameter, and the difference between the left and right pupil diameters. A vehicle driver status detection device according to claim 1 or 2.

4. The imaging unit captures an image of the driver's head along with the image of the eyes. The estimation unit obtains the movement state of the head from the head image and corrects the parameters related to the driver's reaction speed based on the movement state of the head. The vehicle driver status detection device according to feature 3.

5. A vehicle driver status detection device according to claim 1 or 2, The vehicle is equipped with a warning device that warns the driver, The estimation unit, when it is estimated that the driver's cognitive abilities are impaired, will cause the warning device to warn the driver that their cognitive abilities are impaired. A vehicle driver status detection device characterized by the following features.

6. A vehicle driver status detection device according to claim 2, The vehicle is equipped with a warning device that warns the driver, The estimation unit estimates that the driver's cognitive abilities are impaired, and when it initiates the strengthening of the driving restriction control and the strengthening of the driver assistance control, the warning device warns the driver about the strengthening of the driving restriction control and the strengthening of the driver assistance control. The vehicle driver status detection device according to feature 2.

Images