Driver state determination method and device

Abstract

To provide a driver state determination method and a device that can determine an abnormal state of a driver with high accuracy even if a driver's closed eye state cannot be confirmed.SOLUTION: A driver abnormality state determination step performed by an abnormality determination unit determines that a driver is in an abnormal state when a closed eye state is determined in a closed eye determination step S23, if a head posture is an abnormal posture and the closed eye state continues for a period corresponding to a third determination threshold L3, warns the driver when the closed eye state is not determined in the closed eye determination step S23, if the head posture is the abnormal posture and the non-closed eye state continues for a period corresponding to a fourth determination threshold L4 different from the third determination threshold L3, and determines that the driver is in the abnormal state when, after the warning, the head posture acquired in a head posture acquisition step S21 is the abnormal posture and a situation where a driving operation amount acquired in a driving operation amount acquisition step S21 does not change continues for a second non-operation determination time.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a driver condition determination method and device for detecting an abnormal condition of a driver who drives a vehicle. [Background technology]

[0002] BACKGROUND ART Driver support devices have been known that accurately grasp a driver's state of consciousness, particularly the driver's state of alertness, and support the driver in driving a vehicle in accordance with the driver's state of alertness. The drowsy driving prevention device of Patent Document 1 has a drowsiness level detection means that detects the driver's drowsiness level based on the driver's driving (steering) behavior and vehicle behavior, an alarm output means that outputs an alarm when the detected drowsiness level is higher than a predetermined alarm threshold, an alarm threshold setting means that sets the alarm threshold for the period from the start of driving to after the learning time has elapsed, and an alarm threshold correction means that corrects the alarm threshold based on the drowsiness level.The alarm threshold correction means calculates a corrected judgment reference value proportional to the average value of the drowsiness level detected over a period longer than the learning time, and corrects the alarm threshold downward if this corrected judgment reference value is smaller than the alarm threshold and the total time during which a drowsiness level higher than the corrected judgment reference value has been detected is equal to or longer than a judgment time longer than the learning time.

[0003] There is also a known driver assistance device that automatically moves the vehicle toward the shoulder and autonomously stops it when the driver's physical condition suddenly changes and the driver is unable to continue driving, i.e., the driver is in an abnormal state. Typically, a driver's physical condition and drowsiness level are determined based on head-related information such as the driver's head posture and facial state acquired by an imaging device such as a camera. Among the head-related information, the head posture is detected, for example, by the driver's facial direction and head position (three-dimensional coordinates), and the facial state is detected, for example, by the driver's facial expression and eye opening state (eyelid state). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6226135 Summary of the Invention [Problem to be solved by the invention]

[0005] The drowsy driving prevention device of Patent Document 1 does not require facial condition or biological information of the driver, so that a detection device can be omitted, and an increase in the cost of the entire device can be suppressed. However, the drowsy driving prevention device of Patent Document 1 does not involve the actual state of the driver, that is, it indirectly determines the driver's state of consciousness, etc. through the vehicle behavior, so there is a risk that it will not be able to accurately determine the driver's abnormal state. If an abnormal state of the driver is erroneously determined, not only will an alarm be issued to the driver of the vehicle, causing increased inconvenience to the driver, but it may also cause an unexpected situation for other vehicles, such as the driver's vehicle being forced to make an emergency stop.

[0006] Therefore, in order to avoid erroneous determination of an abnormal state, it is conceivable to detect closed eyes, which are a sign of a driver in an abnormal state, and poor head posture, which is a physical state of the driver in an abnormal state, and then, when both of these abnormality determination conditions are met, to confirm the determination of the driver's abnormal state. However, the driver's eyes, which are the subject of judgment, are an extremely small area to be imaged, and depending on the imaging environment and timing, situations may arise where the captured image itself becomes unclear, or where the driver's movements mean that both of the driver's eyes do not fit within the determinable imaging area, making it physically difficult to detect the eyes that are the subject of imaging.

[0007] Whether the driver's eyes are closed is determined using image information (image data) captured by an imaging device installed inside the vehicle. Therefore, if it is physically difficult to detect the driver's eyes themselves using the imaging device, it is not possible to reliably determine whether the driver's eyes are open or closed in terms of control processing. That is, it is not easy to determine with high accuracy whether the driver is in an abnormal state.

[0008] The object of the present invention is to provide a driver state determination method and device that can accurately determine whether a driver is in an abnormal state even in situations where it is not possible to confirm that the driver's eyes are closed, which is a sign of an abnormality. [Means for solving the problem]

[0009] A driver state determination method according to claim 1 is a driver state determination method for detecting an abnormal state of a driver of a vehicle, the method comprising: a head posture acquisition step of acquiring a facial direction and a head position of the driver; an eye closure determination step of determining whether the eyes of the driver are closed; a driving operation amount acquisition step of acquiring at least one driving operation amount of the vehicle by the driver at a predetermined timing; and a driver abnormal state determination step of determining whether the driver is in an abnormal state, wherein the driver abnormal state determination step determines that the driver is in an abnormal state if, in a situation where an eye closure state is determined in the eye closure determination step, the head posture of the driver acquired in the head posture acquisition step is an abnormal posture and the eye closure state continues for a first predetermined time, and, in a situation where an eye closure state is determined not to be a closed state in the eye closure determination step, the state where the head posture of the driver acquired in the head posture acquisition step is an abnormal posture and the eye closure state continues for a first predetermined time shorter than If the situation continues for a second predetermined time, the driver is notified, and if, after the notification, the situation continues for a third predetermined time in which the head posture of the driver acquired in the head posture acquisition process is an abnormal posture and the driving operation amount acquired in the driving operation amount acquisition process does not change, the driver is determined to be in an abnormal state.

[0010] This driver state determination method includes a head posture acquisition process for acquiring the driver's facial direction and head position, an eye closure determination process for determining whether the driver's eyes are closed, a driving operation amount acquisition process for acquiring at least one of the driving operation amounts of the vehicle by the driver at a predetermined timing, and a driver abnormal state determination process for determining whether the driver is in an abnormal state.Therefore, poor head posture, which is the driver's physical state in an abnormal state (hereinafter referred to as the abnormal state), and closed eyes, which are symptoms of the driver in an abnormal state (hereinafter referred to as the abnormal state symptom), can be used as conditions for determining the driver's abnormal state. The driver abnormal state determination process determines that the driver is in an abnormal state if, in a situation where a closed eye state is determined in the closed eye determination process, the driver's head posture acquired in the head posture acquisition process is an abnormal posture and the closed eye state continues for a first predetermined time, so that the driver's abnormal state can be determined based on directly detected abnormal symptoms and abnormal state. The driver abnormal state determination step is a step of determining whether the driver's head posture acquired in the head posture acquisition step is an abnormal posture and the driver's eyes are not closed for a first predetermined time period in a situation where the driver's eyes are not closed in the eye closure determination step. shorter than If this continues for a second predetermined time, the driver is notified, and if after the notification the driver's head posture acquired in the head posture acquisition process is abnormal and the driving operation amount acquired in the driving operation amount acquisition process does not change continues for a third predetermined time, the driver is determined to be in an abnormal state.Therefore, even if closed eyes, which are an abnormal symptom, cannot be confirmed, the driver's behavior in an abnormal state (hereinafter referred to as abnormal state behavior) can be used as a substitute for the abnormal state symptom, and this non-operation of driving, which is an abnormal state behavior, and collapse of head posture, which is an abnormal state, can be used as conditions for determining an abnormal state. Furthermore, since the state is confirmed by notification before the abnormal state of the driver is determined based on the abnormal state and abnormal action, the accuracy of determining the abnormal state can be increased.

[0011] Since the second predetermined time is shorter than the first predetermined time, It is possible to confirm in the shortest possible time that the eyes cannot be closed.

[0012] Claim 2 The invention is 1 In the invention of the present invention, the step of determining whether the driver's abnormal state is present is characterized in that it is determined that the head posture is abnormal based on at least the angle of the driver's face. With this configuration, it is possible to easily determine whether the head posture is abnormal.

[0013] Claim 3 The invention is 1 or 2In the invention, the driver abnormal state determination process is characterized in that, in a situation where an open-eye state is determined in the closed-eye determination process, the head posture of the driver acquired in the head posture acquisition process is an abnormal posture and the vehicle continues to be in a traveling state for a fourth predetermined time, the driver abnormal state determination process determines that the driver is in an abnormal state. With this configuration, an abnormal state when the driver has his eyes open can be determined based on the driver's state and the vehicle behavior.

[0014] Claim 4 The driver state determination device for detecting an abnormal state of a vehicle driver includes a head posture acquisition means for acquiring a facial direction and a head position of the driver, an eye closure determination means for determining whether the eyes of the driver are closed, a driving operation amount acquisition means for acquiring at least one of amounts of driving operation of the vehicle by the driver at a predetermined timing, and a driver abnormal state determination means for determining whether the driver is in an abnormal state, and the driver abnormal state determination means determines that the driver is in an abnormal state when the head posture of the driver acquired by the head posture acquisition means is an abnormal posture and the eye closed state continues for a first predetermined time in a situation where the eye closure determination means determines that the eye is not closed, and when the head posture of the driver acquired by the head posture acquisition means is an abnormal posture and the eye is not closed for the first predetermined time shorter than If the situation continues for a second predetermined time, the driver is notified, and if, after the notification, the situation continues for a third predetermined time in which the head posture of the driver acquired by the head posture acquisition means is an abnormal posture and the driving operation amount acquired by the driving operation amount acquisition means does not change, the driver is determined to be in an abnormal state.

[0015] This driver state determination device has a head posture acquisition means for acquiring the driver's facial direction and head position, an eye closure determination means for determining whether the driver's eyes are closed, a driving operation amount acquisition means for acquiring at least one of the driving operation amounts of the vehicle by the driver at a predetermined timing, and a driver abnormal state determination means for determining whether the driver is in an abnormal state, so that poor head posture, which is an abnormal state, and closed eyes, which are a symptom of an abnormal state, can be used as conditions for determining the driver's abnormal state. The driver abnormal state determination means determines that the driver is in an abnormal state if, in a situation where the eye closure determination means determines that the driver is in a closed eye state, the head posture of the driver acquired by the head posture acquisition means is an abnormal posture and the eye closure state continues for a first predetermined time, and therefore can determine the driver's abnormal state based on directly detected abnormal symptoms and abnormal state. The driver abnormal state determination means determines that the eyes are not closed when the eye closure determination means determines that the head posture of the driver acquired by the head posture acquisition means is an abnormal posture and the eyes are not closed for a first predetermined time. shorter than If this continues for a second predetermined time, the driver is notified, and if after the notification, the driver's head posture acquired by the head posture acquisition means is abnormal and the driving operation amount acquired by the driving operation amount acquisition means does not change for a third predetermined time, the driver is determined to be in an abnormal state, so even if closed eyes, which are an abnormal symptom, cannot be confirmed, an abnormal behavior can be used as a substitute for the abnormal symptom, and no driving operation, which is an abnormal behavior, and poor head posture, which is an abnormal state, can be used as conditions for determining the abnormal state. Moreover, because the condition is confirmed by the notification before determining the driver's abnormal state based on the abnormal state and abnormal behavior, the accuracy of determining the abnormal state can be improved. Since the second predetermined time is shorter than the first predetermined time, it is possible to confirm in the shortest time that the eyes cannot be confirmed to be closed. [Effects of the Invention]

[0016] According to the driver condition determination method and device of the present invention, the conditions for determining an abnormal condition are non-operation of the vehicle, which is an abnormal behavior, and poor head posture, which is an abnormal state, so that the driver's abnormal condition can be determined with high accuracy even in situations where the driver's eyes, which are a sign of an abnormal condition, cannot be confirmed. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a block diagram of a driver state determination device according to a first embodiment. [Figure 2] FIG. 1 is an explanatory diagram of a monitoring support device. [Figure 3] FIG. 2 is an explanatory diagram of a braking assist device. [Figure 4] FIG. 2 is an explanatory diagram of a vehicle stopping assistance device. [Figure 5] 10 is a flowchart showing a control process for determining whether the eyes are closed. [Figure 6] 10 is a flowchart showing a head posture determination control process. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. The following description exemplifies the application of the present invention to a driver state determination device for a vehicle, and does not limit the present invention, its applications, or its uses. [Example]

[0019] A first embodiment of the present invention will be described below with reference to FIGS. The driver state determination device M is a driving assistance device that determines the driver's state of consciousness, particularly an abnormal state that makes it difficult for the driver to operate the vehicle (hereinafter also referred to as the driver's own vehicle), and is capable of autonomously making an emergency stop if an abnormal state of the driver is determined. Hereinafter, the abnormal state of the driver will be explained as a state in which the driver is not consciously awake and is physically unable to perform driving operations due to poor posture (including the onset of illness or drowsiness). The following description also includes a description of a driver state determination method.

[0020] As shown in FIG. 1, the driver state determination device M mainly comprises a plurality of detection means 1 capable of detecting the driver's operating behavior and the vehicle's driving behavior, a driver assistance device 2 that autonomously provides vehicle driving assistance to the driver from the vehicle side, and an ECU (Electronic Control Unit) 3 that performs various calculations based on input signals input from the detection means 1 and outputs various command signals to the driver assistance device 2.

[0021] First, the detection means 1 will be described. 1, the detection means 1 includes a camera 11 that captures an image of the driver, a vehicle speed sensor 12 that detects the speed of the vehicle, a steering angle sensor 13 that detects the steering angle caused by operation of the steering wheel (not shown), a grip sensor 14 that detects the driver's grip on the steering wheel, an accelerator sensor 15 that detects the amount of depression of the accelerator pedal (not shown), a brake sensor 16 that detects the amount of depression of the brake pedal (not shown), and a hazard switch 17 that activates hazard lamps (not shown). Each of the sensors 12 to 16 corresponds to a driving operation amount acquisition means that acquires the driving operation amount at a predetermined timing, i.e., the current time immediately before the judgment time.

[0022] The camera 11 is configured by, for example, a CCD (Charge Coupled Device) camera, and is mounted in a position facing the driver at the center of the upper part of an instrument panel (not shown). This camera 11 is configured to be able to capture an image of the head, including the face, of a driver seated in the driver's seat and fastening a seat belt from the front. The captured head image is configured to be able to extract an image of the driver's face portion and detect the direction of the face (direction of face orientation), identify the driver's facial expression, enlarge and detect the iris of the eye to determine the direction of the line of sight, enlarge and detect the eyelid portion to determine the degree of eye opening and closing, and the three-dimensional coordinate position of the driver's head (center of gravity), etc. The camera 11 is required to be capable of acquiring at least the necessary captured images, and may be equipped with two cameras, a front-facing camera and an upward camera installed directly above the driver's seat, or three or more cameras.

[0023] Next, the driver assistance device 2 will be described. As shown in FIG. 1, the driver assistance device 2 is composed of a monitoring assistance device 21 that notifies the driver of an alert state, a braking assistance device 22 that causes the vehicle to perform a braking operation, and a stopping assistance device 23 that causes the vehicle to perform an emergency stop operation.

[0024] The monitoring support device 21 is configured to recognize the state of consciousness, that is, the wakefulness state of the driver, and, if it determines that recovery is necessary, to notify the driver and encourage the driver to recover consciousness. When the monitoring support device 21 receives an inattentive driving signal or a drowsiness estimation signal from the ECU 3, it issues a warning to the driver from one or more speakers (not shown) provided in the vehicle cabin. 2, a speedometer unit 21a disposed on an instrument panel is provided with a display unit 21b capable of displaying a warning. When the monitoring support device 21 receives an inattentive driving signal or a drowsiness estimation signal from the ECU 3, the monitoring support device 21 displays a warning such as "Watch out for the road ahead" on the display unit 21b.

[0025] The braking assist device 22 is configured to issue a warning when the vehicle approaches another vehicle ahead while traveling, and to apply emergency braking when the braking limit distance at which a collision can be avoided is reached. As shown in FIG. 3, when the host vehicle approaches the leading vehicle B to the position of the host vehicle A1, the braking assist device 22 issues a warning to the driver from the speaker. The separation distance between the other vehicle B and the host vehicle A1 is a distance corresponding to the time (for example, 0.8 seconds) that the driver can normally avoid a collision by his / her own driving operation (manual operation). The braking assist device 22 automatically applies emergency braking when the host vehicle approaches the leading vehicle B up to the position of the host vehicle A2. The separation distance between the leading vehicle B and the host vehicle A2 is the limit distance at which a collision can be avoided by applying emergency braking equivalent to the maximum braking force.

[0026] 3, when the braking assist device 22 receives an inattentive driving signal or a drowsiness estimation signal from the ECU 3, the braking assist device 22 issues an alarm to the driver from the speaker when the host vehicle approaches a preceding vehicle B to a position of the host vehicle A3 that is farther away from the preceding vehicle B than the position of the host vehicle A1. Here, the separation distance between the other vehicle B and the host vehicle A3 is a distance corresponding to the time (e.g., 1.2 seconds) that the driver can avoid a collision by driving the vehicle himself even if he is distracted.

[0027] When the stopping assistance device 23 receives a drowsiness estimation signal from the ECU 3, it issues an alarm to the driver from a speaker, and after receiving a drowsiness confirmation signal, it autonomously brings the vehicle to an emergency stop. As shown in Figure 4, when the vehicle stopping assistance device 23 receives a drowsiness estimation signal corresponding to an eyes-closed state at the location of the host vehicle X1, it activates the speaker to issue an alarm, and receives a drowsiness confirmation signal corresponding to a state of no driving operation at the location of the host vehicle X2. Next, at the location of the host vehicle X3, automatic driving begins even if the host vehicle has been manually driven by the driver until then. After starting automatic driving, the host vehicle moves to a safety zone such as the shoulder of the road and then makes an autonomous emergency stop at the location of the host vehicle X4.

[0028] Furthermore, when the vehicle stopping assistance device 23 receives an abnormality signal from the ECU 3, it autonomously brings the host vehicle to an emergency stop. As shown in FIG. 4, the vehicle stopping assistance device 23 receives an abnormality signal from the driver at the position of the host vehicle X2. Next, at the host vehicle X3, automatic driving is initiated even if the host vehicle has been manually driven by the driver up until that point. After starting automatic driving, the host vehicle moves to a safety zone such as the shoulder of the road and then makes an autonomous emergency stop at the position of the host vehicle X4. Thereafter, the vehicle stopping assistance device 23 makes an emergency call to an emergency center or the like, for example, after a predetermined vehicle stop maintenance period has elapsed. Hereinafter, this autonomous emergency stop control will be referred to as abnormality sequence control.

[0029] Next, the ECU 3 will be described. When the driver has both eyes closed and the head posture is out of order, or when the driver has the head posture out of order and is not operating the vehicle, the ECU 3 determines that the driver is in an abnormal state and outputs an abnormality signal. The ECU 3 is composed of a CPU (Central Processing Unit), ROM, RAM, an internal interface, an external interface, etc. The ROM stores various programs and data for brake control, display control, etc., and the RAM has a processing area used when the CPU performs a series of processes.

[0030] As shown in FIG. 1, the ECU 3 includes a head posture determination unit 31 that determines the head posture state of the driver, an eye state determination unit 32 (eye closure determination means) that determines the open / closed state of both the driver's left and right eyes, and a driver state determination unit 33 that determines the driver state that may be an obstacle to driving the vehicle.

[0031] Head posture determination unit 31 determines the head posture state of the driver based on the captured image information captured by camera 11. This head posture determination unit 31 determines that the head posture state of the driver is out of alignment when it detects that the head posture state of the driver is either head down, lying to the side, leaning face down, or leaning backward. When a head posture error is detected, the first counter C1 is set to 1. The initial value of the first counter C1 is 0.

[0032] For example, the head posture determination unit 31 outputs a head-down determination signal when the absolute value of the face direction angle in the pitch direction is equal to or greater than a predetermined threshold, and outputs a sideways (leaning) determination signal when the absolute value of the face direction angle in the roll direction is equal to or greater than the predetermined threshold. Furthermore, the head posture determination unit 31 outputs a head-down determination signal when the absolute value of the face direction angle in the pitch direction is equal to or greater than the predetermined threshold and the current head position is equal to or greater than a value obtained by subtracting a head position threshold from a reference head position, and outputs a backward tilt determination signal when the face direction angle in the pitch direction is equal to or greater than the predetermined threshold and the current head position is smaller than a value obtained by adding the head position threshold to the reference head position. The respective thresholds related to the angles and head positions used to determine head-down, sideways, leaning, and backward tilt correspond to head posture determination thresholds used to determine abnormal head posture of the driver.

[0033] The eye state determination unit 32 determines whether the driver's eyes are closed or not. The eye state determination unit 32 detects the driver's upper and lower eyelids captured by the camera 11, for example, and detects the degree of eye opening of the driver based on the number of pixels between the edges of the eyelids. If the detected degree of eye opening is equal to or greater than a predetermined determination threshold, it is determined that the driver's eyes are open. If the detected degree of eye opening is less than the predetermined determination threshold, it is determined that the driver's eyes are closed. When the eye state is determined to be closed, 1 is assigned to the second counter C2. The initial value of the second counter C2 is 0. In this embodiment, the eyes are determined to be closed only when both the left and right eyes of the driver are determined to be closed, but in other cases, the eyes are not determined to be closed.

[0034] When the first counter C1 is 1 and the second counter C2 is 1, ie, when it is determined that the driver is in an abnormal state, the third counter C3 is set to 1. The initial value of the third counter C3 is 0. Furthermore, if the first counter C1 is 1 (head posture abnormality) or greater and the open / closed state of the driver's eyes cannot be detected due to physical or environmental conditions, the fourth counter C4 is set to 1. The initial value of the fourth counter C3 is 0. Note that if the state of the driver's eyes can be detected, the driver's head posture state is detectable.

[0035] The driver state determination unit 33 includes an inattentiveness determination unit 33a that determines whether the driver is looking in a direction other than the direction ahead in the direction of travel, a drowsiness determination unit 33b that determines a drowsiness estimation state in which the driver is presumed to be drowsy and a drowsy confirmation state in which the driver is detected to be drowsy with a high probability, and an abnormality determination unit 33c (driver abnormal state determination means) that determines an abnormal state of the driver in which it is difficult to drive the vehicle.

[0036] The inattentive driving determining unit 33a will be described. The inattentiveness determination unit 33a determines whether the driver is looking away from the driver based on the driver's face direction information, gaze direction information, and vehicle information, and if it is determined that the driver is looking away, it outputs an inattentiveness signal to the monitoring support device 21 and the braking support device 22. When the count value of the driver's gaze direction in the yaw direction falls outside the range of the yaw direction threshold, it is determined that the driver is looking aside. The yaw direction threshold is changed according to the vehicle speed or the radius of curvature when the vehicle is traveling. Also, when the count value of the driver's gaze direction in the pitch direction falls outside the range of the pitch direction threshold, it is determined that the driver is looking aside. The pitch direction threshold is changed according to the vehicle speed.

[0037] The count value of the yaw direction gaze direction and the count value of the pitch direction gaze direction are calculated by a long-term continuous inattentive count or a short-term cumulative inattentive count. The long-term continuous inattentive count is the number of times that the determination results for each processing cycle continue in chronological order. The short-term cumulative inattentive count is the cumulative count of the determination results for each processing cycle when the conditions are met within a predetermined period of time prior to that.

[0038] The reliability of the gaze direction information in the yaw and pitch directions is determined using a preset reliability threshold. The reliability of each gaze direction information is set based on the clarity of the captured image of the driver (clarity of the contours of the face, eyelids, pupils, etc.). If the reliability of each gaze direction information is low and each gaze direction is invalid, it is interpolated using face direction information in the yaw and pitch directions.If the reliability of face direction information in the yaw and pitch directions is low and each face direction is invalid, it is interpolated using the previous valid gaze direction.

[0039] Next, the drowsiness determination unit 33b will be described. The drowsiness determination unit 33b determines whether the driver is drowsy or not based on the duration of eye closure and vehicle information, etc., and if it is estimated that the driver is drowsy, it outputs a drowsiness estimation signal to the monitoring support device 21, the braking support device 22, and the stopping support device 23, and if it is confirmed that the driver is drowsy, it outputs a drowsiness confirmation signal to the stopping support device 23.

[0040] The drowsiness determining unit 33b estimates a drowsy state when the driver's continuous eye-closure time exceeds a predetermined time threshold, or when the driver repeatedly closes his / her eyes for short periods exceeding the predetermined time threshold within a certain period of time. In addition, the drowsiness determination unit 33b determines the drowsy state if the driver's continuous eye closure time exceeds a predetermined time threshold, or if the continuous eye closure and no driving operation state exceeds a time threshold, while the vehicle is traveling at a certain speed or above after a drowsiness warning.

[0041] The non-operation state of the driving is determined as an inappropriate driving operation while driving by comparing the detection signal of the detection means 1 with a judgment threshold. Specifically, it is determined by at least one of the following operating states: non-operation of the steering wheel, non-grasping of the steering wheel, and non-operation of the accelerator or brake. Note that an extremely small judgment threshold may be set for each operating member to determine an inappropriate driving operation.

[0042] Next, the abnormality determination unit 33c will be described. The abnormality determination unit 33c determines whether the driver has their eyes closed and their head posture is poor, or whether the driver has their head posture poor and is not operating the vehicle, in other words, whether the driver is in an abnormal state in which it is difficult for them to operate the vehicle, using two of the conditions of closed eyes, poor head posture, and not operating the vehicle.

[0043] The inventors have found that the driver's symptoms when an abnormality occurs can be detected using closed eyes as a parameter, the driver's physical state when an abnormality occurs can be detected using poor head posture as a parameter, and the driver's behavior when an abnormality occurs can be detected using non-operation of the driving force as a parameter. Based on this finding, in this embodiment, the detection of closed eyes and poor head posture is used as the basic judgment condition, and when it is physically or situationally difficult to judge whether the eyes are closed, the detection of poor head posture and non-operation of the driving force is used as the judgment condition.

[0044] Next, the eye closure determination control process will be described with reference to the flowchart of FIG. Incidentally, Si (i=1, 2, . . . ) indicates a step for each process.

[0045] First, as shown in FIG. 5, the ECU 3 receives input of outputs from various detection means 1 such as the camera 11 and the hazard switch 17 (S1), and then proceeds to S2. In S2, it is determined whether or not both of the driver's eyes are closed. If the result of the determination in S2 is that the driver has their eyes closed, the process moves to S3 because closed eyes are detected, which is a sign of an abnormality (disease onset or drowsiness in the driver).If the result of the determination in S2 is that the driver does not have their eyes closed, at least one eye is open, so the process returns.

[0046] In S3, it is determined whether the count value of the first counter C1 is greater than 0, that is, whether the driver's head posture is out of alignment. If the result of the determination in S3 is that the count value of the first counter C1 is greater than 0, both the first counter C1 and the second counter C2 are not currently 0, and the process proceeds to S4. If the result of the determination in S3 is that the count value of the first counter C1 is 0, the driver's head posture is not out of alignment, and the process proceeds to S7. In S4, the current count value of the third counter C3 is incremented by 1, and then the process proceeds to S5.

[0047] In S5, it is determined whether the count value of the third counter C3 is equal to or greater than a third determination threshold L3 (first predetermined time). If the result of the determination in S5 is that the count value of the third counter C3 is equal to or greater than the third determination threshold L3, it is determined that the driver is in an abnormal state, and the process proceeds to S6. If the result of the determination in S5 is that the count value of the third counter C3 is less than the third determination threshold L3, it is not possible to determine whether the driver is in an abnormal state, and the process returns. In S6, the abnormality sequence control is executed, and then the process returns. In S6, 0 is substituted into the first to third counters C1 to C3, respectively.

[0048] In S7, the current count value of the second counter C2 is incremented by 1, and then the process proceeds to S8. In S8, it is determined whether the count value of the second counter C2 is equal to or greater than the second determination threshold L2. If the result of the determination in S8 is that the count value of the second counter C2 is equal to or greater than the second determination threshold L2, the onset of a disease or dozing is predicted, and the process proceeds to S9. If the result of the determination in S8 is that the count value of the second counter C2 is less than the second determination threshold L2, the onset of a disease or dozing cannot be predicted under the current circumstances, and the process returns.

[0049] In S9, an alarm is issued, and then the process proceeds to S10. In S10, it is determined whether or not the first determination condition is met. The first determination conditions are the following three conditions. (1) The driver's head posture is abnormal. (2) The driver has not operated the vehicle for the first non-operation determination period. (3) The driver's eyes are closed for a certain period of time.

[0050] If the first determination condition is met as a result of the determination in S10, the driver will not start driving even if an alarm is issued, so the process proceeds to S11. If the first determination condition is not met as a result of the determination in S10, the driver will start driving due to the alarm, so the process returns.

[0051] In S11, it is determined whether or not a cancel signal for stopping the alarm has been generated. If the result of the determination in S11 is that a cancel signal has been generated, the driver has performed an operation to generate a cancel signal, so the count value of the second counter C2 is set to 0 (S12), and then the process returns. If the result of the determination in S11 is that a cancel signal has not been generated, the driver cannot cancel, so the process proceeds to S6. In this embodiment, a cancel signal is output when the hazard switch 17 is turned on. The output of the cancel signal stops the alarm and stops the abnormal state determination.

[0052] Next, the head posture determination control process will be described with reference to the flowchart of Fig. 6. This head posture determination control process is controlled independently of and in parallel with the eye closure determination control process. Si (i=21, 22...) indicates the step for each process.

[0053] First, as shown in FIG. 6, the ECU 3 receives input of outputs from various detection means 1 such as the camera 11 and the hazard switch 17 (S21), and then proceeds to S22. In S22, it is determined whether the driver's head is out of position based on the first head posture determination threshold. If the result of the determination in S22 is that the driver's head posture is out of alignment, the process proceeds to S23 because the driver's head posture is out of alignment, which is an abnormal physical state of the driver. If the result of the determination in S22 is that the driver's head posture is not out of alignment, the process returns.

[0054] In S23, it is determined whether the count value of the second counter C2 is greater than 0, that is, whether both of the driver's eyes are closed. If the result of the determination in S23 is that the count value of the second counter C2 is greater than 0, the count values ​​of the first counter C1 and the second counter C2 are not currently 0, so the process proceeds to S24. If the result of the determination in S23 is that the count value of the second counter C2 is 0, the driver's head posture is not out of alignment, so the process proceeds to S27. In S24, the current count value of the third counter C3 is incremented by 1, and then the process proceeds to S25.

[0055] In S25, it is determined whether the count value of the third counter C3 is equal to or greater than the third determination threshold L3. If the result of the determination in S25 is that the count value of the third counter C3 is equal to or greater than the third determination threshold L3, it is determined that the driver is in an abnormal state, and the process proceeds to S26. If the result of the determination in S25 is that the count value of the third counter C3 is less than the third determination threshold L3, it is not possible to determine that the driver is in an abnormal state, and the process returns. In S26, abnormality sequence control is executed, and then the process returns. In this S26, 0 is substituted into each of the first to fourth counters C1 to C4.

[0056] In S27, it is determined whether or not the open / closed state has been detected based on the determined eye opening degree. If the result of the determination in S27 is that the open / closed state has been detected, the driver's eyes are open, so the process proceeds to S28. If the result of the determination in S27 is that the open / closed state has not been detected, the open / closed state of the driver is unknown, so the process proceeds to S34.

[0057] In S28, it is determined whether the driver is operating the various operating parts (steering wheel, accelerator pedal, brake pedal, etc.) appropriately. If the result of the determination in S28 is that the driver is operating the vehicle appropriately, even if the head posture is out of alignment, the process returns. If the result of the determination in S28 is that the driver is not operating the vehicle appropriately, for example, if the various operating units are not being operated at all, the process proceeds to S29.

[0058] In S29, it is determined whether the vehicle speed is equal to or greater than the determination threshold value. If the result of the determination in S29 is that the vehicle speed is equal to or greater than the determination threshold, the driver's posture is unstable and the driver is unable to drive despite having their eyes open, so the process proceeds to S30. If the result of the determination in S29 is that the vehicle speed is less than the determination threshold, the driver may be stopped to wait for recovery, so the process returns. In S30, 1 is added to the current count value of the first counter C1, and then the process proceeds to S31.

[0059] In S31, it is determined whether the count value of the first counter C1 is equal to or greater than the first determination threshold L1 (fourth predetermined time). If the result of the determination in S31 is that the count value of the first counter C1 is equal to or greater than the first determination threshold L1, the onset of a disease is predicted, and the process proceeds to S32. If the result of the determination in S31 is that the count value of the first counter C1 is less than the first determination threshold L1, the onset of a disease cannot currently be predicted, and the process returns.

[0060] In S32, it is determined whether a cancel signal for stopping the alarm has been generated. If the result of the determination in S32 is that a cancel signal has been generated, the driver has performed the operation to generate a cancel signal, so the count value of the first counter C1 is set to 0 (S33) and then the process returns.If the result of the determination in S32 is that a cancel signal has not been generated, the driver is not in a state to perform the operation to generate a cancel signal, so the process proceeds to S26.

[0061] In S34, the current count value of the fourth counter C4 is incremented by 1, and then the process proceeds to S35. In S35, it is determined whether the count value of the fourth counter C4 is equal to or greater than a fourth determination threshold L4 (a second predetermined time). This fourth determination threshold L4 is set to a period shorter than the third determination threshold L3.

[0062] If the result of the determination in S35 is that the count value of the fourth counter C4 is equal to or greater than the fourth determination threshold L4, the onset of a disease or dozing is suspected, and the process proceeds to S36. If the result of the determination in S35 is that the count value of the fourth counter C4 is less than the fourth determination threshold L4, the onset of a disease or drowsiness cannot be estimated, and the process returns.

[0063] In S36, an alarm is issued, and then the process proceeds to S37. In S37, it is determined whether or not the second determination condition is met. The second determination conditions are the following two conditions. (1) The driver's head posture is abnormal. (2) The driver has not operated the vehicle for the second non-operation determination time (third predetermined time). The certain period of time during which the driver does not perform any driving operation is shorter than the period corresponding to the third determination threshold L3.

[0064] If the second determination condition is met in S37, the driver will not start driving even if the warning is issued, so the process proceeds to S38. If the second determination condition is not met in S37, the driver will start driving in response to the warning, so the process returns.

[0065] In S38, it is determined whether a cancel signal for stopping the alarm has been generated. If the result of the determination in S38 is that a cancel signal has been generated, the driver has performed the operation to generate a cancel signal, so the count values ​​of the first and fourth counters C1 and C4 are set to 0 (S39), and then the process returns.If the result of the determination in S38 is that a cancel signal has not been generated, the driver cannot perform the operation to generate a cancel signal, so the process proceeds to S26.

[0066] Next, the operation and effects of the driver state determination method and the device M will be described. According to this driver state determination method, there are provided head posture acquisition processes S1, S21 for acquiring the driver's facial direction and head position, eye closure determination processes S2, S23 for determining whether the driver's eyes are closed, driving operation amount acquisition processes S1, S21 for acquiring at least one of the vehicle driving operations performed by the driver at the current time, and driver abnormal state determination processes S5, S31, S37 for determining whether the driver is in an abnormal state using the abnormality determination unit 33c, so that poor head posture, which is an abnormal state, and closed eyes, which is a symptom of an abnormality, can be used as conditions for determining the driver's abnormal state. In the driver abnormal state determination process by the abnormality determination unit 33c, in a situation where a closed eye state is determined in the closed eye determination process S2, if the driver's head posture acquired in the head posture acquisition process S1 is an abnormal posture and the closed eye state continues for a period corresponding to the third determination threshold L3, it is determined that the driver is in an abnormal state, so the driver's abnormal state can be determined based on the directly detected abnormal symptoms and abnormal state. In the driver abnormal state determination process by the abnormality determination unit 33c, in a situation where it is determined that the eyes are not closed in the eye closure determination process S23, if the driver's head posture acquired in the head posture acquisition process S21 is abnormal and the eyes are not closed for a period corresponding to a fourth determination threshold L4 different from the third determination threshold L3, an alert is issued to the driver.After the alert, if the driver's head posture acquired in the head posture acquisition process S21 is abnormal and the driving operation amount acquired in the driving operation amount acquisition process S21 does not change for a second no-operation determination time, the driver is determined to be in an abnormal state.Therefore, even in a situation where closed eyes, which are an abnormal symptom, cannot be confirmed, an abnormal action can be substituted for the abnormal symptom, and this no driving operation, which is an abnormal action, and poor head posture, which is an abnormal state, can be used as conditions for determining an abnormal state. Furthermore, since the state is confirmed by an alarm before the abnormal state of the driver is determined based on the abnormal state and abnormal action, the accuracy of determining the abnormal state can be increased.

[0067] Since the period corresponding to the fourth determination threshold L4 is shorter than the period corresponding to the third determination threshold L3, it is possible to confirm in the shortest time that the eyes are not closed.

[0068] The driver abnormal state determination steps S5 and S37 determine whether the head posture is abnormal based on at least the face direction angle of the driver, and therefore can easily determine whether the head posture is abnormal.

[0069] In the driver abnormal state determination step S31, when the driver's head posture acquired in the head posture acquisition step S21 is abnormal and the vehicle continues to be in a traveling state for a period corresponding to the first determination threshold L1 in a situation where the driver's eyes are determined to be open in the eye closure determination steps S23 and S27, the driver abnormal state determination step S31 determines that the driver is in an abnormal state. This makes it possible to determine whether the driver has their eyes open based on the driver's state and the behavior of the vehicle.

[0070] This driver condition determination device M detects an abnormal state of the driver of a vehicle and includes a camera 11 that acquires the driver's facial direction and head position, an eye condition determination unit 32 that determines whether the driver's eyes are closed, sensors 12 to 16 that acquire at least one of the vehicle driving operations performed by the driver at the current time, and an abnormality determination unit 33c that determines whether the driver is in an abnormal state.Therefore, the conditions for determining whether the driver is in an abnormal state can be determined to be a collapse of head posture, which is an abnormal state, and closed eyes, which is a symptom of an abnormality. The abnormality determination unit 33c determines that the driver is in an abnormal state if the driver's head posture acquired by sensors 12 to 16 is abnormal and the eye closed state continues for a period corresponding to the third determination threshold L3 in a situation where the eye state determination unit 32 determines that the driver is in an abnormal state, and therefore can determine the driver's abnormal state based on the directly detected abnormal symptoms and abnormal state. The abnormality determination unit 33c issues an alert to the driver when the driver's head posture acquired by the camera 11 is abnormal and the state in which the eyes are not closed continues for a period corresponding to a fourth determination threshold L4 different from the third determination threshold L3 in a situation in which the eye state determination unit 32 determines that the eyes are not closed. After the alert, if the driver's head posture acquired by the camera 11 is abnormal and the state in which the driving operation amount acquired by the sensors 12 to 16 does not change continues for a second no-operation determination time, the abnormality determination unit 33c determines that the driver is in an abnormal state. Therefore, even in a situation in which the eye closure, which is an abnormal symptom, cannot be confirmed, an abnormal operation can be substituted for the abnormal symptom, and this no driving operation, which is an abnormal operation, and a collapsed head posture, which is an abnormal state, can be used as conditions for determining the abnormal state. Moreover, because the state is confirmed by the alert before determining the driver's abnormal state based on the abnormal state and the abnormal operation, the accuracy of determining the abnormal state can be improved.

[0071] In addition, a person skilled in the art can implement the present invention in a form in which various modifications are added to the above-described embodiment without departing from the spirit of the present invention, and the present invention also includes such modifications. [Explanation of symbols]

[0072] 1. Detection Methods 11 Camera 32 Eye condition determination unit 33c Abnormality determination section M Driver status determination device

Claims

1. A driver condition determination method for detecting an abnormal condition of a driver of a vehicle, comprising: a head posture acquiring step of acquiring a face direction and a head position of the driver; an eye closure determination step of determining whether the driver's eyes are closed; a driving operation amount acquisition step of acquiring at least one driving operation amount of the vehicle performed by the driver at a predetermined timing; a driver abnormal state determination step of determining whether the driver is in an abnormal state, The driver abnormal state determination step includes: In a situation where a closed eye state is determined in the closed eye determination step, if the head posture of the driver acquired in the head posture acquisition step is an abnormal posture and the closed eye state continues for a first predetermined time, it is determined that the driver is in an abnormal state, In a situation where it is determined that the eyes are not closed in the eye closure determination step, if the head posture of the driver acquired in the head posture acquisition step is abnormal and the state in which the eyes are not closed continues for a second predetermined time that is shorter than the first predetermined time, the driver is notified, and if after the notification, the state in which the head posture of the driver acquired in the head posture acquisition step is abnormal and the driving operation amount acquired in the driving operation amount acquisition step does not change continues for a third predetermined time, the driver is determined to be in an abnormal state. A driver state determination method comprising:

2. The driver state determination method described in Claim 1, characterized in that the driver abnormal state determination process determines that the head posture is an abnormal posture based at least on the driver's facial angle.

3. The driver abnormal state determination process includes: A driver state determination method as described in claim 1 or 2, characterized in that, in a situation where an open-eye state is determined in the closed-eye determination process, if the driver's head posture acquired in the head posture acquisition process is an abnormal posture and the vehicle continues to be in a traveling state for a fourth predetermined time, the driver is determined to be in an abnormal state.

4. A driver condition determination device for detecting an abnormal condition of a vehicle driver, a head posture acquisition means for acquiring a face direction and a head position of the driver; eye closure determination means for determining whether the driver's eyes are closed; a driving operation amount acquisition means for acquiring at least one driving operation amount of the vehicle performed by the driver at a predetermined timing; a driver abnormal state determination means for determining whether the driver is in an abnormal state; The driver abnormal state determination means In a situation where the eye closure determination means determines that the driver has an eye closed state, if the head posture of the driver acquired by the head posture acquisition means is an abnormal posture and the eye closed state continues for a first predetermined time, the driver is determined to be in an abnormal state, A driver state determination device characterized by the fact that, in a situation where the eye closure determination means determines that the driver is not in a closed state, if the driver's head posture acquired by the head posture acquisition means is abnormal and the driver's eyes are not closed, and this state continues for a second predetermined time that is shorter than the first predetermined time, the device notifies the driver, and if, after the notification, the driver's head posture acquired by the head posture acquisition means is abnormal and the driving operation amount acquired by the driving operation amount acquisition means does not change, the device determines that the driver is in an abnormal state.

Images