Driver state determination method and device

Abstract

To provide a driver state determination method and a device that can determine an abnormal state of a driver with high accuracy.SOLUTION: A driver state determination method includes an open or closed eye determination step S2, S3 of detecting an open or closed state of both left and right eyes of a driver, a driving operation amount acquisition step S1 of acquiring at least one of amounts of driving operation for an operation device of a vehicle performed by the driver at predetermined timing, and a driver abnormal state determination step S5, S10, S11 of determining whether the driver is in an abnormal state. The driver abnormal state determination step S10, S11 determines that the driver is in the abnormal state when both the left and right eyes of the driver are in a closed eye state and the driver's non-driving operation state continues for a first predetermined time T1, and determines that the driver is in the abnormal state when one eye of the driver is in the closed eye state and specific operation to an extent not hindering the driving is not detected within a second predetermined time T2 despite a request for the specific operation to the driver.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a driver condition determination method and device for detecting an abnormal condition of a driver who drives a vehicle. [Background technology]

[0002] BACKGROUND ART Driver support devices have been known that accurately grasp a driver's state of consciousness, particularly the driver's state of alertness, and support the driver in driving a vehicle in accordance with the driver's state of alertness. An abnormal state of a driver, including the level of drowsiness and a state of poor alertness, is usually determined based on head-related information such as the driver's head posture and facial state acquired by an imaging means such as a camera. Among the head-related information, the head posture is detected, for example, via the driver's facial direction and head position (three-dimensional coordinates), and the facial state is detected, for example, via the driver's facial expression and eye opening state (eyelid state).

[0003] The driver monitoring device of Patent Document 1 has an image acquisition means for acquiring an image of the driver, a recognition stability determination means for determining the recognition stability using the image acquisition means, an attention calling means for calling the driver's attention, and an action request means for requesting the driver to perform a predetermined action.The recognition stability determination means activates the attention calling means when the recognition stability is high and the degree of eye opening is low, and requests the driver to perform a non-image action to detect actions without using image processing when the recognition stability is low, and determines the driver's level of alertness based on the presence or absence of this non-image action. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-095162 Summary of the Invention [Problem to be solved by the invention]

[0005] The driver monitoring device of Patent Document 1 is capable of determining an abnormal state (poor alertness state) of the driver based on non-imaging actions by the driver when imaging conditions such as the in-vehicle imaging environment in which the driver is imaged or the operational performance of the imaging image acquisition means are not met. However, the driver monitoring device of Patent Document 1 may not be able to determine whether the driver is in an abnormal state even when the imaging conditions are met.

[0006] The technology of Patent Document 1 determines whether the driver is in an abnormal state based on the degree to which the driver's eyes are open when the imaging conditions for imaging the driver are met. Therefore, if the image acquisition means captures an image of the driver with one eye closed, the other eye is determined to be open because it is open. However, if a driver develops facial nerve paralysis such as blepharospasm due to a brain disease (such as cerebral infarction or stroke), a situation may occur in which one eye is open and the other eye is closed.

[0007] On the other hand, when the light ahead is high, such as during the day in summer, or when dust or other debris gets into the driver's eyes, the driver may close only one eye even if they are fully awake. Therefore, even if the driver's one eye closed state is accurately detected after being imaged by an imaging device, the acquisition of this detected one eye closed state cannot be unambiguously regarded as a symptom of the driver's illness. In other words, it is not easy to accurately determine the driver's abnormal state.

[0008] An object of the present invention is to provide a driver condition determination method and device therefor that can determine an abnormal condition of a driver with high accuracy. [Means for solving the problem]

[0009] The driver state determination method of claim 1 is a driver state determination method for detecting an abnormal state of a driver of a vehicle, comprising: an eye open / closed state determination step of detecting an open / closed state of both the left and right eyes of the driver; a driving operation amount acquisition step of acquiring at least one of an amount of driving operation performed by the driver on an operating device of the vehicle at a predetermined timing; and a driver abnormal state determination step of determining whether the driver is in an abnormal state or not, wherein the driver abnormal state determination step determines that the driver is in an abnormal state when both the left and right eyes of the driver are closed and the driver has not performed any driving operation for a first predetermined time, and determines that one eye of the driver is closed and does not hinder driving for the driver. Special If a specific operation is requested but the specific operation is not detected within a second predetermined time, the driver is determined to be in an abnormal state.

[0010] This driver state determination method includes an eye open / closed determination process for detecting the open / closed state of both the driver's left and right eyes, a driving operation amount acquisition process for acquiring at least one of the driving operation amounts of the driver on the vehicle's operating devices at a predetermined timing, and a driver abnormal state determination process for determining whether the driver is in an abnormal state or not.Therefore, closed eyes, which are a driver's symptom in an abnormal state (hereinafter referred to as an abnormal state symptom), and no driving operation, which is the driver's behavior in an abnormal state (hereinafter referred to as an abnormal state behavior), can be used as conditions for determining the driver's abnormal state. The driver abnormal state determination step determines that the driver is in an abnormal state when both the left and right eyes of the driver are closed and the driver has not operated the vehicle for a first predetermined time, so that the abnormal state of the driver can be determined with high accuracy based on two abnormality determination conditions, namely, abnormality symptoms and abnormality actions. Special If a specific operation is requested but the specific operation is not detected within a second predetermined time, the driver is determined to be in an abnormal state, so that the abnormal state of the driver with one eye closed can be reliably determined based on the driver's actual operation without interfering with the driving of the vehicle.

[0011] The invention of claim 2 is characterized in that, in the invention of claim 1, the driver abnormal state determination process issues an alert when both the driver's left and right eyes are closed and when one eye is closed. According to this configuration, it is possible to more reliably determine whether the driver is in an abnormal state when both the left and right eyes are not open.

[0012] The invention of claim 3 is characterized in that, in the invention of claim 1 or 2, it has a head posture acquisition process for acquiring the driver's facial direction and head position, and the driver abnormal state determination process determines that the driver is in an abnormal state if the head posture of the driver acquired in the head posture acquisition process is an abnormal posture and one of the driver's eyes is closed, and the driver's state of no operation of the vehicle continues for a third predetermined time that is shorter than the first predetermined time. According to this configuration, it is possible to reliably and quickly determine whether the driver is in an abnormal state using the abnormal posture as a parameter.

[0013] A driver condition determination device according to claim 4 is a driver condition determination device for detecting an abnormal state of a driver of a vehicle, and comprises an eye open / closed state determination means for detecting an open / closed state of both the left and right eyes of the driver, a driving operation amount acquisition means for acquiring at least one of the amount of driving operation performed by the driver on an operating device of the vehicle at a predetermined timing, and a driver abnormal condition determination means for determining whether the driver is in an abnormal state or not, wherein the driver abnormal condition determination means determines that the driver is in an abnormal state when both the left and right eyes of the driver are closed and the driver has not performed any driving operation for a first predetermined time, and determines that one eye of the driver is closed and does not hinder driving for the driver. Special If a specific operation is requested but the specific operation is not detected within a second predetermined time, the driver is determined to be in an abnormal state.

[0014] This driver condition determination device has an eye open / closed determination means for detecting the open / closed state of the driver's left and right eyes, a driving operation amount acquisition means for acquiring at least one of the driving operation amounts of the driver on the vehicle's operating devices at a predetermined timing, and a driver abnormal condition determination means for determining whether the driver is in an abnormal condition, so that closed eyes, which are an abnormal symptom, and no driving operation, which is an abnormal behavior, can be used as conditions for determining whether the driver is in an abnormal condition. The driver abnormal state determination means determines that the driver is in an abnormal state when both the left and right eyes of the driver are closed and the driver has not operated the vehicle for a first predetermined time, so that the abnormal state of the driver can be determined with high accuracy based on two abnormality determination conditions, namely, abnormality symptoms and abnormality actions. Special If a specific operation is requested but the specific operation is not detected within a second predetermined time, the driver is determined to be in an abnormal state, so that the abnormal state of the driver with one eye closed can be reliably determined without interfering with the driving of the vehicle. [Effects of the Invention]

[0015] According to the driver condition determination method and device of the present invention, the abnormal condition of the driver can be determined with high accuracy by determining the disease symptoms of the driver when one eye is closed. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a block diagram of a driver state determination device according to a first embodiment. [Figure 2] FIG. 1 is an explanatory diagram of a monitoring support device. [Figure 3] FIG. 2 is an explanatory diagram of a braking assist device. [Figure 4] FIG. 2 is an explanatory diagram of a vehicle stopping assistance device. [Figure 5] 10 is a flowchart showing a control process for determining whether the eyes are closed. [Figure 6] 10 is a flowchart showing a head posture determination control process. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. The following description exemplifies the application of the present invention to a driver state determination device for a vehicle, and does not limit the present invention, its applications, or its uses. [Example]

[0018] A first embodiment of the present invention will be described below with reference to FIGS. The driver state determination device M is a driving assistance device that determines the driver's state of consciousness, particularly an abnormal state that makes it difficult for the driver to operate the vehicle (hereinafter also referred to as the driver's own vehicle), and is capable of autonomously making an emergency stop if an abnormal state of the driver is determined. Hereinafter, the abnormal state of the driver will be explained as a state of poor alertness (including the onset of illness or drowsiness) in which the driver is not consciously awake and is physically unable to drive due to poor posture. The following description also includes a description of a driver state determination method.

[0019] As shown in FIG. 1, the driver state determination device M mainly comprises a plurality of detection means 1 capable of detecting the driver's operating behavior and the vehicle's driving behavior, a driver assistance device 2 that autonomously provides vehicle driving assistance to the driver from the vehicle side, and an ECU (Electronic Control Unit) 3 that performs various calculations based on input signals input from the detection means 1 and outputs various command signals to the driver assistance device 2.

[0020] First, the detection means 1 will be described. 1, the detection means 1 includes a camera 11 that captures an image of the driver, a vehicle speed sensor 12 that detects the speed of the vehicle, a steering angle sensor 13 that detects the steering angle caused by operation of the steering wheel (not shown), a grip sensor 14 that detects the driver's grip on the steering wheel, an accelerator sensor 15 that detects the amount of depression of the accelerator pedal (not shown), a brake sensor 16 that detects the amount of depression of the brake pedal (not shown), and a hazard switch 17 that activates hazard lamps (not shown). Each of the sensors 12 to 16 corresponds to a driving operation amount acquisition means that acquires the driving operation amount at a predetermined timing, i.e., the current time immediately before the judgment time.

[0021] The camera 11 is configured by, for example, a CCD (Charge Coupled Device) camera, and is mounted in a position facing the driver at the center of the upper part of an instrument panel (not shown). This camera 11 is configured to be able to capture an image of the head, including the face, of a driver seated in the driver's seat and fastening a seat belt from the front. The captured head image is configured to be able to extract an image of the driver's face and detect the direction of the face (direction of face orientation), identify the driver's facial expression, enlarge and detect the iris to determine the direction of the line of sight, enlarge and detect the eyelids to determine the degree of eye opening and closing, and the three-dimensional coordinate position of the driver's head (center of gravity). The camera 11 is required to be capable of acquiring at least the necessary captured images, and may be equipped with two cameras, a front-facing camera and an upward camera installed directly above the driver's seat, or three or more cameras.

[0022] Next, the driver assistance device 2 will be described. As shown in FIG. 1, the driver assistance device 2 is composed of a monitoring assistance device 21 that notifies the driver of an alert state, a braking assistance device 22 that causes the vehicle to perform a braking operation, and a stopping assistance device 23 that causes the vehicle to perform an emergency stop operation.

[0023] The monitoring support device 21 is configured to recognize the state of consciousness, that is, the wakefulness state of the driver, and, if it determines that recovery is necessary, to notify the driver and encourage the driver to recover consciousness. When the monitoring support device 21 receives an inattentive driving signal or a drowsiness estimation signal from the ECU 3, it issues a warning to the driver from one or more speakers (not shown) provided in the vehicle cabin. 2, a speedometer unit 21a disposed on an instrument panel is provided with a display unit 21b capable of displaying a warning. When the monitoring support device 21 receives an inattentive driving signal or a drowsiness estimation signal from the ECU 3, the monitoring support device 21 displays a warning such as "Watch out for the road ahead" on the display unit 21b.

[0024] The braking assist device 22 is configured to issue a warning when the vehicle approaches another vehicle ahead while traveling, and to apply emergency braking when the braking limit distance at which a collision can be avoided is reached. As shown in FIG. 3, when the host vehicle approaches the leading vehicle B to the position of the host vehicle A1, the braking assist device 22 issues a warning to the driver from the speaker. The separation distance between the other vehicle B and the host vehicle A1 is a distance corresponding to the time (for example, 0.8 seconds) that the driver can normally avoid a collision by his / her own driving operation (manual operation). The braking assist device 22 automatically applies emergency braking when the host vehicle approaches the leading vehicle B up to the position of the host vehicle A2. The separation distance between the leading vehicle B and the host vehicle A2 is the limit distance at which a collision can be avoided by applying emergency braking equivalent to the maximum braking force.

[0025] 3, when the braking assist device 22 receives an inattentive driving signal or a drowsiness estimation signal from the ECU 3, the braking assist device 22 issues an alarm to the driver from the speaker when the host vehicle approaches a preceding vehicle B to a position of the host vehicle A3 that is farther away from the preceding vehicle B than the position of the host vehicle A1. Here, the separation distance between the other vehicle B and the host vehicle A3 is a distance corresponding to the time (e.g., 1.2 seconds) that the driver can avoid a collision by driving the vehicle himself even if he is distracted.

[0026] When the stopping assistance device 23 receives a drowsiness estimation signal from the ECU 3, it issues an alarm to the driver from a speaker, and after receiving a drowsiness confirmation signal, it autonomously brings the vehicle to an emergency stop. 4, when the stopping assistance device 23 receives a drowsiness estimation signal corresponding to a state in which both eyes are closed at the position of the vehicle X1, it activates the speaker to issue an alarm, and receives a drowsiness confirmation signal corresponding to a state in which the driver is not operating the vehicle at the position of the vehicle X2. Hereinafter, unless otherwise specified, when both eyes are closed, it is simply referred to as the eyes closed state, and when only one eye is closed, it is referred to as the one eye closed state. Next, for host vehicle X3, autonomous driving begins even if the vehicle has been manually driven by the driver up until then. After autonomous driving begins, the host vehicle moves to a safe area such as the shoulder of the road and then makes an autonomous emergency stop at the position of host vehicle X4.

[0027] Furthermore, when the vehicle stopping assistance device 23 receives an abnormality signal from the ECU 3, it autonomously brings the host vehicle to an emergency stop. As shown in FIG. 4, the vehicle stopping assistance device 23 receives an abnormality signal from the driver at the position of the host vehicle X2. Next, at the host vehicle X3, automatic driving is initiated even if the host vehicle has been manually driven by the driver up until that point. After starting automatic driving, the host vehicle moves to a safety zone such as the shoulder of the road and then makes an autonomous emergency stop at the position of the host vehicle X4. Thereafter, the vehicle stopping assistance device 23 makes an emergency call to an emergency center or the like, for example, after a predetermined vehicle stop maintenance period has elapsed. Hereinafter, this autonomous emergency stop control will be referred to as abnormality sequence control.

[0028] Next, the ECU 3 will be described. The ECU 3 makes a judgment based on the driver's symptoms at the time of an abnormality (hereinafter referred to as "abnormal symptoms"), the driver's physical condition at the time of an abnormality (hereinafter referred to as "abnormal condition"), and the driver's behavior at the time of an abnormality (hereinafter referred to as "abnormal behavior"). The ECU 3 is composed of a CPU (Central Processing Unit), ROM, RAM, an internal interface, an external interface, etc. The ROM stores various programs and data for brake control, display control, etc., and the RAM has a processing area that the CPU uses when performing a series of processes.

[0029] As shown in Figure 1, the ECU 3 includes a head posture determination unit 31 that determines the head posture state of the driver, an eye state determination unit 32 (eye open / close determination means) that determines the open / close state of each of the driver's left and right eyes based on the degree of eye opening, and a driver state determination unit 33 that determines the driver state that may be an obstacle to driving the vehicle.

[0030] Head posture determination unit 31 determines the head posture state of the driver based on the captured image information captured by camera 11. This head posture determination unit 31 determines that the head posture state of the driver is out of alignment when it detects that the head posture state of the driver is either head down, lying to the side, leaning face down, or leaning backward. When it is determined that the head posture is out of alignment, the count value of the first counter C1 is set to 1. The initial value of the first counter C1 is 0.

[0031] For example, the head posture determination unit 31 outputs a head-down determination signal when the absolute value of the face direction angle in the pitch direction is equal to or greater than a predetermined threshold, and outputs a sideways (leaning) determination signal when the absolute value of the face direction angle in the roll direction is equal to or greater than the predetermined threshold. Furthermore, the head posture determination unit 31 outputs a head-down determination signal when the absolute value of the face direction angle in the pitch direction is equal to or greater than the predetermined threshold and the current head position is equal to or greater than a value obtained by subtracting a head position threshold from a reference head position, and outputs a backward tilt determination signal when the face direction angle in the pitch direction is equal to or greater than the predetermined threshold and the current head position is smaller than a value obtained by adding the head position threshold to the reference head position. The thresholds associated with the angles and head positions used to determine head-down, sideways, leaning, and backward tilt correspond to head posture determination thresholds used to determine abnormal head posture of the driver.

[0032] The eye state determination unit 32 determines whether both the left and right eyes of the driver are closed, and whether only one of the left and right eyes is closed. The eye state determination unit 32 detects the upper and lower eyelids of the driver captured by the camera 11, and detects the degree of opening of each of the driver's eyes based on the number of pixels between the edges of the eyelids. When the detected degree of opening of each of the left and right eyes is equal to or greater than a predetermined judgment threshold, it is determined that the eyes are open, and when the detected degree of opening of each of the left and right eyes is less than the predetermined judgment threshold, it is determined that the eyes are closed.

[0033] Furthermore, the eye state determination unit 32 determines that one eye is closed when the degree of opening of one eye is equal to or greater than a predetermined determination threshold and the degree of opening of the other eye is less than the predetermined determination threshold. When it is determined that both the left and right eyes of the driver are closed, the count value of the second counter C2 is set to 1. The initial value of the second counter C2 is 0.

[0034] The driver state determination unit 33 includes an inattentiveness determination unit 33a that determines whether the driver is looking in a direction other than the direction ahead in the direction of travel, a drowsiness determination unit 33b that determines a drowsiness estimation state in which the driver is presumed to be drowsy and a drowsy confirmation state in which the driver is detected to be drowsy with a high probability, and an abnormality determination unit 33c (driver abnormal state determination means) that determines an abnormal state of the driver in which it is difficult to drive the vehicle.

[0035] The inattentive driving determining unit 33a will be described. The inattentiveness determination unit 33a determines whether the driver is looking away from the driver based on the driver's face direction information, gaze direction information, and vehicle information, and if it is determined that the driver is looking away, it outputs an inattentiveness signal to the monitoring support device 21 and the braking support device 22. When the count value of the driver's gaze direction in the yaw direction falls outside the range of the yaw direction threshold, it is determined that the driver is looking aside. The yaw direction threshold is changed according to the vehicle speed or the radius of curvature when the vehicle is traveling. Also, when the count value of the driver's gaze direction in the pitch direction falls outside the range of the pitch direction threshold, it is determined that the driver is looking aside. The pitch direction threshold is changed according to the vehicle speed.

[0036] The count value of the yaw direction gaze direction and the count value of the pitch direction gaze direction are calculated by a long-term continuous inattentive count or a short-term cumulative inattentive count. The long-term continuous inattentive count is the number of times that the determination results for each processing cycle continue in chronological order. The short-term cumulative inattentive count is the cumulative count of the determination results for each processing cycle when the conditions are met within a predetermined period of time prior to that.

[0037] The reliability of the gaze direction information in the yaw and pitch directions is determined using a preset reliability threshold. The reliability of each gaze direction information is set based on the clarity of the captured image of the driver (clarity of the contours of the face, eyelids, pupils, etc.). If the reliability of each gaze direction information is low and each gaze direction is invalid, it is interpolated using face direction information in the yaw and pitch directions.If the reliability of face direction information in the yaw and pitch directions is low and each face direction is invalid, it is interpolated using the previous valid gaze direction.

[0038] Next, the drowsiness determination unit 33b will be described. The drowsiness determination unit 33b determines whether the driver is dozing or not based on the duration of eye closure and vehicle information, and if it is estimated that the driver is dozing, it outputs a drowsiness estimation signal to the monitoring support device 21, the braking support device 22, and the stopping support device 23. If it is determined that the driver is dozing, it outputs a drowsiness confirmation signal to the stopping support device 23. Note that this drowsiness determination is made using the eye closure state in which both eyes are closed.

[0039] The drowsiness determining unit 33b estimates a drowsy state when the driver's continuous eye-closure time exceeds a predetermined time threshold, or when the driver repeatedly closes his / her eyes for short periods exceeding the predetermined time threshold within a certain period of time. In addition, the drowsiness determination unit 33b determines the drowsy state if the driver's continuous eye closure time exceeds a predetermined time threshold, or if the continuous eye closure and no driving operation state exceeds a time threshold, while the vehicle is traveling at a certain speed or above after a drowsiness warning.

[0040] The non-operation state of the driving is determined as an inappropriate driving operation while driving by comparing the detection signal of the detection means 1 with a judgment threshold. Specifically, it is determined by at least one of the following operating states: non-operation of the steering wheel, non-grasping of the steering wheel, and non-operation of the accelerator or brake. Note that an extremely small judgment threshold may be set for each operating member to determine an inappropriate driving operation.

[0041] Next, the abnormality determination unit 33c will be described. The abnormality determination unit 33c determines that an abnormal state has occurred in which it is difficult for the driver to operate the vehicle when the driver has closed eyes and poor head posture, or when the driver has poor head posture and is not operating the vehicle.

[0042] The inventors have found that the driver's symptoms when an abnormality occurs can be detected using closed eyes as a parameter, the driver's physical state when an abnormality occurs can be detected using poor head posture as a parameter, and the driver's behavior when an abnormality occurs can be detected using non-operation of the driving force as a parameter. Based on this finding, in this embodiment, the detection of closed eyes and poor head posture, and the detection of poor head posture and non-operation of the driving force are used as judgment conditions.

[0043] In addition, the abnormality determination unit 33c determines that the driver is in an abnormal state when both the left and right eyes are closed and the driving non-operation state continues for a first predetermined time T1, and determines that one eye is closed and does not hinder the driver from driving. Special If the execution of a specific operation is not detected within a second predetermined time T2 even when the execution of the specific operation is requested, it is determined that the driver is in an abnormal state. This makes it possible to determine whether the driver has an abnormal condition when one eye is closed without interfering with the driving of the vehicle.

[0044] Here, do not disturb driving. Special The constant operation is, for example, a steering angle operation equivalent to the play of the steering wheel (approximately ±5°) or a brake pedal depression operation equivalent to 0.1 G. In addition, such specific operations are requested of the driver by the vehicle side, for example, by voice instructions through the speaker or by display on the display unit 21b, such as "Please press the hazard switch 17" or "Please turn the steering wheel slightly to the right."

[0045] Next, the eye closure determination control process will be described with reference to the flowchart of FIG. Incidentally, Si (i=1, 2, . . . ) indicates a step for each process.

[0046] First, as shown in FIG. 5, the ECU 3 receives input of outputs from various detection means 1 such as the camera 11 and various information such as the second counter C2 (S1), and then proceeds to S2. In S2, it is determined whether or not at least one eye of the driver is closed. If the result of the determination in S2 is that at least one eye of the driver is closed, the driver has closed his / her eye for some reason, so the process moves to S3. If the result of the determination in S2 is that at least one eye of the driver is not closed, both eyes are open, so the process returns.

[0047] In S3, it is determined whether the driver has one eye closed. If the result of the determination in S3 is that the driver has one eye closed, it is not possible to distinguish whether the one eye closed state is in a good wakefulness state or a bad wakefulness state, and therefore the process proceeds to S4. If the result of the determination in S3 is that the driver does not have one eye closed, both eyes are closed, and the process proceeds to S7.

[0048] In S4, the vehicle requests the driver to perform a specific operation that does not interfere with driving, and then the process proceeds to S5. At the same time as the request to perform the specific operation in S4, a timer (not shown) is started. In S5, it is determined whether or not the specific operation is performed by the driver within a second predetermined time T2 after the request to perform the specific operation is made.

[0049] If the result of the determination in S5 is that the driver has performed the specific operation within the second predetermined time T2 after the request to perform the specific operation, the driver's consciousness is in a well-awake state, so the timer is reset and the process returns. If the result of the determination in S5 is that the driver has not performed the specific operation within the second predetermined time T2 after the request to perform the specific operation, the driver's consciousness is in a poorly-awake state, so the timer is reset and the process proceeds to S6. In S6, abnormality sequence control is executed, and then the process returns. In this S6, 0 is substituted for the count values ​​of the first and second counters C1 and C2.

[0050] In S7, the current count value of the second counter C2 is incremented by 1, and then the process proceeds to S8. In S8, it is determined whether the count value of the second counter C2 is equal to or greater than the second determination threshold L2. If the result of the determination in S8 is that the count value of the second counter C2 is equal to or greater than the second determination threshold L2, both eyes are closed, and it is estimated that the patient is suffering from an illness or is drowsy, so the process proceeds to S9. If the result of the determination in S8 is that the count value of the second counter C2 is less than the second determination threshold L2, the onset of a disease or drowsiness cannot be estimated at present, and therefore the process returns. In S9, an alarm is issued, and then the process proceeds to S10.

[0051] In S10, it is determined whether or not the first determination condition is met. The first determination conditions are the following three conditions. (1) The driver's head posture is abnormal. (2) The driver has not operated the vehicle for a first predetermined time T1. (3) The driver's eyes are closed for a certain period of time. When all three of these conditions are met, the first judgment condition is met.

[0052] If the result of the determination in S10 is that the first determination condition is met, the driver will not start driving even if an alarm is issued, so the process proceeds to S11. If the result of the determination in S10 is that the first determination condition is not met, any of the conditions (1) to (3) is not met, and it is not possible to determine whether an abnormal state exists, so the process returns.

[0053] In S11, it is determined whether or not a cancel signal for stopping the alarm has been generated. If the result of the determination in S11 is that a cancel signal has been generated, the driver has performed an operation to generate a cancel signal, so the count values ​​of the first and second counters C1 and C2 are each assigned 0 (S12), and then the process returns.If the result of the determination in S11 is that a cancel signal has not been generated, the driver cannot perform an operation to generate a cancel signal for the alarm, so the process proceeds to S6. In this embodiment, a cancel signal is output when the hazard switch 17 is turned on. The output of the cancel signal stops the alarm and stops the abnormal state determination.

[0054] Next, the head posture determination control process will be described with reference to the flowchart of Fig. 6. This head posture determination control process is controlled independently of and in parallel with the eye closure determination control process. Si (i=21, 22...) indicates the step for each process.

[0055] First, as shown in FIG. 6, the ECU 3 receives an input of an output from the detection means 1 such as the camera 11 and the hazard switch 17 (S21), and then proceeds to S22. In S22, it is determined whether the driver's head posture is out of alignment. If the result of the determination in S22 is that the driver's head posture is out of alignment, the state is abnormal, and the process proceeds to S23. If the result of the determination in S22 is that the driver's head posture is not out of alignment, the state is not abnormal, and the process returns.

[0056] In S23, it is determined whether the driving operation is appropriate. If the determination in S23 indicates that the driving operation is appropriate, the process returns because the driver is intentionally driving with a poor posture. If the determination in S23 indicates that the driving operation is inappropriate (no driving operation), the process proceeds to S24. The driving operation is determined by detecting at least one of the steering wheel, accelerator, and brake. In this embodiment, the determination is made using the operation of the steering wheel.

[0057] In S24, it is determined whether the vehicle is traveling at a speed equal to or greater than a predetermined determination threshold. If the result of the determination in S24 is that the vehicle speed is equal to or greater than the determination threshold, the vehicle is moving but there is no driving operation by the driver, so the current count value of the first counter C1 is incremented by 1 (S25) and the process proceeds to S26. If the result of the determination in S24 is that the vehicle speed is less than the determination threshold, the vehicle is already stopped or moving slowly, so the process returns.

[0058] In S26, it is determined whether the count value of the first counter C1 is equal to or greater than the first determination threshold value L1. If the result of the determination in S26 is that the count value of the first counter C1 is equal to or greater than the first determination threshold L1, an abnormal state is estimated regardless of the state of the driver's eyes, and the process proceeds to S27. If the result of the determination in S26 is that the count value of the first counter C1 is less than the first determination threshold L1, an abnormal state cannot be estimated under the current circumstances, so the process returns.

[0059] In S27, an alarm is generated, and then the process proceeds to S28. In S28, it is determined whether a cancel signal for stopping the alarm has been generated. If the result of the determination in S28 is that a cancel signal has been generated, the driver has performed an operation to generate a cancel signal, so the count values ​​of the first and second counters C1 and C2 are each assigned 0 (S29), and then the process returns. If the result of the determination in S28 is that a cancel signal has not been generated, the driver cannot perform an operation to generate a cancel signal using the hazard switch 17, so the process proceeds to S30. In S30, abnormality sequence control is executed, and then the process returns. In this S30, the count values ​​of the first and second counters C1 and C2 are each assigned 0.

[0060] Next, the operation and effects of the driver state determination method and the device M will be described. This driver state determination method includes open / closed eye determination processes S2, S3 for detecting the open / closed state of the driver's left and right eyes, a driving operation amount acquisition process S1 for acquiring at least one of the driving operation amounts of the driver on the vehicle's operating devices at a predetermined timing, and driver abnormal state determination processes S5, S10, S11 for determining whether the driver is in an abnormal state or not, so that closed eyes, which are an abnormal symptom, and no driving operation, which is an abnormal behavior, can be used as conditions for determining whether the driver is in an abnormal state. The driver abnormal state determination steps S10 and S11 determine that the driver is in an abnormal state when both the left and right eyes of the driver are closed and the driver has not operated the vehicle for a first predetermined time T1, so that the driver's abnormal state can be determined with high accuracy based on two abnormality determination conditions, abnormality symptoms and abnormality actions. The driver abnormal state determination step S5 determines that one eye of the driver is closed and does not hinder the driver from driving. Special If a specific operation is requested but no specific operation is detected within the second predetermined time T2, the driver is determined to be in an abnormal state, so that the abnormal state of the driver with one eye closed can be reliably determined based on the driver's actual operation without interfering with the driving of the vehicle.

[0061] The driver abnormality determination processes S4 and S27 alert when both the driver's eyes are closed or when one eye is closed, so that the driver's abnormal state can be determined more reliably in situations where both the driver's eyes are not open.

[0062] This driver state determination device M has an eye state determination unit 32 that detects the open / closed state of the driver's left and right eyes, a detection means 1 that acquires at least one of the driving operation amounts of the driver on the vehicle's operating devices at a predetermined timing, and an abnormality determination unit 33c that determines whether the driver is in an abnormal state or not, so that closed eyes, which are signs of an abnormality, and no driving operation, which is an abnormal behavior, can be used as conditions for determining whether the driver is in an abnormal state. The abnormality determination unit 33c determines that the driver is in an abnormal state when both the left and right eyes of the driver are closed and the driver has not operated the vehicle for a first predetermined time T1, so that the abnormal state of the driver can be determined with high accuracy based on two abnormality determination conditions, namely, abnormality symptoms and abnormality actions. Special If a specific operation is requested but no specific operation is detected within the second predetermined time T2, the driver is determined to be in an abnormal state, so that the abnormal state of the driver with one eye closed can be reliably determined based on the driver's actual operation without interfering with the driving of the vehicle.

[0063] Next, a modified example in which the above embodiment is partially modified will be described. 1) In the above embodiment, an example has been described in which the driver is determined to be in an abnormal state if a specific operation is not detected within the second predetermined time T2 when one eye of the driver is closed. However, in addition to the above judgment conditions, a judgment condition may be added in which the driver is determined to be in an abnormal state if the driver's head posture captured by the camera 11 is abnormal, one eye of the driver is closed, and the driver's no operation state continues for a third predetermined time T3 that is shorter than the first predetermined time T1. This makes it possible to reliably and quickly determine whether the driver is in an abnormal state using the abnormal posture as a parameter.

[0064] 2) In addition, a person skilled in the art may implement the present invention in a form in which various modifications are added to the above-described embodiment without departing from the spirit of the present invention, and the present invention also includes such modifications. [Explanation of symbols]

[0065] 1. Detection Methods 11 Camera 32 Eye condition determination unit 33c Abnormality judgment section M Driver status determination device

Claims

1. A driver condition determination method for detecting an abnormal condition of a driver of a vehicle, comprising: an eye open / closed state determination step of detecting an open / closed state of both the left and right eyes of the driver; a driving operation amount acquisition step of acquiring at least one driving operation amount of an operating device of the vehicle by the driver at a predetermined timing; a driver abnormal state determination step of determining whether the driver is in an abnormal state, The driver abnormal state determination step includes: When both the left and right eyes of the driver are closed and the driver does not perform any driving operation for a first predetermined time, the driver is determined to be in an abnormal state; When one eye of the driver is closed and a specific operation that does not impede driving is requested of the driver, if the specific operation is not detected within a second predetermined time, the driver is determined to be in an abnormal state. A driver state determination method comprising:

2. 2. The method for determining a driver's condition according to claim 1, wherein the driver's abnormal condition determination step includes issuing a notification when both the left and right eyes of the driver are closed and when one eye of the driver is closed.

3. a head posture acquiring step of acquiring a face direction and a head position of the driver, The driver abnormal state determination step includes: A driver state determination method as described in claim 1 or 2, characterized in that if the head posture of the driver acquired in the head posture acquisition process is an abnormal posture, one of the driver's eyes is closed, and the driver's state of not operating the vehicle continues for a third predetermined time that is shorter than the first predetermined time, the driver is determined to be in an abnormal state.

4. A driver condition determination device for detecting an abnormal condition of a vehicle driver, an eye open / closed state determination means for detecting whether the driver's left and right eyes are open or closed; a driving operation amount acquisition means for acquiring at least one driving operation amount of an operating device of the vehicle by the driver at a predetermined timing; a driver abnormal state determination means for determining whether the driver is in an abnormal state; The driver abnormal state determination means When both the left and right eyes of the driver are closed and the driver does not perform any driving operation for a first predetermined time, the driver is determined to be in an abnormal state; A driver condition determination device characterized by determining that the driver is in an abnormal state when one eye of the driver is closed and the driver is requested to perform a specific operation that does not interfere with driving, but the specific operation is not detected within a second predetermined time.

Images