Condition determination device, condition determination program and computer-readable non-perishable storage medium

Abstract

A condition determining device for determining that a driver driving a vehicle is in a driving difficulty state, the device comprising: a driver abnormality detection unit (71) that detects an abnormality in a state of the driver, an operation abnormality detection unit (73) that detects an abnormality in a driving operation input by the driver, a traveling abnormality detection unit (74) that detects an abnormality in a traveling state of the vehicle, and an integrated determination unit (75) that, in response to only the driver abnormality detection unit detecting the abnormality in the driver's state alone, does not determine that the driver is in the driving difficulty state, but determines that the driver is in the driving difficulty state in response to: the driver abnormality detection unit detecting the abnormality in the driver's state, thereafter the operation abnormality detection unit detecting an abnormal operation, and thereafter the traveling abnormality detection unit detecting an abnormal traveling.

Description

Cross-reference to related applications

[0001] This application is based on Japanese Patent Application No. 2018-56139 filed on March 23, 2018, the description of which is incorporated herein by reference. Technical field

[0002] The present disclosure relates to a condition determination technology for determining a driving difficulty state of a driver. State of the art

[0003] For example, JP 5 919 150 B2 discloses a driving assistance device that performs driving assistance, such as automatic deceleration or automatic stop of a vehicle, in response to a driver entering a driving difficulty state. In JP 5 919 150 B2, for the driver's state, a driving posture, a heartbeat state, a breathing state, or the like is detected by a driver state detection sensor. Then, a state determination unit, which is a functional element of a vehicle ECU, determines whether or not the driver is in a driving difficulty state based on information transmitted from the driver state detection sensor.

[0004] As in JP 5 919 150 B2, a determination result indicating that the driver is in a driving difficulty state is used as a trigger for whether to initiate an emergency operation such as automatic deceleration and automatic stop. Therefore, erroneous detection of the driving difficulty state is difficult to tolerate. However, in JP 5 919 150 B2, for example, if posture collapse or the like occurs in the driver in a normal state, it may be erroneously determined that the driver is in the driving difficulty state based on such a change in driving posture.

[0005] In addition, each of the following documents, DE 10 2017 117 472 A1, DE 11 2016 002 612 B4, DE 10 2015 122 603 A1 and DE 10 2015 201 790 A1, discloses a driver abnormality detection unit that detects an abnormality in a state of the driver and an operation abnormality detection unit that detects an abnormality in a driving operation input by the driver. Summary of the invention

[0006] It is an object of the present disclosure to provide a condition determination device, a condition determination program, and a computer-readable non-transitory tangible storage medium capable of reducing erroneous detection of a driver's driving difficulty state.

[0007] According to a first aspect of the present disclosure, a state determination device for determining that a driver driving a vehicle is in a driving difficulty state includes: a driver abnormality detection unit that detects an abnormality in a state of the driver, a supplementary abnormality detection unit that detects at least one of the abnormalities: abnormality in a driving operation input by the driver or abnormality in a traveling state of the vehicle, and an integrated determination unit that, in response to the driver abnormality detection unit detecting the abnormality only in the state of the driver, does not determine that the driver is in the driving difficulty state, but determines that the driver is in the driving difficulty state in response tothat the driver abnormality detection unit detects the abnormality in the state of the driver and, in addition, the supplementary abnormality detection unit detects at least one of the abnormalities, abnormality in the driving operation input by the driver or abnormality in the traveling state of the vehicle.,

[0008] The integrated determination unit of the above aspect does not determine that the driver is in a driving difficulty state in response to only detecting the abnormality in the driver's state. In response to, in addition to detecting such an abnormality in the driver's state, also detecting at least one of the driving operation and / or the traveling state, the integrated determination unit determines that the driver is in the driving difficulty state. According to the above, a situation in which a posture collapse or the like of the driver in a normal state is mistaken for a driving difficulty state is unlikely to occur. Therefore, it is possible to reduce erroneous detection of the driver's driving difficulty.

[0009] According to a second aspect of the present disclosure, a state determination program for determining that a driver driving a vehicle is in a driving difficulty state causes at least one processing unit to function as: a driver abnormality detection unit that detects an abnormality in a state of the driver, a supplementary abnormality detection unit that detects at least one of the abnormalities, abnormality in a driving operation input by the driver or abnormality in the traveling state of the vehicle, and an integrated determination unit that, in response to the driver abnormality detection unit detecting the abnormality only in the state of the driver, does not determine that the driver is in the driving difficulty state, but determines that the driver is in the driving difficulty state, in response tothat the driver abnormality detection unit detects the abnormality in the state of the driver and, in addition, the supplementary abnormality detection unit detects at least one of the abnormalities, abnormality in the driving operation input by the driver or abnormality in the traveling state of the vehicle.,

[0010] The integrated determination unit in the above aspect does not determine that the driver is in a driving difficulty state in response to only detecting the abnormality in the driver's state. In response to, in addition to detecting such an abnormality in the driver's state, also detecting at least one of the driving operation and / or the traveling state, the integrated determination unit determines that the driver is in the driving difficulty state. According to the above, a situation in which a posture collapse or the like of the driver in a normal state is mistaken for a driving difficulty state is unlikely to occur. Therefore, it is possible to reduce erroneous detection of the driver's driving difficulty.

[0011] According to a third aspect of the present disclosure, in a computer-readable non-transitory storage medium having computer-executable instructions comprising a computer-implemented method of determining that a driver driving a vehicle is in a driving difficulty state, the method comprises: detecting an abnormality in a state of the driver, detecting at least one of the abnormalities, abnormality in a driving operation input by the driver or abnormality in a traveling state of the vehicle, determining that the driver is in the driving difficulty state in response to detecting the abnormality in the state of the driver, and additionally detecting the at least one of the abnormalities, abnormality in the driving operation input by the driver or abnormality in the traveling state of the vehicle,while in response to only detecting the abnormality in the driver's condition alone, it is not determined that the driver is in the driving difficulty state.

[0012] In the computer-readable non-transitory storage medium of the above aspect, it is not determined that the driver is in a driving difficulty state in response to only detecting the abnormality in the state of the driver. It is determined that the driver is in the driving difficulty state in response to detecting the abnormality in the state of the driver and additionally detecting at least one of the abnormalities, an abnormality in the driving operation input by the driver, or an abnormality in the traveling state of the vehicle. According to the above, a situation in which a posture collapse or the like by the driver of a normal state is mistaken for a driving difficulty state is unlikely to occur. Therefore, it is possible to reduce erroneous detection of the driver's driving difficulty. Short description of the drawings

[0013] The above and other objects, features, and advantages of the present disclosure will become more apparent from the following detailed description made with reference to the accompanying drawings. In the drawings: is Fig. 1 is a block diagram showing an overall view of a driving support system mounted on a vehicle, is Fig. 2 a diagram illustrating a number of functions implemented in an integrated ECU, is Fig. 3 is a diagram illustrating details of a first mapping matrix showing a mapping among driver abnormality points and driving operation abnormality points, is Fig. 4 is a diagram showing details of a second mapping matrix showing mapping among abnormal driving operation points and abnormal locomotion state points is Fig. 5 is a flowchart showing details of a dead man determination processing, is Fig. 6 is a flowchart showing details of a dead man determination processing in a first modification, and is Fig. 7 is a flowchart showing details of a dead man determination processing in a second modification. Modes for Carrying Out the InventionFirst Embodiment

[0014] An integrated ECU 100 implements functions of a state determination device according to an embodiment of the present disclosure, which are described in Fig. 1 and Fig. 2. The integrated ECU (electronic control unit) 100 is a computing device used in a driving support system 10 mounted in a vehicle, and is a core of the driving support system 10. The driving support system 10 functions as a deceleration-stop type abnormality response system when a driver falls into an abnormal physical state and hardly continues driving.

[0015] The driving assistance system 10 estimates the driver's driving difficulty state through the integrated ECU 100 and initiates emergency travel. The driver's driving difficulty state is a so-called deadman's state, in which it is difficult to restore a normal driving feasible state due to an unpredictable sudden change in a physical state. For example, the driver's inattention and carelessness do not correspond to the driving difficulty state (deadman's state) to be detected because they do not represent a sudden change in a physical state. The driver's drowsiness may not correspond to the driving difficulty state to be detected because, even if caused by a sudden change in a physical state, it may be possible to restore it to the normal driving feasible state.

[0016] The driving assistance system 10 includes a plurality of vehicle-mounted devices 20 that are directly or indirectly connected to the integrated ECU 100. In addition, the driving assistance system 10 includes a force application ECU 41, a lane departure warning ECU 43, a V2V change ECU or V2V warning ECU 45, a travel control ECU 50, and the like, along with the above-described integrated ECU 100.

[0017] The vehicle-mounted devices 20 include a vehicle speed sensor 21, a seatbelt sensor 22, a navigation device 23, a G sensor 24, a seat sensor 25, an accelerator pedal sensor 26, a steering angle sensor 27, and the like. Furthermore, the vehicle-mounted devices 20 include a driver camera 28, a front camera 29, a rear camera 30, a front / rear sensor 31, a yaw rate sensor 32, a steering grip state sensor 33, a brake sensor 34, and the like.

[0018] The vehicle speed sensor 21 is a sensor that detects a rotational speed of a tire wheel as information corresponding to the speed of the vehicle. The seatbelt sensor 22 is an encoder that detects a rotation angle of a motor that extends and retracts a seatbelt of a driver's seat or a passenger's seat. The navigation device 23 is configured to include a GNSS (Global Navigation Satellite System) receiver, a map database, and the like. The navigation device 23 specifies a current position of the vehicle based on positioning signals received by the GNSS receiver, etc. The navigation device 23 supplies the integrated ECU 100 with the current position of the vehicle and map data around the current position including a planned route of the vehicle.

[0019] The G sensor 24 is a sensor that detects accelerations acting on the vehicle. The seat sensor 25 is a sensor that detects pressure distribution on the driver's seat or passenger seat. The accelerator pedal sensor 26 is a sensor that detects the degree of depression of an accelerator pedal (accelerator pedal opening). The steering angle sensor 27 is a sensor that detects the steering direction and the steering angle.

[0020] The driver camera 28 includes a near-infrared light source and a near-infrared camera, and a control unit that controls them. The driver camera 28 is arranged, for example, on an upper surface of a dashboard, with the near-infrared camera facing the driver's seat. Using the near-infrared camera, the driver camera 28 captures an upper half of the driver's body illuminated by the near-infrared light source to monitor the driver's condition. The driver camera 28 primarily captures a head portion above the driver's neck and successively outputs the captured facial images to the integrated ECU 100.

[0021] The front camera 29 is a camera that captures an area in a traveling direction (front or front side) of the vehicle. The rear camera 30 is a camera that captures an area behind the vehicle. The front / rear sensor 31 is a millimeter-wave radar, a lidar, an ultrasonic sensor, etc., and detects a distance to an object in front of or behind the vehicle. The yaw rate sensor 32 is a sensor that detects a yaw rate acting on the vehicle. The steering grip state sensor 33 is installed on a peripheral portion of a steering wheel and detects the driver's grip state of the steering wheel. The brake sensor 34 is a sensor that detects a depression amount or an operation speed of an accelerator pedal or a brake pedal.

[0022] The force application ECU 41 is electrically connected to a pedal actuator 35 and a steering actuator 36 in addition to the accelerator pedal sensor 26, the steering angle sensor 27, and the brake sensor 34. The pedal actuator 35 is provided at each of the accelerator and brake pedals and can apply a force to a corresponding pedal. The steering actuator 36 can apply a reaction force to a steering shaft.

[0023] The force application ECU 41 performs a diagnosis of whether or not the driver can correctly perform a driving operation based on the driver's response to the force applied by the actuator 35, 36. In addition, the force application ECU 41 has a function of detecting a decrease in the operating force on the accelerator pedal and the brake pedal as one of the diagnostic functions.

[0024] A pedal operation force decrease detection function detects a decrease in pedal operation force based on a deterioration in the driver's response to the force applied by the pedal actuator 35, such as an increase in operation delay and excessive operation input. The force application ECU 41 provides the integrated ECU 100 with the detection result obtained by the pedal operation force decrease detection function.

[0025] The lane departure warning ECU 43 is an electronic control unit that has a function of detecting a lane marking crossing of the vehicle. The lane departure warning ECU 43 is electrically connected to a lane marking detection sensor 37 that detects a lane marking (lane line) dividing a traveling lane. The lane departure warning ECU 43 warns the driver with a warning sound or a warning display in response to detecting such a vehicle behavior as lane marking crossing. In addition, the lane departure warning ECU 43 provides the integrated ECU 100 with the detection result of the lane marking crossing. The front camera 29 or the rear camera 30 can also serve as the lane marking detection sensor 37.Alternatively, the lidar included in the front / rear sensor 31 may also serve as the lane marking detection sensor 37.

[0026] The V2V warning ECU 45 is an electronic control unit that has a function of detecting a continuation of a short inter-vehicle distance. The V2V warning ECU 45 is electrically connected to the V2V distance sensor 38, which measures the inter-vehicle distance. The V2V warning ECU 45 warns the driver with a warning sound or a warning display when the vehicle continues traveling in a state where it is excessively close to a preceding vehicle. In addition, in response to detecting the continuation of the short inter-vehicle distance, the V2V warning ECU 45 provides the detection result to the integrated ECU 100. The front camera 29 or the front / rear sensor 31 can also serve as the V2V distance sensor 38.

[0027] The above-described operations of the force application ECU 41, the lane departure warning ECU 43, and the V2V warning ECU 45 can be manually switched to the OFF state by the driver's operation. For example, the application of force by the force application ECU 41 changes a feeling of operating an actuation system. The driver can therefore turn off the force application ECU 41 if they feel that the application of force by the actuators 35 and 36 is annoying. Similarly, the lane departure warning ECU 43 and the follow-up distance ECU 45 have a tendency to give a preliminary warning. The driver can therefore turn off these operations if they feel that the warning sound and warning display are annoying.

[0028] The locomotion control ECU 50 is an electronic control unit that includes, as its main component, a computer having a processor, a RAM (random-access memory), a storage device, an input / output interface, etc. The locomotion control ECU 50 controls locomotion of the vehicle in an integrated manner through the processor executing a locomotion control program. The locomotion control ECU 50 is electrically connected to the drive unit 51, the automatic braking device 52, the automatic steering device 53, the warning device 54, the headlight 55, the direction indicator 56, etc.

[0029] The drive unit 51 is configured to include an internal combustion engine, a transmission, a motor generator, etc. The drive unit 51 generates driving force for propelling the vehicle. The automatic braking device 52 applies braking force to the vehicle based on the detection results of the front camera 29, the front / rear sensor 31, the V2V distance sensor 38, etc., without depending on the driver's braking operation. The automatic steering device 53 performs steering control of the steering so that the vehicle moves along the travel lane. The warning device 54 alerts, alerts, and warns the driver with a warning sound or a warning display. The headlight 55 is an illumination device installed to face the forward travel direction of the vehicle.The direction indicator 56 is a device that informs the surroundings about the direction of movement of the vehicle in cases of right and left turns and / or route changes.

[0030] The above-described travel control ECU 50 has a lane departure prevention function and a road departure prevention function, making it possible to perform substantially autonomous travel. Based on the integrated ECU 100 determining the occurrence of the travel difficulty state, the travel control ECU 50 causes the vehicle to gradually accelerate and stop within a certain space through cooperative control of the drive unit 51, the automatic braking device 52, and the automatic steering device 53. In addition, at the time of occurrence of the travel difficulty state, the travel control ECU 50 uses the warning device 54 and the direction indicator 56 to warn vehicles traveling around and nearby.

[0031] The integrated ECU 100 is an electronic control unit that has a function of determining that the driver driving the vehicle is in the driving difficulty state. The integrated ECU 100 detects a driver abnormality, a driving operation abnormality, and a vehicle state abnormality for determining the driving difficulty state. The driver abnormality is an abnormality in the driver's state other than his or her driving operation and is detected based on the image captured by the driver camera 28. The driving operation abnormality is an abnormality regarding the driver's driving operation actions and is basically detected based on the operation information of respective pedals and steering.The vehicle state abnormality is an abnormality in the vehicle behavior and traveling state that occurs as a result of the driver's driving operation, and is basically detected based on the vehicle speed information or the vehicle outside cameras 29 and 30.

[0032] The integrated ECU 100 basically includes a control circuit 60 including a processor unit 61, a RAM 62, a storage device 63, and an input / output interface. The processor unit 61 may include a GPU (graphics processing unit), etc., in addition to including a CPU (central processing unit). Furthermore, the processor unit 61 may be provided with an FPGA (field-programmable gate array), an accelerator associated with learning from, interacting with, or influencing an AI (artificial intelligence), or the like.

[0033] The storage device 63 stores various programs executed by the processor unit 61. A plurality of programs stored in the storage device 63 include a condition determination program for performing a deadman determination. Through the processor unit 61 executing the condition determination program, the integrated ECU 100 implements a driver abnormality detection unit 71, a supplementary abnormality detection unit 72, and a deadman determination unit 75, and the like, as functional units.

[0034] The driver abnormality detection unit 71 detects driver abnormality points based on image analysis of the driver's facial image captured by the driver camera 28. Such driver abnormality points include abnormalities in driving posture and abnormalities in biological information. Abnormalities in driver operations are not included in the driver abnormality points. As the detection function for the driving posture abnormalities, the driver abnormality detection unit 71 includes a forward falling posture detection function, a lateral falling posture detection function, a stiff posture detection function, and a steering non-grip detection function. As the abnormalities in biological information, the driver abnormality detection unit 71 includes a white-eye detection function and a pulse wave abnormal detection function.A corresponding detection function performs a detection process to detect a specific driver condition abnormality.

[0035] The forward fall posture detection function detects the forward fall state of the driver's posture. A driver's head in the forward fall state has moved forward and downward while maintaining a downward posture angle compared to a normal driving posture. The forward fall posture detection function specifies a position and posture angle of the head by analyzing the face image of the driver camera 28 to detect the driver's forward fall state.

[0036] The side-fall posture detection function detects a side-fall state of the driver's posture. The driver's head in the side-fall state has moved sideways and downward while maintaining a sideways posture angle compared to the normal driving posture. The side-fall posture detection function specifies a position and posture angle of the head by analyzing the facial image of the driver's camera 28 to detect the driver's side-fall state.

[0037] In this regard, due to the forward fall state or the side fall state, the driver's head may be framed out of the shooting range of the driver camera 28. This situation is identifiable based on the information from the seatbelt sensor 22 and the seat surface sensor 25. The forward fall posture detection function and the side fall posture detection function detect the driver's abnormality according to a situation in which the driver is detected by the seat surface sensor 25 and the seatbelt is extremely pulled out.

[0038] The stiff posture detection function detects a stiff state, which is a state in which the driver's body has become stiff. For example, a driver in a stiff state due to impaired consciousness, such as fainting, has a smaller amplitude of head sway than in a normal state. The stiff posture detection function analyzes the behavior of the head by analyzing the facial image of the driver's camera 28 to detect the driver's stiff state.

[0039] The steering non-grip detection function detects a non-grip state of the steering wheel. The non-grip detection function analyzes the detection signal of the steering grip state sensor 33 to determine whether or not the driver's grip state of the steering wheel corresponds to a grip state predefined as normal. As the non-grip state of the steering wheel, the non-grip detection function detects a case where the current grip state deviates from the normal grip state.

[0040] The white-eye detection function detects when the driver rolls back his or her eye. In the state of eye rollback, a significant exfoliation of the black eye region increases. The white-eye detection function calculates the exfoliation of the black eye region through image analysis of the driver camera 28 and detects the white-eye state in response to a specific threshold being exceeded.

[0041] The pulse wave abnormality detection function detects an abnormality in the driver's pulse wave. The pulse wave is a propagation of arterial pressure waves caused by the pumping action of the heart. There is a correlation between the time it takes for the pulse wave to propagate through a blood vessel (pulse wave propagation time) and blood pressure fluctuation. Therefore, it is possible to estimate an abnormal blood pressure fluctuation of the driver based on the pulse wave. Based on brightness information extracted from the facial image of the driver's camera 28, the pulse wave abnormality detection function monitors the condition associated with the driver's pulse wave, and possibly the blood pressure, to detect the abnormality or deviation from a normal value.

[0042] The driver abnormality detection unit 71 can detect an abnormality in heart rate, an abnormality in breathing, etc. as the abnormalities associated with the biological information. For example, the heart rate can be detected using a detection signal from the steering grip state sensor 33 installed in the steering wheel. The breathing state can be detected through a process of analyzing the change in the detection signal from the seat surface sensor 25.

[0043] The supplementary abnormality detection unit 72 detects the above-described driving operation abnormality and vehicle state abnormality as abnormality points not included in the driver abnormality. The supplementary abnormality detection unit 72 can perform an abnormality detection process based on the information acquired from the vehicle-mounted device 20 and can perform an abnormality detection process using a detection signal output from a corresponding ECU 41, 43, 45 described above, which indicates an abnormal operation or an abnormal traveling. The supplementary abnormality detection unit 72 is configured to include an operation abnormality detection unit 73 and a traveling abnormality detection unit 74.

[0044] The operation abnormality detection unit 73 detects driving operation abnormality points. In response to the driver abnormality detection unit 71 detecting an abnormality in the driver's state, the operation abnormality detection unit 73 starts detection processing for detecting an abnormality in a driving operation. A plurality of abnormal operations are specified in advance as these driving operation abnormality points. The operation abnormality detection unit 73 has a plurality of detection functions, each of which is capable of executing a detection process for detecting a specific abnormal operation.

[0045] Specifically, the operation abnormality detection unit 73 includes a steering wobble input detection function, a steering input force decrease detection function, a brake input magnitude abnormality detection function, a visual fixation detection function, and an accelerator input abnormality detection function as detection functions for driving operation abnormality points. In addition, the operation abnormality detection unit 73 can detect decreases in the operation force of the accelerator pedal and the brake pedal as abnormal operations by obtaining the detection result from the force application ECU 41 through the pedal operation force decrease detection function.

[0046] The steering wobble input detection function detects a wobbly operation input by the driver. A wobbly operation is an operation that periodically changes the vehicle's position to the right and left in the lane. The steering wobble input detection function analyzes the amount of operation on the steering wheel from the information from the steering angle sensor 27, etc., and detects the driver's wobbly operation.

[0047] The steering operation force decline detection function detects a decrease in stability of the driver's steering operation input to the steering wheel based on information from the vehicle speed sensor 21, the steering angle sensor 27, and the like. Specifically, a current steering angle, assuming that steering is performed smoothly while traveling at a constant vehicle speed, is estimated, and an error between the estimated value and the current actual value is calculated by the steering operation force decline detection function. Then, when a distribution of errors and the steering operation is gradual, the steering operation force decline detection function detects a decrease in the steering operation force.

[0048] The brake operation amount abnormality detection function detects an abnormality in the driver's brake operation input to the brake pedal based on information from the vehicle speed sensor 21, the front camera 29, the front / rear sensor 31, the brake sensor 34, and the like. The brake operation amount abnormality detection function learns a normal operation speed of the brake operation in accordance with the vehicle speed and the V2V distance. The brake operation amount abnormality detection function detects an abnormal state of the brake operation when the brake operation speed detected by the brake sensor 34 deviates from the learned normal brake operation speed.

[0049] The visual fixation detection function specifies the driver's visual position based on information from the front camera 29, the driver camera 28, etc., to detect visual fixation as an abnormality. Specifically, the visual fixation detection function detects an eye contour and the central position of the black eye from the face image through the driver camera 28, and detects a line of sight direction from a positional relationship between them. Then, when the line of sight does not follow an object captured by the front camera 29, the visual fixation detection function detects fixation of the visual movement.

[0050] The accelerator operation abnormality detection function detects, as an abnormality, an unnatural accelerator operation such as a continuation of depression of the accelerator pedal, based on information from the vehicle speed sensor 21, the accelerator pedal sensor 26, the front / rear sensor 31, etc. An example is such that in each of the normal operation state and the abnormal operation state, the accelerator operation abnormality detection function predicts the accelerator opening at the next time and compares the accelerator opening actually detected at the next time with a corresponding estimated value.As a result of such a comparison, when the actual accelerator opening is closer to the estimated value of the abnormal operation state than to the estimated value of the normal operation state, the accelerator operation abnormality detection function detects an abnormality in the accelerator operation.

[0051] The traveling abnormality detection unit 74 detects abnormal vehicle state points. When the operation abnormality detection unit 73 detects an abnormality in the driving operation, the traveling abnormality detection unit 74 starts detection processing for detecting an abnormality in the traveling state. As such vehicle state abnormality points, a plurality of abnormal traveling states are defined in advance. The traveling abnormality detection unit 74 has a plurality of detection functions, each of which is capable of performing detection processing for detecting a specific abnormal vehicle traveling.

[0052] Specifically, as detection functions for the vehicle state abnormality points, the traveling abnormality detection unit 74 includes a vehicle traveling direction swing detection function, a long-term overspeed detection function, an on-main-road abnormal low-speed detection function, a collision detection function, etc. Furthermore, the traveling abnormality detection unit 74 is capable of detecting these abnormal travels by obtaining the detection results by the lane mark crossing detection function from the lane departure warning ECU 43 and the detection result by the short V2V distance continuation detection function from the V2V warning ECU 45.

[0053] The vehicle guidance direction sway detection function extracts an acceleration component in the vehicle's traveling direction from the detection result of the G sensor 24 and calculates a jerk by first-order differentiation of the acceleration. Then, the number of times the absolute value of the jerk reaches a certain value (e.g., 10 m / s^3) or more is counted, and the sway detection function detects an abnormality in response to counting a certain number of times (e.g., 4 times per 5 seconds) or more within a predetermined time.

[0054] The long-term overspeed detection function acquires a speed limit from the road on which the vehicle is currently traveling from the navigation device 23. When a state in which a difference between the vehicle speed indicated by the vehicle speed sensor 21 and a control speed is equal to or greater than a certain value (for example, 20 km / h) continues for a certain time (for example, 5 s) or more, the overspeed detection function detects that it is abnormal.

[0055] The abnormal low-speed detection function for major roads extracts information about whether the current traveling road is a major road from the map data provided by the navigation device 23. Then, when traveling on the major road, the abnormal low-speed detection function detects an abnormal low-speed condition using the vehicle speed indicated by the vehicle speed sensor 21. For example, when the condition of traveling below a certain value (e.g., 60 km / h) continues for a certain time (e.g., 5 seconds) or more, the abnormal low-speed detection function detects that there is an abnormality.

[0056] As an abnormality, the collision detection function detects a rapid change in acceleration, which indicates an occurrence of a collision, based on the information from the G sensor 24.

[0057] The deadman determination unit 75 determines the driving difficulty state (deadman) of the driver through a process of fusing the detection results of the driver abnormality detection unit 71 and the supplementary abnormality detection unit 72. The deadman determination unit 75 sequentially performs detection for the driver abnormality points by the driver abnormality detection unit 71, detection for driving operation abnormality points by the operation abnormality detection unit 73, and detection for the vehicle state abnormality points by the traveling abnormality detection unit 74. Then, in response to detecting both an abnormal driving operation point and an abnormal vehicle state in addition to detecting an abnormal driver point, the dead man determination unit 75 determines that the driver is in a driving difficulty state.In other words, the deadman determination unit 75 does not determine that the vehicle is in a driving difficulty state in response to only detecting the abnormal state of the driver abnormality point. In cases where the abnormal points of the three types are not detected in a specific order, the deadman determination unit 75 also does not determine that the driving difficulty exists.

[0058] The deadman determination unit 75 first operates only the detection function for the driver abnormality points, and in response to detecting an abnormality, operates the detection function for a driving operation abnormality point that has a causal relationship (association) with a content of the detected abnormality. Further, when an abnormality in the driving operation is detected, the deadman determination unit 75 operates the detection function for a traveling state abnormality point that has a causal relationship (association) with the detected abnormal operation.

[0059] The detection functions for corresponding abnormal points are related to each other by, for example, an assignment matrix, which is Fig. 3, Fig. 4. Even more specific is the first allocation matrix, which is shown in Fig. 3, a corresponding abnormal point of the driver abnormal points is associated with at least one abnormal point of the driving operation abnormal points that is assumed to have a causal relationship with it. When a driver abnormality is detected, the operation abnormality detection unit 73 starts a detection process for an abnormal operation associated with the detected abnormality content among a plurality of predefined abnormal operations.

[0060] An example is such that forward fall posture detection is associated with accelerator operation abnormality detection and pedal operation force decrease detection. Lateral fall posture detection is associated with steering wobble operation detection, accelerator operation abnormality detection, and pedal operation force decrease detection. Stiffness posture detection is associated with steering operation force decrease detection, brake operation amount abnormality detection, visual fixation detection, accelerator operation abnormality detection function, and pedal operation force decrease detection. White-eye detection is associated with steering operation force decrease detection, accelerator operation abnormality detection, and pedal operation force decrease detection.Pulse wave abnormality detection is associated with steering operation force decline detection, visual fixation detection, accelerator operation abnormality detection, and pedal operation force decline detection. Steering non-grip detection is associated with steering wobble detection and steering operation force decline detection.

[0061] In the second allocation matrix, which is shown in Fig. As shown in Figure 4, a corresponding abnormal point of the abnormal driving operation points is associated with, among the vehicle state abnormal points, at least one or more abnormal points that have a causal relationship therewith. When a driving operation abnormality is detected, the traveling abnormality detection unit 74 starts a detection process for, among a plurality of predefined abnormal travelings, an abnormal traveling associated with the detected abnormal operation.

[0062] An example is such that lane-marker crossing detection and collision detection are associated with steering wobble detection and operation force decrease detection. Brake operation amount abnormality detection is associated with vehicle-guiding-direction sway detection. Visual fixation detection is associated with short-distance V2V continuation detection, lane-marker crossing detection, and collision detection. Accelerator operation abnormality detection is associated with vehicle-guiding-direction sway detection, short-distance V2V continuation detection, long-term overspeed detection, and collision detection. The pedal force decrease detection is assigned to the vehicle direction swing detection, the long-term overspeed detection and the abnormal low speed detection on main road.

[0063] As it is in Fig. 1 and Fig. As shown in FIG. 2, the deadman determination unit 75 controls, in an integrated manner, a start and stop of each detection function of the driver abnormality detection unit 71 and the supplementary abnormality detection unit 72. The deadman determination unit 75 basically maintains the activated state of each driver abnormality point detection function of the driver abnormality detection unit 71. In contrast, a corresponding detection function of the supplementary abnormality detection unit 72 is basically stopped and activated by the deadman determination unit 75 at a required timing.

[0064] In addition, the force application ECU 41, the lane departure warning ECU 43, and the V2V warning ECU 45, which are in a stopped state, can be individually activated by the deadman determination unit 75. As described above, the ECUs 41, 43, 45 external to the integrated ECU 100 can be stopped by the driver's operation. When a driver abnormal point or a driving operation abnormal point is detected, the deadman determination unit 75 forcibly activates a required ECU among the ECUs 41, 43, 45 in the stopped state.

[0065] When the driver abnormality detection unit 71 detects a driver abnormality point, the dead man determination unit 75 draws the first allocation matrix (see Fig. 3) and selectively activates an abnormal operation detection function associated with the detected abnormality content among the plurality of abnormal operations. Furthermore, when the operation abnormality detection unit 73 detects a travel operation abnormality point, the deadman determination unit 75 extracts the second association matrix (see Fig. 4) and selectively activates the abnormal locomotion detection function associated with the detected abnormal operation among the plurality of abnormal locomotions. In a similar manner to activating the detection functions of the operation abnormality detection unit 73 and the locomotion abnormality detection unit 74, the deadman determination unit 75 activates the detection function of each of the units 41, 43, and 45 external thereto. In addition, the deadman determination unit 75 performs a process of raising the priority of execution, by the processor unit 61, of each detection function of the operation abnormality detection unit 73 and the locomotion abnormality detection unit 74.A corresponding detection function of the operation abnormality detection unit 73 and the locomotion abnormality detection unit 74 transitions to a stopped state in response to the detection process being finished.

[0066] The dead man determination processing executed by the integrated ECU 100 described above is described in detail based on Fig. 5 and with reference to Fig. 1 and Fig. 2. The dead man determination processing, which is described in Fig. 5, for example, starts in response to the vehicle's ignition being switched to an ON state and is repeated until the ignition is turned off.

[0067] In S101, detection for abnormal driver points is started, and processing proceeds to S102. In S101, for example, the plurality of detection functions of the driver abnormality detection unit 71 sequentially performs detection processes. In S102, it is determined whether or not an abnormality is detected among the driver abnormality points by the detection process executed in S101. If it is determined in S102 that no driver abnormality point is detected, processing proceeds to S110. If it is determined in S102 that an abnormal driver point is detected, processing proceeds to S103.

[0068] In S103, with reference to the first allocation matrix (see Fig. 3) The detection process for a driving operation abnormality point associated with the abnormal content of the driver abnormality point detected at S101 is started, and processing proceeds to S104. In S104, it is determined whether or not an abnormality is detected among the driving operation abnormality points. If it is determined in S104 that no abnormal driving operation point is detected, processing proceeds to S109. If it is determined in S104 that an abnormal driving operation point is detected, processing proceeds to S105.

[0069] In S105, reference is made to the first or second allocation matrix (see Fig. 4) the detection process for the vehicle condition abnormal point (see Fig. 4) associated with the abnormal operation of the abnormal driving operation point detected in S103 is started, and processing proceeds to S106. In S106, it is determined whether or not an abnormality is detected among the vehicle state abnormality points. If it is determined in S106 that no vehicle state abnormality point is detected, processing proceeds to S108. If it is determined in S106 that a vehicle state abnormality point is detected, processing proceeds to S107. In S107, an estimation for the driver's driving difficulty state, that is, the deadman, is performed.

[0070] In S108, it is determined whether or not there is another detection process for a vehicle state abnormality point associated with the abnormal operation. If it is determined in S108 that there is no other detection process for the associated vehicle state abnormality point, the processing proceeds to S109. On the other hand, if it is determined in S108 that there is another detection process for the associated vehicle state abnormality point, this detection process is executed and the processing returns to S106. Then, if an abnormal movement is detected in S106, it is estimated as a dead man in S107.

[0071] In S109, it is determined whether or not there is another process for detecting an abnormal operation item associated with the abnormal content. If it is determined in S109 that there is no other process for the associated driving operation abnormality item, the processing proceeds to S110. On the other hand, if it is determined in S109 that there is another detection process for the associated driving operation abnormality item, the detection process is executed and the processing returns to S104. Then, if the abnormal operation is detected in S104, the processing proceeds to S105.

[0072] In S110, it is determined whether or not there is another driver abnormality detection process that has not yet been executed. If it is determined in S110 that there is another driver abnormality detection process, the detection process is executed and the processing returns to S102. Conversely, if it is determined in S110 that there is no other driver abnormality detection process, the processing proceeds to S111. In S111, this is estimated as the normal state of the driver, that is, the non-deadman.

[0073] Detecting the driver abnormality point alone does not enable the deadman determination unit 75 of the present embodiment described above to determine that the vehicle is in a driving difficulty state. The deadman determination unit 75 determines that the driver is in a driving difficulty state in response to detecting a driver operation abnormality point and a vehicle state abnormality point in addition to detecting the driver abnormality point. According to the above, a situation in which posture collapse or the like is mistaken by the normal-state driver for a driving difficulty state is unlikely to occur. Therefore, erroneous detection of a driving difficulty state can be reduced.

[0074] Additionally, in the present embodiment, both an abnormality in the driving operation and an abnormality in the vehicle state are detected. Simply detecting the abnormalities in the three groups allows the deadman determination unit 75 to determine that the vehicle is in a driving difficulty state; rather, the deadman determination unit 75 determines that the vehicle is in a driving difficulty state under the condition that the driver abnormality point, the driving operation abnormality point, and the vehicle state abnormality point are detected in an assumed order. According to the determination method described above, it is even more difficult for the erroneous detection of the driving difficulty state to occur.

[0075] Furthermore, in the present embodiment, based on the detected abnormality content of the driver abnormality point, the driving operation abnormality point detection function associated with this abnormality content is activated. Additionally, in the present embodiment, based on the detected abnormal operation of the abnormal driving operation point, the detection function for the vehicle state abnormality point associated with this abnormal operation is activated. Selectively executing only the detection functions that have a causal relationship in the above manner avoids executing another detection process that is likely to cause erroneous detection. Therefore, the accuracy of determining the driving difficulty state can be further improved.Furthermore, if the detection function that does not contribute to improving determination accuracy remains stopped, it is possible to reduce power consumption of the integrated ECU 100 and preferentially perform an important detection process. Alternatively, it is possible to set a power required by the integrated ECU 100 to low.

[0076] Furthermore, the deadman determination unit 75 of the present embodiment can forcibly start each of the external ECUs 41, 43, 45 in a stopped state based on the detection results of the driver abnormality detection unit 71 and the operation abnormality detection unit 73. Therefore, even if each detection function is turned off by the driver, it is possible to avoid a situation where the detection result of each ECU 41, 43, 45 is not available. According to the above, at a time when the driver is abnormal, the deadman determination can be appropriately performed even by the integrated ECU 100, which determines the driving difficulty state using the detection function of each of the external ECUs 41, 43, 45.Furthermore, by effectively utilizing calculation resources from the external ECUs 41, 43, 45, it is possible to reduce calculation processing resources required for the integrated ECU 100 while ensuring accuracy in determining the driving difficulty state.

[0077] In the above embodiment, the force application ECU 41, the lane departure warning ECU 43, and the V2V warning ECU 45 each correspond to an "abnormality detection device." Furthermore, the control circuit 60 corresponds to a "processing unit," the deadman determination unit 75 corresponds to an "integrated determination unit," and the integrated ECU 100 corresponds to a "state determination device." Other embodiments

[0078] Although an embodiment according to the present disclosure has been described above, the present disclosure is not construed as being limited to the above embodiment, and it is possible to adopt various embodiments and combinations without departing from the scope and spirit of the present disclosure.

[0079] In the above embodiment, the detection process is performed in the order of the driver abnormality, the driving operation abnormality, and the vehicle state abnormality. However, for example, the detection process for the abnormal driving operation point and the detection process for the vehicle state abnormality point may be performed concurrently with the detection process for the driver abnormality point. As a result, a temporary order of these abnormality detections may be exchanged. One example is such that the state determination device can determine the driving difficulty state even when the driving operation abnormality and the vehicle state abnormality are detected at substantially the same time.

[0080] Furthermore, the state determining device may determine the driving difficulty state based on, for example, a driver abnormality such as posture collapse, and a driving operation abnormality or a vehicle state abnormality.

[0081] For example, in a first modification of the above-described embodiment, the supplementary abnormality detection unit essentially has the detection function for only abnormal driving operation points. In the deadman determination processing of the first modification, which is Fig. As shown in Figure 6, if a driver abnormality is detected (S202: Yes), a driving operation abnormality point detection process is performed (S203). Then, if the driving operation abnormality is further detected (S204: Yes), it is estimated that the driver is a dead man (S205).

[0082] Furthermore, in a second modification of the above-described embodiment, the supplementary abnormality detection unit essentially has only the vehicle state abnormality point detection function. In the deadman determination processing of the second modification shown in FIG. 7, if a driver abnormality is detected (S302: Yes), a vehicle state abnormality point detection process is performed (S303). Then, if the vehicle state abnormality is also detected (S304: Yes), it is estimated that the driver is a deadman (S305).

[0083] Furthermore, in a third modification of the above-described embodiment, the supplementary abnormality detection unit has the driving operation abnormality point detection function and the vehicle state abnormality detection function without distinction. In the deadman determination processing of the third modification, when a driver abnormality is detected, the supplementary abnormality detection unit performs the detection process for a supplementary abnormality point not included in the driver abnormality. Then, if another abnormality is further detected, it is estimated that the driver is a deadman.

[0084] Note that S201, S206, S207, and S208 in the first modification are substantially the same as S101, S109, S110, and S111 of the above-described embodiment. Furthermore, S301, S306, S307, and S308 in the second modification are substantially the same as S101, S108, S110, and S111 of the above-described embodiment.

[0085] The driver abnormality detection unit, the operation abnormality detection unit, and the locomotion abnormality detection unit do not need to have all the detection functions described in the above embodiment. The driver abnormality detection unit, the operation abnormality detection unit, and the locomotion abnormality detection unit may each have at least one detection function. Furthermore, the operation abnormality detection unit and the locomotion abnormality detection unit may be configured to easily obtain the detection result from the external ECU and perform detection based on the detection result.In this way, the cooperation with the external ECU is strengthened and the processing in the integrated ECU is specialized for the detection process for the driver abnormality point, further enabling the achievement of both process optimization and accuracy improvement.

[0086] At least part of the detection functions described in the above embodiment can detect a specific driver abnormality, driving operation abnormality, and vehicle state abnormality using a learning model learned by machine learning. By using such a learning model, it is possible to improve the accuracy in a corresponding individual detection function. In addition, detection using the learning model tends to consume resources of the processor unit. Therefore, an effect of reducing consumption resources by suspending the detection function is further utilized.

[0087] In the above embodiment, the detection functions of the driver abnormality detection unit are substantially in the activated state. However, the deadman determination unit may start and stop a corresponding detection function of the driver abnormality detection unit as needed. The deadman determination unit may maintain at least part of the detection functions of the supplementary abnormality detection unit in the activated state. In this case, the deadman determination unit performs control to increase, on an operating system, for example, an execution priority of this detection processing task of the detection function when necessary.

[0088] In the above embodiment, only the detection processes associated with the detected abnormal points are selectively activated based on the two association matrices. However, the deadman's position determination unit may perform the detection processes for all of the driving operation abnormality points regardless of the detected abnormality content of the driver's state. Similarly, the deadman's position determination unit may perform the detection processes for all of the vehicle state abnormality points regardless of the detected abnormal operation of the driver. Further, in an association matrix, points of the vehicle abnormality group may be associated with points of the driver abnormality group. Furthermore, points associated with each other in the association matrix may be appropriately changed according to a report that is the reason for assuming the causal relationship.

[0089] The integrated ECU of the above-described embodiment can forcibly start the external ECUs in the stopped state. However, at least some of the external ECUs cannot be forcibly activated by the integrated ECU. Furthermore, for the dead man determination, the integrated ECU can use the detection results from various electronic control units mounted on the vehicle and detection results from a mobile terminal and a portable device possessed by the driver. Alternatively, the integrated ECU may be capable of performing all of the detection processes through the control circuit of the integrated ECU.

[0090] A corresponding function of the state determination device provided by the integrated ECU in the above embodiment may be provided by, for example, a control unit of the driver's camera. Specifically, the control unit of the driver's camera may be a processing unit that executes the state determination program. Furthermore, a corresponding function of the state determination device may be provided by a control unit dedicated to deadman determination instead of the integrated ECU. Furthermore, a plurality of ECUs may cooperate to implement corresponding functions of the state determination device.

[0091] Each of the above functions may be provided by software and hardware executing the software, solely software, solely hardware, or combinations thereof. Furthermore, if such a function is provided by an electronic circuit that is hardware, each function may also be provided by a digital circuit containing a large number of logic circuits or by an analog circuit.

[0092] In the above embodiment, various non-perishable tangible storage media are usable as a storage device that stores the state determination program. Forms of such a storage medium can be changed as appropriate. For example, the storage medium may be in the form of a memory card or the like, and may be configured to be inserted into a slot portion provided in a seat air conditioner ECU and electrically connected to the control circuit. In addition, a storage medium that stores the state determination program is not limited to a storage medium configured to be mounted on a vehicle, and may be an optical disc that is an original of a copy of the storage medium, a hard disk drive of a general-purpose computer, etc.

[0093] Control devices and methods described in the present disclosure may be implemented by a special-purpose computer created by configuring a memory and a processor programmed to perform one or more particular functions embodied in computer programs. Alternatively, control devices and methods described in the present disclosure may be implemented by a special-purpose computer created by configuring a processor provided by one or more special-purpose hardware logic circuits. Alternatively, control devices and methods described in the present disclosure may be implemented by one or more special-purpose computers created orcreated by configuring a combination of memory and a processor programmed to perform one or more specific functions, and a processor provided by one or more hardware logic circuits. Computer programs may be stored in a tangible, non-transitory, computer-readable medium as instructions to be executed by a computer.

[0094] Herein, a flowchart or processes in the flowchart described in the present application are configured with a plurality of sections (or referred to as steps), and each section is expressed as, for example, S101. Furthermore, each section may be divided into multiple subsections, while multiple sections may be combined into one section. Furthermore, each section configured in this way may be referred to as a device, a module, or a means.

[0095] While the invention has been described with reference to various embodiments thereof, it should be understood that the invention is not limited to the above-described embodiments and constructions. In addition, various combinations and forms, and other combinations and forms including only a single element, more or fewer elements, are also within the spirit and scope of the present disclosure.

Claims

[1] A condition determining device for determining that a driver driving a vehicle is in a driving difficulty state, the device comprising: a driver abnormality detection unit (71) that detects an abnormality in a state of the driver, an operation abnormality detection unit (73) that detects an abnormality in a driving operation input by the driver, a traveling abnormality detection unit (74) that detects an abnormality in a traveling state of the vehicle, and an integrated determination unit (75) that, in response to only the driver abnormality detection unit detecting the abnormality in the driver's state alone, does not determine that the driver is in the driving difficulty state, but determines that the driver is in the driving difficulty state in response to: the driver abnormality detection unit detecting the abnormality in the driver's state, thereafter the operation abnormality detection unit detecting an abnormal operation, and thereafter the traveling abnormality detection unit detecting an abnormal traveling. [2] A state determining device according to claim 1, wherein: in response to detecting the abnormality in the state of the driver, the operation abnormality detection unit starts a detection process for, among a plurality of abnormal operation points predefined for the driving operation, an abnormal operation associated with an abnormality content detected by the driver abnormality detection unit. [3] A state determining device according to any one of claims 1 to 2, wherein: in response to detecting the abnormality in the traveling operation, the traveling abnormality detection unit starts detection for, among a plurality of abnormal traveling points predefined for the traveling state, an abnormal traveling associated with the abnormal operation detected by the operation abnormality detection unit. [4] A state determining device according to any one of claims 1 to 3, wherein: the integrated determination unit activates a detection function for the abnormal locomotion point associated with the abnormal operation detected by the operation abnormality detection unit or increases an execution priority of the detection function in the locomotion abnormality detection unit. [5] A state determining device according to any one of claims 1 to 4, wherein: the integrated determination unit activates a detection function for the abnormal operation point associated with the abnormality content detected by the driver abnormality detection unit or increases an execution priority of the detection function in the operation abnormality detection unit. [6] A state determining device according to any one of claims 1 to 5, wherein: the travel abnormality detection unit acquires a detection signal indicative of an abnormal operation output from an abnormality detection device (41, 43, 45) mounted on the vehicle and uses the detection signal to detect the abnormal operation, and the integrated determination unit acquires a detection signal indicative of abnormal locomotion outputted from the abnormality detection device and uses the detection signal to detect the abnormal locomotion. [7] A state determining device according to claim 6, wherein: operation of the abnormality detection device can be stopped by the driver, and in response to the driver abnormality detection unit detecting the abnormality in the state of the driver, the integrated determination unit activates the abnormality detection device in a stopped state. [8] A condition determination program for determining that a driver driving a vehicle is in a driving difficulty condition, the program causing at least one processing unit (60) to act as: a driver abnormality detection unit (71) that detects an abnormality in a state of the driver, an operation abnormality detection unit (73) that detects an abnormality in a driving operation input by the driver, a traveling abnormality detection unit (74) that detects an abnormality in a traveling state of the vehicle, and an integrated determination unit (75) that, in response to only the driver abnormality detection unit detecting the abnormality in the driver's state, does not determine that the driver is in the driving difficulty state, but determines that the driver is in the driving difficulty state in response to: the driver abnormality detection unit detecting the abnormality in the driver's state, thereafter the operation abnormality detection unit detecting an abnormal operation, and thereafter the traveling abnormality detection unit detecting an abnormal traveling. [9] A computer-readable non-transitory storage medium comprising computer-executable instructions comprising a computer-implemented method of determining that a driver operating a vehicle is in a driving difficulty condition, the method comprising: detecting an abnormality in a driver’s condition, detecting an abnormality in a driving operation input by the driver, detecting an abnormality in a moving state of the vehicle, determining that the driver is in the driving difficulty state in response to: detecting the abnormality in the driver's state, then detecting an abnormal operation, and then detecting an abnormal travel, while not determining that the driver is in the driving difficulty state in response to detecting the abnormality only in the driver's state.

Images