Determination device

Abstract

If a driver malfunction is suspected, the cause of the malfunction is determined. [Solution] The determination device for determining the state of the vehicle driver estimates whether there is a driver abnormality based on the driver's posture captured by a driver-capturing camera and the output of an odor sensor that detects biological gases. If it is estimated that there is a driver abnormality, it estimates the cause of the driver abnormality based on the driver's posture and increases the detection sensitivity of a specific odor component detected by the odor sensor according to the cause of the abnormality, and determines the driver abnormality based on the output of the odor sensor.

Description

[Technical Field]

[0001] The present disclosure relates to a determination device. [Background Art]

[0002] Conventionally, various techniques have been studied to prevent a driver from driving a vehicle when the driver is in an abnormal state. For example, Patent Document 1 discloses a state detection system that uses sensors such as chemical sensors, biosensors, gas sensors, and ion sensors to detect chemical substances contained in a driver's biological gas and detects the driver's state. Further, Patent Document 2 discloses a technique for estimating alcohol detection caused by drinking using an alcohol detection sensor to prevent drunk driving. [Prior Art Documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2018-108287 [Patent Document 2] Japanese Unexamined Patent Application Publication No. 2009-202655 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] Here, causes of a driver's abnormality may be attributed to fatigue, or may be attributed to poor physical condition. Since the driver's judgment ability and operation ability can vary depending on the cause of the abnormality, if the cause of the abnormality can be determined, appropriate measures can be taken by the vehicle system when an abnormality occurs in the driver.

[0005] The present disclosure has been made in view of the above problems, and an object of the present disclosure is to provide a determination device capable of determining the cause of an abnormality when it is estimated that an abnormality has occurred in a driver. [Means for Solving the Problem]

[0006] To solve the above problems, according to one aspect of this disclosure, a determination device for determining the state of a vehicle driver is provided, comprising one or more processors and one or more memories connected to the one or more processors in a communicative manner, wherein the one or more processors estimate whether or not there is an abnormality in the driver based on the driver's posture captured by a driver camera that photographs the driver and the output of an odor sensor that detects biological gases, and if it is estimated that an abnormality has occurred in the driver, the determination device estimates the cause of the driver abnormality based on the driver's posture and increases the detection sensitivity of a specific odor component detected by the odor sensor according to the cause of the abnormality, and determines the driver abnormality based on the output of the odor sensor. [Effects of the Invention]

[0007] As explained above, according to this disclosure, when it is presumed that a driver malfunction has occurred, the cause of the malfunction can be determined. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram showing an example of the configuration of a vehicle equipped with a determination device according to an embodiment of the present disclosure. [Figure 2] This is an explanatory diagram showing an example of the configuration of the determination device according to the same embodiment. [Figure 3] This flowchart shows the processing operation by the determination device according to the same embodiment. [Figure 4] This flowchart shows the processing operation by the determination device according to the same embodiment. [Figure 5] This flowchart shows the abnormality detection process by the determination device according to the same embodiment. [Figure 6] This is an explanatory diagram showing a method for estimating the cause of an abnormality using the determination device according to the same embodiment. [Figure 7] This flowchart shows the stop support process by the determination device according to the same embodiment. [Figure 8]This flowchart shows the first emergency stop process by the determination device according to the same embodiment. [Figure 9] This is a flowchart showing the second emergency stop process by the determination device according to the same embodiment. [Figure 10] This is a flowchart showing the third emergency stop process by the determination device according to the same embodiment. [Modes for carrying out the invention]

[0009] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. The specific dimensions, materials, numerical values, etc., shown in the following embodiments are merely examples to facilitate understanding of the invention and do not limit the present invention unless otherwise specified. In this specification and drawings, components having substantially the same configuration are denoted by the same reference numerals to avoid redundant explanations.

[0010] <1. Overall Vehicle Configuration> First, the overall configuration of a vehicle equipped with a determination device according to the embodiment of this disclosure will be described.

[0011] Figure 1 is a schematic diagram showing an example of the overall configuration of a vehicle 1 equipped with a determination device 50. The illustrated vehicle 1 is configured as a four-wheel drive vehicle that transmits the drive torque output from the drive source 3 to the front, rear, left, and right wheels. The drive source 3 may be an internal combustion engine such as a gasoline engine or a diesel engine, a drive motor, or both.

[0012] Note that vehicle 1 may be, for example, an electric vehicle including two drive motors, that is, a front wheel drive motor and a rear wheel drive motor, or may be an electric vehicle including a drive motor corresponding to each wheel. Further, vehicle 1 may be a two-wheel drive vehicle of front-wheel drive or rear-wheel drive. Further, when vehicle 1 is an electric vehicle or a hybrid electric vehicle, vehicle 1 may include a secondary battery that accumulates electric power supplied to the drive motor, or a generator such as a motor or a fuel cell that generates electric power to be charged into a battery.

[0013] Vehicle 1 includes a driving force source 3, an electric steering device 13, and a brake device 9 as devices used for driving control of vehicle 1. The driving force source 3 outputs driving torque that is transmitted to a front wheel drive shaft 7F and a rear wheel drive shaft 7R via a transmission (not shown), a front wheel differential mechanism 5F, and a rear wheel differential mechanism 5R. The driving of the driving force source 3 and the transmission is controlled by a vehicle control device 20 configured to include one or a plurality of electronic control units (ECU: Electronic Control Unit).

[0014] The electric steering device 13 includes an electric motor and a gear mechanism (not shown), and adjusts the steering angles of the left and right front wheels by being controlled by the vehicle control device 20. While the driving mode is set to the manual driving mode, the vehicle control device 20 controls the electric steering device 13 based on the steering angle of a steering wheel operated by a driver. Further, while a driving support function or an automatic driving mode is activated, the vehicle control device 20 controls the electric steering device 13 such that vehicle 1 travels along a predetermined position.

[0015] The brake device 9 applies braking force to the front, rear, left and right wheels, respectively. The brake device 9 is configured as, for example, a hydraulic brake device. A predetermined braking force is generated by controlling the hydraulic pressure supplied to each brake device 9 by a hydraulic pressure control unit 11 in accordance with a driving command input from the vehicle control device 20. When vehicle 1 is an electric vehicle or a hybrid electric vehicle, the brake device 9 is used in combination with regenerative braking by the drive motor.

[0016] The vehicle control device 20 includes one or more electronic control devices that control the driving of the driving force source 3, the electric steering device 13, and the hydraulic pressure control unit 11. The vehicle control device 20 may have a function of controlling the driving of a transmission that shifts an output output from the driving force source 3 and transmits the output to wheels. A vehicle state sensor 23 is connected to the vehicle control device 20. The vehicle state sensor 23 includes one or more sensors that detect the operating state and behavior of the vehicle 1. The vehicle state sensor 23 includes, for example, a steering angle sensor, an accelerator position sensor, a brake stroke sensor, and a brake pressure sensor. Further, the vehicle state sensor 23 includes, for example, a vehicle speed sensor, an acceleration sensor, and an angular velocity sensor. The vehicle state sensor 23 transmits a sensor signal indicating detected information to the vehicle control device 20. The vehicle control device 20 is configured to be able to execute driving support processing for the vehicle 1 in accordance with a drive command input from the determination device 50.

[0017] Further, the vehicle 1 includes exterior imaging cameras 21 (21FL, 21FR). The exterior imaging camera 21 is connected to the determination device 50 via a dedicated line or a communication means such as CAN or LIN.

[0018] The exterior imaging camera 21 images the front of the vehicle 1 and generates an image. In the present embodiment, the vehicle 1 includes a pair of left and right stereo cameras 21FL and 21FR as the exterior imaging cameras 21. The stereo cameras 21FL and 21FR are imaging devices provided with an imaging element such as a CCD or CMOS. The stereo cameras 21FL and 21FR transmit the generated stereo images to the determination device 50.

[0019] Note that the vehicle 1 may be provided with a monocular camera in addition to the stereo cameras 21FL and 21FR. Further, the vehicle 1 may be provided with a camera that images the rear of the vehicle 1, or a camera that is provided on a side mirror or the like and images the left rear or right rear of the vehicle 1. Further, the vehicle 1 may be provided with a distance measuring sensor such as LiDAR or millimeter-wave radar in addition to the stereo cameras 21FL and 21FR.

[0020] Vehicle 1 is equipped with a status determination system 10 that determines the driver's condition. The status determination system 10 includes a driver camera 27, a notification device 29, an odor sensor 31, and a determination device (processing device) 50.

[0021] The driver imaging camera 27 photographs the driver of vehicle 1. The driver imaging camera 27 is installed inside the vehicle and oriented to photograph the driver sitting in the driver's seat 71, and generates an image. The driver imaging camera 27 is an imaging device equipped with an image sensor such as a CCD (Charged-Coupled Devices) or CMOS (Complementary Metal-Oxide-Semiconductor). The driver imaging camera 27 transmits the generated image to the judgment device 50.

[0022] The notification device 29 is driven by the determination device 50 and notifies the occupants of various information by means such as image display and audio output. The notification device 29 includes, for example, a display device provided in the instrument panel and a speaker provided in the vehicle 1. The display device may be a display device of a navigation system.

[0023] The odor sensor 31 detects the driver's biological gases. The odor sensor 31 identifies odors and outputs a sensor signal corresponding to the type of odor. For example, the odor sensor 31 is composed of MEMS (Micro Electro Mechanical Systems). The odor sensor 31 has a detection element in which different sensitive films are coated on a piezoelectric thin film. By applying a voltage to the piezoelectric thin film, odor molecules adhere to the resonant sensitive film, which is converted into an electrical signal. The odor is then identified by comparing the pattern of the electrical signal emitted by the detection element.

[0024] For example, the odor sensor 31 adsorbs odor molecules such as acetone, methyl ethyl ketone, ammonia, dimethylformaldehyde, alcohol, carbon dioxide, hydrogen sulfide, methyl sulfate, methyl dioxide, methyl mercaptan, n-butyric acid, isovaleric acid, and n-valeric acid. It identifies odors by comparing the pattern of electrical signals emitted by the detection element according to the adsorbed odor molecules. The odor sensor 31 is driven by the determination device 50 and transmits sensor signals to the determination device 50.

[0025] The odor sensor 31 has a configuration that allows setting the detection sensitivity. The detection sensitivity of the odor sensor 31 can be adjusted by changing the threshold value of the current or voltage value used to identify an odor based on the pattern of electrical signals. Increasing the detection sensitivity corresponds to lowering the threshold value of the current or voltage value used to determine that a certain pattern of electrical signals indicates a predetermined odor.

[0026] The odor sensor 31 may be installed at any location where biological gases emitted by the driver sitting in the driver's seat 71 tend to accumulate. The biological gases emitted by the driver are typically exhaled breath and rise from the body surface, so for example, the odor sensor 31 may be installed on the headrest or the upper part of the seat back of the driver's seat 71. In this embodiment, an example in which the odor sensor 31 is installed on the headrest of the driver's seat 71 will be described.

[0027] The determination device 50 determines the state of the driver of vehicle 1. The determination device 50 comprises one or more processors such as CPUs (Central Processing Units) and one or more memories connected to the one or more processors in a communicative manner. The determination device 50 functions as a device that performs a process to determine the state of the driver by having one or more processors execute a computer program. The computer program is a computer program that causes the processor to execute the operations that the determination device 50 should perform, which will be described later. The computer program executed by the processor may be recorded on a recording medium that functions as a memory in the determination device 50, or it may be recorded on a recording medium built into the determination device 50 or on any external recording medium that can be attached to the determination device 50.

[0028] Recording media for storing computer programs may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs, DVDs, and Blu-ray®; magneto-optical media such as floppy disks; memory elements such as RAM (Random Access Memory) or ROM (Read Only Memory); flash memory such as USB memory and SSDs; and other media capable of storing programs.

[0029] <2. Judgment device> Next, we will explain the state determination system 10 installed in vehicle 1.

[0030] Figure 2 is an explanatory diagram showing the configuration of the determination device 50 according to this embodiment, represented by functional blocks. The determination device 50 is connected to the odor sensor 31 and the driver camera 27 via a dedicated line or a communication means such as CAN or LIN. The determination device 50 is also connected to the external camera 21, the vehicle status sensor 23, the seat position sensor 75, the first pressure sensor 76, the second pressure sensor 77, the notification device 29, the vehicle control device 20, and other control devices (not shown) mounted on the vehicle 1 via a dedicated line or a communication means such as CAN or LIN.

[0031] The seat position sensor 75 includes a plurality of sensors that detect the forward-backward position of the driver's seat 71, which slides back and forth, and the tilt of the backrest. The first pressure sensor 76 is provided on the backrest of the driver's seat 71 and detects different surface pressure distributions on the backrest depending on the driver's posture. The second pressure sensor 77 is provided on the seat surface of the driver's seat 71 and detects different surface pressure distributions on the seat surface depending on the driver's posture.

[0032] The determination device 50 comprises a processing unit 51 and a storage unit 61. The processing unit 51 is configured with one or more processors such as CPUs. Part or all of the processing unit 51 may be configured with updatable components such as firmware, or it may be a program module executed by commands from the CPU or the like.

[0033] The storage unit 61 is composed of one or more memory elements (RAM, ROM, etc.) that are connected to the processing unit 51 in a communicative manner. However, the number and type of storage units 61 are not particularly limited. The storage unit 61 stores data such as computer programs executed by the processing unit 51, various parameters used in arithmetic processing, detection data, and calculation results. A portion of the storage unit 61 is used as the work area of ​​the processing unit 51.

[0034] In addition, the determination device 50 is equipped with an interface (not shown) for sending and receiving data with the odor sensor 31, driver camera 27, exterior camera 21, vehicle status sensor 23, seat position sensor 75, first pressure sensor 76, second pressure sensor 77, notification device 29, and vehicle control device 20.

[0035] The processing unit 51 includes a posture detection processing unit 53, a state determination processing unit 55, and a driving support processing unit 57. The posture detection processing unit 53, the state determination processing unit 55, and the driving support processing unit 57 are implemented by the execution of a program by one or more processors. Note that some or all of the posture detection processing unit 53, the state determination processing unit 55, and the driving support processing unit 57 may be composed of hardware such as analog circuits.

[0036] The following briefly describes the functions of the attitude detection processing unit 53, the state determination processing unit 55, and the driving support processing unit 57, and then explains the specific processing operations in detail.

[0037] (Posture detection processing unit) The posture detection processing unit 53 detects information about the driver's posture based on the image from the driver camera 27 and the outputs of the seat position sensor 75, the first pressure sensor 76, and the second pressure sensor 77. The means for detecting information about the driver's posture are not limited to the above example. For example, one or two of the seat position sensor 75, the first pressure sensor 76, and the second pressure sensor 77 may not be used. Furthermore, the posture detection processing unit 53 may detect the driver's posture based solely on the image from the driver camera 27.

[0038] (State determination processing unit) The state determination processing unit 55 estimates whether there is a driver abnormality based on the driver's posture and the output of the odor sensor 31. If it estimates that the driver is abnormal, it sets the odor components to be detected by the odor sensor 31 according to the driver's posture and determines the cause of the driver abnormality. For example, the state determination processing unit 55 may determine that the cause of the driver abnormality is the driver's poor physical condition or fatigue.

[0039] (Driving support processing unit) When the state determination processing unit 55 determines that an abnormality has occurred in the driver, the driver assistance processing unit 57 executes a stop support process to safely stop the vehicle 1 according to the determination result of the cause of the abnormality. For example, the driver assistance processing unit 57 activates the notification device 29 to notify the driver to stop the vehicle 1. The driver assistance processing unit 57 also searches for a stopping area to stop the vehicle 1 according to the cause of the abnormality and provides guidance to lead the vehicle 1 to the stopping area.

[0040] <3. State determination processing operation> Next, we will explain the state determination processing operation by the determination device 50.

[0041] Figures 3 and 4 are flowcharts showing the processing routine of the determination device 50.

[0042] First, the processing unit 51 performs preparatory operations for the state determination system 10 (step S11). For example, after the start switch that activates the drive system of vehicle 1 is turned on, the state determination processing unit 55 confirms that all doors are closed, the shift position is set to the drive range, and the vehicle speed is greater than zero. This preparatory operation is a process to confirm that the conditions for starting odor determination of the driver's biological gases after the vehicle 1 has started to run are met. The preparatory operation is not limited to the above example and may include any other conditions.

[0043] Next, the processing unit 51 performs an initial odor determination of the driver based on the output of the odor sensor 31 (step S13). The initial determination is a process of acquiring initial data from the odor sensor 31 at the start of operation in order to make subsequent odor determinations relative determinations by comparing them with the data at the start of operation. The state determination processing unit 55 records the odor obtained from the output of the odor sensor 31 as initial data.

[0044] Next, the processing unit 51 determines whether or not an abnormality in the driver was detected in the initial determination (step S15). The determination in step S15 aims to determine whether the driver's condition is clearly not suitable for driving. For example, the condition determination processing unit 55 determines, based on the output of the odor sensor 31, whether the detection value of one or more predetermined odor components is abnormal and whether it is detected at a predetermined intensity or higher. The predetermined odor components may be specific odor components emitted from the driver due to poor physical condition, or they may be alcohol components emitted when the driver is intoxicated.

[0045] If the processing unit 51 detects a driver abnormality (S15 / Yes), it proceeds to step S35 and performs the stop support processing described later. In other words, if the processing unit 51 detects a driver abnormality in the initial determination, it assists the driver in stopping the vehicle 1 quickly without going through the stage of continuously monitoring the driver's status.

[0046] On the other hand, if the processing unit 51 does not detect any abnormality in the driver (S15 / No), it determines whether the conditions for starting monitoring the driver's condition have been met (step S17). A characteristic of people when driving vehicle 1 is that once a predetermined amount of time has elapsed since starting to drive, or once the distance traveled exceeds a predetermined distance, the thought of turning back may not arise. Therefore, even if a driver feels unwell after a predetermined amount of time has elapsed since starting to drive, or after the distance traveled has exceeded a predetermined distance, the driver tends to continue driving despite their condition. For this reason, in this embodiment, the condition determination processing unit 55 determines whether either of the following conditions has been met as the conditions for starting to monitor the driver's condition: that a predetermined amount of time has elapsed since starting to drive, or that the distance traveled has exceeded a predetermined distance. The predetermined amount of time may be, for example, 15 minutes, but may be set to any appropriate time. The predetermined distance traveled may be, for example, 20 km, but may be set to any appropriate distance.

[0047] If the processing unit 51 does not determine that the conditions for starting monitoring have been met (S17 / No), it determines whether the function of the state determination system 10 has stopped (step S19). If the processing unit 51 does not determine that the function of the state determination system 10 has stopped (S19 / No), it returns to step S17. On the other hand, if the processing unit 51 determines that the function of the state determination system 10 has stopped (S19 / Yes), it terminates the processing routine.

[0048] If the processing unit 51 determines in step S17 that the conditions for starting monitoring have been met (S17 / Yes), it starts monitoring the driver's posture and the driver's state based on the output of the odor sensor 31 (step S21). As the monitoring of the driver's state begins, the posture detection processing unit 53 of the processing unit 51 starts estimating the driver's skeletal structure and facial expression based on the image from the driver camera 27 using pattern recognition by image processing. The posture detection processing unit 53 also starts detecting the position of the driver's seat 71, the angle of the backrest, and the surface pressure distribution of the backrest and seat surface based on the output of the seat position sensor 75, the first pressure sensor 76, and the second pressure sensor 77.

[0049] Next, the processing unit 51 performs an odor determination process (step S23). The odor determination process in step S23 is a process that roughly determines the odor inside the vehicle and predicts whether there is an abnormality in the odor inside the vehicle. In other words, the odor determination process in step S23 is not a process that confirms a driver abnormality, but rather a process that determines whether there is a possibility that a driver abnormality is occurring. For example, the state determination processing unit 55 determines, based on the output of the odor sensor 31, whether one or more predetermined odor components set in advance have been detected above a predetermined threshold. The predetermined odor components may be specific odor components generated from the driver due to poor health.

[0050] The processing unit 51 determines whether or not an odor abnormality has been detected as a result of the odor detection process (step S25). If the processing unit 51 does not determine that an odor abnormality has been detected (S25 / No), it determines whether or not the function of the state determination system 10 has stopped (step S27). If the processing unit 51 does not determine that the function of the state determination system 10 has stopped (S27 / No), it returns to step S23. On the other hand, if the processing unit 51 determines that the function of the state determination system 10 has stopped (S27 / Yes), it terminates the processing routine.

[0051] If the processing unit 51 determines that an abnormality in the odor has been detected (S25 / Yes), it performs a process to determine an abnormality in the driver based on the driver's posture and the odor (step S29).

[0052] Figure 5 shows a flowchart of the abnormality detection process in step S29. The processing unit 51 determines the driver's posture (step S51). For example, the state determination processing unit 55 determines, based on the information about the driver's posture detected by the posture detection processing unit 53, whether or not a change in the driver's posture that may occur when there is a problem with the driver has occurred. Changes in the driver's posture that may occur when there is a problem with the driver include, for example, a sudden decrease in the surface pressure applied to the backrest, the driver taking either their left or right hand off the steering wheel, or the driver's facial expression changing to one of distress, but are not limited to the above examples.

[0053] The processing unit 51 determines whether or not an abnormality has been detected in the driver's posture (step S53) based on the result of determining the driver's posture. If the processing unit 51 does not determine that an abnormality has been detected in the driver's posture (S53 / No), it returns to step S51. On the other hand, if the processing unit 51 determines that an abnormality has been detected in the driver's posture (S53 / Yes), it estimates the cause of the abnormality occurring in the driver (step S55). In step S55, it is determined whether the abnormality occurring in the driver is due to poor physical condition or fatigue.

[0054] Figure 6 shows an example of a method for estimating the cause of a driver malfunction. The illustrated example shows how to estimate the cause of a driver malfunction based on the surface pressure distribution of the backrest, the surface pressure distribution of the seat surface, and the driver's posture identified from the image of the driver camera 27.

[0055] For example, if the load on the backrest suddenly decreases, the load on the seat shifts forward, and the driver's arms are in front of their body, the driver is hunched over and holding their abdomen with their hands, and the state determination processing unit 55 estimates that the cause of the driver's abnormality is a physical ailment related to the abdominal system. Also, if the load on the backrest suddenly decreases, the load on the seat shifts forward, and the driver's arms are at their head, the driver is hunched over and holding their head with their hands, and the state determination processing unit 55 estimates that the cause of the driver's abnormality is a physical ailment related to the brain system.

[0056] Furthermore, if the load on the backrest suddenly decreases, the load on the seat shifts to either the left or right side, and the driver's arms are on their chest, the driver's body is tilted to either the left or right side, and they are holding their chest with their hands, and the state determination processing unit 55 estimates that the cause of the driver's abnormality is poor health of the heart or internal organs. In addition, if the load on the backrest suddenly decreases, the load on the seat shifts to either the left or right side, and the driver's arms are on their head or behind their body, the driver's body is tilted to either the left or right side, and they are holding their head or back with their hands, and the state determination processing unit 55 estimates that the cause of the driver's abnormality is fatigue.

[0057] It should be noted that the example shown in Figure 6 is merely one example, and the cause of the driver abnormality and the expected driver posture pattern at that time may be set in advance as reference information. In addition, information other than the driver's posture identified from the backrest surface pressure distribution, the seat surface pressure distribution, and the driver camera 27 may be used as information to estimate the cause of the driver abnormality. For example, a load sensor that detects the load on the steering wheel may be provided to detect how the driver is leaning against the steering wheel, and this may be used to estimate the cause of the driver abnormality.

[0058] Next, the processing unit 51 determines whether the estimated cause of the abnormality is poor physical condition (step S57). If the estimated cause of the abnormality is not poor physical condition (S57 / No), the processing unit 51 determines that the cause of the driver's abnormality is fatigue or external injury (step S59). On the other hand, if the estimated cause of the abnormality is poor physical condition (S57 / Yes), the processing unit 51 increases the detection sensitivity of specific odor components detected by the odor sensor 31 according to the driver's posture (step S61).

[0059] For example, abdominal health problems can be caused by gastroenteritis, gastric or duodenal ulcers, intestinal obstruction or torsion, and diabetic ketoacidosis. In this case, due to acid reflux, indigestion, ketoacidosis, etc., the biological gases emitted by the driver may include odors such as sour, putrid, and acetone. Therefore, if the cause of the abnormality estimated from the driver's posture is an abdominal health problem, the state determination processing unit 55 increases the detection sensitivity of the expected odor components mentioned above.

[0060] Furthermore, brain-related health problems can be caused by migraines, hypoglycemia, dehydration, and stress or anxiety. In these cases, the bio-gas emitted by the driver may include odors such as acetone, metallic, and bitter or sour due to increased ketones, changes in blood flow, and acid reflux. Therefore, the state determination processing unit 55 increases the detection sensitivity of the expected odor components if the cause of the abnormality estimated from the driver's posture is a brain-related health problem.

[0061] Furthermore, heart or internal organ problems can be caused by heart failure, angina pectoris, myocardial infarction, and chronic heart disease. In these cases, due to hypoxia, metabolic abnormalities, and poor blood circulation, the bio-gases emitted by the driver may include odors such as metallic, sweet, and putrid smells. Therefore, the state determination processing unit 55 increases the detection sensitivity of the expected odor components if the cause of the abnormality estimated from the driver's posture is heart or internal organ problems.

[0062] In addition, the state determination processing unit 55 may increase the detection sensitivity of odor components that may be contained in the bio-gas emitted by the driver, depending on the cause of poor physical condition estimated based on the driver's posture, as shown in the example above.

[0063] Next, the processing unit 51 performs an odor determination of the driver's biological gases based on the output of the odor sensor 31 (step S63). For example, the state determination processing unit 55 determines whether a specific odor component has been detected based on the output of the odor sensor 31. The processing unit 51 repeatedly performs steps S51 to S63 to determine that there is an abnormality in the driver and to identify the cause of the abnormality.

[0064] Returning to Figure 4, the processing unit 51 determines whether or not a driver abnormality has been detected based on the abnormality determination process using the driver's posture and odor (step S31). If the processing unit 51 does not determine that a driver abnormality has been detected (S31 / No), it determines whether or not the function of the state determination system 10 has stopped (step S33). If the processing unit 51 does not determine that the function of the state determination system 10 has stopped (S33 / No), it returns to step S29. On the other hand, if the processing unit 51 determines that the function of the state determination system 10 has stopped (S33 / Yes), it terminates the processing routine.

[0065] If the processing unit 51 determines that a driver abnormality has been detected (S31 / Yes), it performs a stop support process to stop the vehicle 1 (step S35). In this embodiment, the processing unit 51 changes the content of the stop support process according to the result of the determination of whether the driver is unwell or fatigued.

[0066] (Stop support processing) Figure 7 is a flowchart showing the stop support process when the cause of driver malfunction is fatigue or external injury. First, the driver assistance processing unit 57 issues a warning to the driver (step S71). For example, the driver assistance processing unit 57 activates the notification device 29 to notify the driver to stop driving vehicle 1 and take a break.

[0067] Next, the driver assistance processing unit 57 searches for a stopping area to stop the vehicle 1 (step S73). The stopping area for resting is, for example, an area where the vehicle 1 can be parked within 30 minutes, and may be a parking lot of a drive-in, public facility or convenience store, or a roadside with sufficient width. The time required to reach the stopping area may be set to any time. The driver assistance processing unit 57 may search for a parking area, for example, from map data, or it may search for a parking area based on images from the external camera 21.

[0068] Next, the driver assistance processing unit 57 assists the driver's driving operations and starts guidance to lead vehicle 1 to the stopping area (step S75). For example, the driver assistance processing unit 57 limits the speed and acceleration of vehicle 1 and guides the vehicle 1 from its current position to the stopping area. The driver assistance processing unit 57 may also limit the angular velocity of the steering angle of the wheels.

[0069] Next, the driver assistance processing unit 57 determines whether or not vehicle 1 has arrived at the stopping area (step S77). For example, the driver assistance processing unit 57 determines whether or not the vehicle speed has become zero and vehicle 1 has stopped in a safe stopping area. The stopping area may be the stopping area that was guided by the guidance process, or it may be any other stopping area.

[0070] If the driver assistance processing unit 57 does not determine that vehicle 1 has arrived at the stopping area (S77 / No), it repeats the determination in step S77. On the other hand, if the driver assistance processing unit 57 determines that vehicle 1 has arrived at the stopping area (S77 / Yes), it terminates the stopping assistance process.

[0071] (First emergency shutdown support process) Figure 8 is a flowchart showing the stop support process when the cause of the driver's malfunction is a physical ailment related to the abdominal system. In the case of a physical ailment related to the abdominal system, the processing unit 51 performs a first emergency stop process to bring the vehicle 1 to an emergency stop.

[0072] First, the driver assistance processing unit 57 issues a warning to the driver (step S71). For example, the driver assistance processing unit 57 activates the notification device 29 to notify the driver that an abnormality has been detected and that the vehicle 1 should be brought to an emergency stop.

[0073] Next, the driver assistance processing unit 57 searches for an emergency stop area for bringing the vehicle 1 to an emergency stop (step S81). The emergency stop area may be a stopping area or a refuge space where the vehicle 1 can be temporarily stopped. The driver assistance processing unit 57 may search for an emergency stop area based on images from the external camera 21, or it may search for an emergency stop area by referring to high-precision map data including road width information.

[0074] Next, the driver assistance processing unit 57 activates the notification device 29 and presents the driver with one or more emergency stop area options (step S83). For example, the driver assistance processing unit 57 presents the driver with one or more emergency stop areas that can be reached within 5 minutes. This allows the driver to freely select an emergency stop area depending on their physical condition. The driver assistance processing unit 57 sets the emergency stop area selected by the driver from the presented emergency stop areas as the destination.

[0075] Next, the driver assistance processing unit 57 determines whether or not to guide vehicle 1 to the emergency stop area by automatic driving (step S85). For example, the driver assistance processing unit 57 activates the notification device 29 to notify the driver to choose whether to move to the emergency stop area by automatic driving or by manual driving, and determines whether to guide vehicle 1 to the emergency stop area by automatic driving according to the driver's selection. The driver assistance processing unit 57 may also decide whether to move to the emergency stop area by automatic driving or by manual driving depending on the distance to the emergency stop area selected by the driver or the traffic conditions.

[0076] If the driver assistance processing unit 57 determines that it will guide vehicle 1 to an emergency stop area by automatic driving (S85 / Yes), it sets the vehicle to an automatic driving control mode with a speed limit (step S87) and starts automatic driving (step S89). In the automatic driving control mode with a speed limit, the driver assistance processing unit 57 drives vehicle 1 at a speed that allows for emergency braking in case of an emergency.

[0077] On the other hand, if the driver assistance processing unit 57 does not determine that it is appropriate to guide vehicle 1 to the emergency stop area by autonomous driving (S85 / No), it sets the driver assistance level to be strengthened (step S91) and starts guidance to guide vehicle 1 to the emergency stop area by manual driving by the driver (step S93). For example, the driver assistance processing unit 57 limits the speed and acceleration of vehicle 1 and guides the vehicle 1 from its current position to the emergency stop area. The driver assistance processing unit 57 may also limit the angular velocity of the steering angle of the wheels. Furthermore, if the driver assistance processing unit 57 has a lane departure prevention function, it may change the settings to a safer state, such as issuing a warning or automatically controlling the steering angle earlier.

[0078] After starting autonomous driving or guidance, the driver assistance processing unit 57 activates the notification device 29 and periodically asks the driver questions (step S95). For example, the driver assistance processing unit 57 asks questions such as "How are you feeling?" or "Are you feeling alright?" at predetermined time intervals or after a predetermined distance traveled.

[0079] Next, the driver assistance processing unit 57 determines whether or not vehicle 1 has arrived at the emergency stop area (step S97). For example, the driver assistance processing unit 57 determines whether or not the vehicle speed has become zero and vehicle 1 has stopped in the emergency stop area. If the driver assistance processing unit 57 does not determine that vehicle 1 has arrived at the emergency stop area (S97 / No), it repeats the determination in step S97. On the other hand, if the driver assistance processing unit 57 determines that vehicle 1 has arrived at the emergency stop area (S97 / Yes), it executes an automatic parking assistance process to park vehicle 1 in a safe parking space and terminates the first emergency stop process (step S99). The automatic parking assistance process may be an assistance process using a parking assistance function that moves vehicle 1 to a specific parking space using images from the external camera 21, or it may be an assistance process using an auto valet parking function that communicates with an external parking system and moves vehicle 1 to an empty space in the motor pool.

[0080] (Second emergency shutdown support processing) Figure 9 is a flowchart showing the stop support process when the cause of the driver's malfunction is a brain-related physical ailment. Brain-related physical ailments are more urgent than abdominal-related physical ailments, and the processing unit 51 performs a second emergency stop process to bring the vehicle 1 to an emergency stop.

[0081] First, the driver assistance processing unit 57 issues a warning to the driver (step S71), similar to the first emergency stop process, and searches for an emergency stop area to bring the vehicle 1 to an emergency stop (step S81).

[0082] Next, the driver assistance processing unit 57 sets the emergency stop area that can be reached in the shortest time (step S84). For example, the driver assistance processing unit 57 sets the emergency stop area with the shortest travel distance or the emergency stop area that can be reached in the shortest time considering the traffic conditions as the destination from among the one or more emergency stop areas that have been identified as candidates as a result of the search.

[0083] Next, the driver assistance processing unit 57 determines whether or not to guide vehicle 1 to the emergency stop area by automatic driving, similar to the first emergency stop process (step S85). If the driver assistance processing unit 57 determines that vehicle 1 should be guided to the emergency stop area by automatic driving (S85 / Yes), it sets the vehicle to an automatic driving control mode with a speed limit (step S87) and starts automatic driving (step S89). On the other hand, if the driver assistance processing unit 57 does not determine that vehicle 1 should be guided to the emergency stop area by automatic driving (S85 / No), it sets the driver assistance level to be strengthened (step S91) and starts guidance to guide vehicle 1 to the emergency stop area by manual driving by the driver (step S93).

[0084] After starting automatic driving or guidance, the driver assistance processing unit 57 displays a warning to the area around vehicle 1 (step S94). For example, the driver assistance processing unit 57 may turn on the hazard lights or flash the headlights in a continuous manner to alert the area around vehicle 1 of an abnormality.

[0085] Next, the driver assistance processing unit 57 activates the notification device 29 and periodically asks the driver questions, similar to the first emergency stop process (step S95). Then, the driver assistance processing unit 57 determines whether or not vehicle 1 has arrived at the emergency stop area (step S97). If the driver assistance processing unit 57 does not determine that vehicle 1 has arrived at the emergency stop area (S97 / No), it repeats the determination in step S97. On the other hand, if the driver assistance processing unit 57 determines that vehicle 1 has arrived at the emergency stop area (S97 / Yes), it executes an automatic parking assistance process to park vehicle 1 in a safe parking space and terminates the second emergency stop process (step S99).

[0086] (Third emergency shutdown support processing) Figure 10 is a flowchart showing the stop support process when the cause of the driver's malfunction is a heart or internal organ system disorder. In the case of a heart or internal organ system disorder, the urgency is even higher than that of a brain system disorder, and the processing unit 51 performs a third emergency stop process to bring the vehicle 1 to an emergency stop.

[0087] First, the driver assistance processing unit 57 issues a warning to the driver, similar to the first emergency stop process (step S71). Next, the driver assistance processing unit 57 detects a road shoulder where the vehicle 1 can be stopped in an emergency (step S101). For example, the driver assistance processing unit 57 detects a road shoulder where the vehicle 1 can be stopped based on the image from the external camera 21. The driver assistance processing unit 57 may also detect a road shoulder where the vehicle 1 can be stopped based on map data.

[0088] Next, the driver assistance processing unit 57 enhances the recognition level of following vehicles traveling behind vehicle 1 (step S103). The driver assistance processing unit 57 enhances the recognition level of following vehicles in advance to prevent vehicle 1 from being rear-ended if it stops on the shoulder of the road. For example, the driver assistance processing unit 57 uses images from a rear-facing camera and vehicle-to-vehicle communication to acquire detailed information such as the distance to an object estimated to be a following vehicle and the speed at which the object is moving.

[0089] Next, the driver assistance processing unit 57 determines whether or not to guide vehicle 1 to the shoulder of the road by automatic driving, similar to the first emergency stop process (step S85). If the driver assistance processing unit 57 determines that vehicle 1 should be guided to the shoulder of the road by automatic driving (S85 / Yes), it sets the vehicle to an automatic driving control mode with a speed limit (step S87) and starts automatic driving (step S89). On the other hand, if the driver assistance processing unit 57 does not determine that vehicle 1 should be guided to the emergency stop area by automatic driving (S85 / No), it sets the driver assistance level to be strengthened (step S91) and starts guidance to guide vehicle 1 to the emergency stop area by manual driving by the driver (step S93).

[0090] After starting automatic driving or guidance, the driver assistance processing unit 57 displays a warning to the area around vehicle 1 (step S94). For example, the driver assistance processing unit 57 may turn on the hazard lights or flash the headlights in a continuous manner to alert the area around vehicle 1 of an abnormality.

[0091] Next, the driver assistance processing unit 57 determines whether or not vehicle 1 has entered the shoulder of the road (step S98). If the driver assistance processing unit 57 does not determine that vehicle 1 has entered the shoulder of the road (S98 / No), it repeats the determination in step S98. On the other hand, if the driver assistance processing unit 57 determines that vehicle 1 has entered the shoulder of the road (S98 / Yes), it executes an automatic parking assistance process to park vehicle 1 in a safe position on the shoulder of the road and terminates the third emergency stop process (step S99).

[0092] Returning to Figure 4, the processing unit 51, after executing the stop support process, determines whether or not vehicle 1 has stopped (step S37). If it determines that vehicle 1 has stopped (S37 / Yes), it performs post-stopping actions (step S39). For example, the driving support processing unit 57 switches the shift position to the parking range to prevent vehicle 1 from starting. The driving support processing unit 57 may also connect to a call center for communication. This allows the driver to request assistance from a call center operator or the like.

[0093] Furthermore, if the processing unit 51 has already detected a driver abnormality in the initial determination in step S13 (S15 / Yes), it will execute either a stop support process or one of the first to third emergency stop processes. For example, the driving support processing unit 57 may select a process to execute according to the detected odor component and its intensity, or it may select a process to execute according to the urgency estimated based on the driver's posture.

[0094] As described above, the state determination system 10 according to this embodiment includes a driver camera 27 that photographs the driver of the vehicle 1, an odor sensor 31 that detects biological gases, and a determination device 50 that estimates whether or not there is a driver abnormality based on the driver's posture photographed by the driver camera 27 and the output of the odor sensor 31, and if it is estimated that there is a driver abnormality, estimates the cause of the driver abnormality based on the driver's posture, and increases the detection sensitivity of a specific odor component detected by the odor sensor 31 according to the cause of the abnormality, and determines the driver abnormality based on the output of the odor sensor 31. This makes it possible to identify the cause of the abnormality when determining a driver abnormality using the odor sensor 31. Therefore, the system can take appropriate subsequent actions according to the urgency corresponding to the cause of the abnormality.

[0095] Furthermore, in the state determination system 10 according to this embodiment, the determination device 50 determines the driver's physical condition and fatigue based on the driver's posture and sets odor components that increase the detection sensitivity of the odor sensor 31. This makes it possible to determine the cause of the driver's abnormality, which is estimated based on the driver's posture, based on the odor of biological gases emitted by the driver.

[0096] Furthermore, in the state determination system 10 according to this embodiment, the determination device 50 changes the content of the stopping support process that stops the vehicle 1 according to the determination result of the driver's physical condition or fatigue. This makes it possible to change the time until the vehicle 1 is stopped, the stopping position, and the safety measures taken for the surroundings according to the urgency corresponding to the cause of the driver's abnormality, and the vehicle 1 can be stopped safely without excessively restricting the operation of the vehicle 1.

[0097] Furthermore, in the state determination system 10 according to this embodiment, if the output of the odor sensor 31 shows a predetermined abnormal value before the determination device 50 estimates whether there is an abnormality in the driver based on the driver's posture and the output of the odor sensor, it omits the estimation of whether there is an abnormality in the driver and determines that there is an abnormality in the driver, and executes a stop support process to stop the vehicle 1. As a result, if it is determined that there is clearly an abnormality in the driver after the vehicle 1 has started to run, the vehicle 1 can be stopped immediately and its operation can be prohibited.

[0098] Furthermore, in the state determination system 10 according to this embodiment, the determination device 50 estimates whether or not there is an abnormality in the driver after a predetermined time has elapsed since the start of driving the vehicle 1 or after a predetermined distance has been traveled. As a result, the determination of abnormalities due to changes in the driver's physical condition begins after a period has elapsed in which, as is characteristic of human behavior, the thought of turning back may arise after the start of driving the vehicle 1, thus preventing the driver from continuing to drive under strain.

[0099] While preferred embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the present disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art to which the present disclosure pertains that various modifications or alterations may be conceived within the scope of the technical idea set forth in the claims, and these will naturally also be understood to fall within the technical scope of the present disclosure. [Explanation of Symbols]

[0100] 1: Vehicle 10: State determination system 21: External camera 23: Vehicle condition sensor 27: Driver's Camera 29: Notification device 31: Odor sensor 50: Judgment device 51: Processing Unit 53: Posture detection processing unit 55: State determination processing unit 57: Driver Support Processing Unit

Claims

1. In a determination device for determining the state of a vehicle driver, It comprises one or more processors and one or more memories connected to the one or more processors in a communicative manner, The aforementioned one or more processors Based on the driver's posture captured by a driver-capturing camera and the output of an odor sensor that detects biological gases, the presence or absence of a driver malfunction is estimated. If it is estimated that an abnormality has occurred in the driver, the cause of the abnormality in the driver is estimated based on the driver's posture, and the detection sensitivity of a specific odor component detected by the odor sensor is increased according to the cause of the abnormality, and the abnormality of the driver is determined based on the output of the odor sensor. A determination device that performs the following actions.

2. The aforementioned one or more processors The determination device according to claim 1, which predicts the driver's physical condition and fatigue based on the driver's posture and sets odor components that increase the detection sensitivity of the odor sensor.

3. The aforementioned one or more processors The determination device according to claim 2, which changes the content of the stop support process for stopping the vehicle according to the cause of the driver abnormality.

4. The aforementioned one or more processors The determination device according to claim 3, wherein, before estimating whether or not there is an abnormality based on the driver's posture and the output of the odor sensor, if the output of the odor sensor shows a predetermined abnormal value, the determination of whether or not there is an abnormality in the driver is omitted, and the determination that an abnormality has occurred in the driver is executed, and the stopping support process for stopping the vehicle is executed.

5. The aforementioned one or more processors The determination device according to claim 1, which estimates whether or not there is an abnormality in the driver after a predetermined time has elapsed since the start of operation of the vehicle or after a predetermined distance has been traveled.

Images