Driving assistance systems

Abstract

To provide driving assistance control that can perform driving assistance control appropriately reflecting a driver's state of feeling.SOLUTION: A driving assistance device comprises: vehicle state detection units 310, 320, 330 that detect a state of a vehicle 1; a driver's feeling determination unit that determines a driver P's feeling; and a driving assistance control unit 200 that performs driving assistance control including at least one of provision of information to the driver and vehicle travel assistance control according to output from at least one of the vehicle state detection units and the driver's feeling determination unit. The driver's feeling determination unit determines a feeling unsuitable for driving by comparing a difference between reference values and current values of parameters indicating the driver's physical state with a predetermined threshold, and updates the reference values for every driving cycle.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a driving support device provided in a vehicle such as an automobile.

Background Art

[0002] In vehicles such as automobiles, various driving support devices have been proposed that present information useful for vehicle operation to passengers such as drivers, or automate part of the driving operation to reduce the driver's burden. In addition, in such a driving support device, it has been proposed to detect the state of the passenger and reflect it in the content of the driving support.

[0003] For example, in Patent Document 1, physical information such as a driver's pulse, blood pressure, electroencephalogram, and visual field response ability is acquired, and when an abnormal value of the physical information occurs, a message for notifying the driver is output, and when a physical condition in which continued driving becomes a burden is determined, a safe driving support device that stops the vehicle or switches to automatic driving and drives to a hospital or the like is described. Patent Document 2 describes monitoring the driver's physical information and determining whether the driver's health state is normal based on the monitored physical information and a predetermined determination criterion. Further, when a predetermined amount of the monitored past physical information is accumulated, a second determination criterion is determined based on the accumulated past physical information, and thereafter, the driver's health state is determined based on the second determination criterion and the monitored physical information. Patent Document 3 describes a driving support device that learns a personal driver model unique to a driver based on the driver's driving data, analyzes the current emotional state of the driver based on the voice data of the driver included in the driving data, and determines vehicle control processing recommended according to the analyzed emotional state based on the personal driver model. In addition, it is described that a general driver model provided from big data is referred to in the analysis of the emotional state. Patent Document 4 describes a driving state estimation device comprising a steering-type electrocardiograph, a steering-type sweat meter, a seat-type electrocardiograph, a driver physiological state estimation unit, a driving load estimation unit, and a driving state estimation unit. It also describes that the driving state estimation unit compares work load and psychological burden and detects normal state, aimless state, aimless state resolved state, hyperarousal state, suppressed state, decreased arousal (low arousal) state, and arousal effort state. Patent Document 5 describes a psychological state determination device having a psychological determination processing unit that processes input signals to determine the driver's psychological state. It also describes that the input signals are obtained from a vehicle speed sensor, a driver's side door switch, an ignition key, a turn signal switch, a seat belt switch, an illumination light sensor, a wiper switch, and a driver identification device. Patent Document 6 describes a vehicle driving support device that detects the driver's emotions based on the driver's voice and the driver's operating state, and modifies the content of the support control according to the detected emotions of the driver. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2018-195249 [Patent Document 2] Japanese Patent Application Publication No. 11-328593 [Patent Document 3] Japanese Patent Publication No. 2018-169704 [Patent Document 4] Japanese Patent Publication No. 2015-33457 [Patent Document 5] Japanese Patent Publication No. 2003-48449 [Patent Document 6] International Publication No. WO2013 / 171970 [Overview of the project] [Problems that the invention aims to solve]

[0005] When attempting to incorporate the emotional state of the driver or other occupants into vehicle driving assistance systems, there are significant individual differences in the correlation between the parameters used to determine the emotional state and the actual emotional state. Furthermore, even for the same occupant, the same parameters do not always represent the same emotional state. Therefore, there are concerns that if the identification status is set using the occupant's past data or data from general drivers, the occupant's mental state may not be accurately determined. In view of the above-mentioned problems, the object of the present invention is to provide a driver assistance control that can perform driver assistance control that appropriately reflects the emotional state of the occupant. [Means for solving the problem]

[0006] To solve the above-mentioned problems, the driver assistance device of the present invention comprises a vehicle state detection unit for detecting the state of the vehicle, an occupant emotion discrimination unit for determining the emotions of the occupant, and a driver assistance control unit that performs driver assistance control including at least one of providing information to the occupant and driving assistance control of the vehicle in accordance with the output of at least one of the vehicle state detection unit and the occupant emotion discrimination unit, wherein the occupant emotion discrimination unit compares the difference between a reference value and the current value of a parameter indicating the occupant's physical state with a predetermined threshold to determine the occupant's emotions as unsuitable for driving. level discrimination And The aforementioned reference value is updated with each driving cycle. Furthermore, the driver assistance control unit shall, if the level of the driver's unsuitability for driving deteriorates beyond a predetermined level, perform at least one of the following: present information prompting the vehicle to stop and perform a forced stop control to automatically stop the vehicle; and, if it is determined that the occupant has taken a break for a predetermined period of time or longer, cancel the presentation of information prompting the vehicle to stop and the forced stop control that has been performed. It is characterized by the following. According to this, by updating the baseline values ​​of the parameters used to determine the occupant's emotions with each driving cycle (typically a period from turning on the vehicle's main power, through a series of drives, to turning off the main power), it is possible to determine the appropriate emotional state even when there are individual differences among occupants or daily changes in their physical condition. According to this, it is possible to prevent situations where a vehicle is driven while the driver is in an emotional state that is significantly unsuitable for driving, thus preventing dangerous situations from occurring. According to this, the convenience of the vehicle can be ensured by allowing it to run after the emotional state has improved. To solve the above-mentioned problems, the driver assistance device of the present invention comprises a vehicle state detection unit for detecting the state of the vehicle, an occupant emotion discrimination unit for determining the emotions of the occupant, and a driver assistance control unit that performs driver assistance control including at least one of providing information to the occupant and driving assistance control of the vehicle in accordance with the output of at least one of the vehicle state detection unit and the occupant emotion discrimination unit, wherein the occupant emotion discrimination unit has a function to determine unsuitable driving emotions by comparing the difference between a reference value and the current value of a parameter indicating the occupant's physical state with a predetermined threshold, and to update the reference value for each driving cycle, and has a function to determine the occupant's state of impatience and anger, and the driver assistance control unit is characterized in that, when the state of impatience and anger is determined, it suppresses the intervention frequency of at least some of the controls included in the driver assistance control compared to when the state of impatience and anger is not determined. According to this, by updating the baseline values ​​of the parameters used to determine the occupant's emotions with each driving cycle (typically a period from turning on the vehicle's main power, through a series of drives, to turning off the main power), it is possible to determine the appropriate emotional state even when there are individual differences among occupants or daily changes in their physical condition. According to this, frequent intervention by the driver assistance control system in the case of an occupant in a state of agitation and anger can prevent the occupant from feeling annoyed and worsening their state of agitation and anger.

[0007] In the present invention, the passenger emotion determination unit can be configured to update the reference value a plurality of times during the same driving cycle. According to this, it is possible to perform driving support control that appropriately reflects the changing emotions of the passenger that change moment by moment during vehicle operation.

Effect of the Invention

[0010] As described above, according to the present invention, it is possible to provide driving support control that can perform driving support control that appropriately reflects the emotional state of the passenger.

Brief Description of the Drawings

[0011] [Figure 1] FIG. 1 is a diagram schematically showing the system configuration of a vehicle having an embodiment of a driving support device to which the present invention is applied. [Figure 2] FIG. 20 is a diagram schematically showing the arrangement of a driver monitoring camera and a seat surface pressure sensor in the driving support device of the embodiment. [Figure 3] FIG. 23 is a flowchart showing the basic configuration of passenger emotion determination in the driving support device of the embodiment. [Figure 4] FIG. 26 is a diagram showing an example of the state of the fingers of a passenger holding a steering wheel. [Figure 5] FIG. 29 is a flowchart showing the content of basic control in the driving support device of the embodiment. [Figure 6] FIG. 32 is a diagram showing a change in the content of driving support control according to the emotional state of a passenger in the driving support device of the embodiment.

Mode for Carrying Out the Invention

[0012] Hereinafter, an embodiment of a driving support device to which the present invention is applied will be described. The driving support device of the embodiment is provided, for example, in a four-wheeled vehicle such as a passenger car. FIG. 44 is a diagram schematically showing the system configuration of a vehicle having the driving support device of the embodiment.

[0013] Vehicle 1 is equipped with an environmental recognition unit 100, a driver assistance control unit 200, an electric power steering (EPS) control unit 310, a power unit control unit 320, a brake control unit 330, an occupant status determination unit 400, and the like. Each of these units can be configured as a microcontroller having, for example, an information processing unit such as a CPU, a storage unit such as RAM or ROM, an input / output interface, and a bus to connect them. Furthermore, each unit can communicate with each other, either via an in-vehicle LAN such as a CAN communication system, or directly.

[0014] The environmental recognition unit 100 recognizes the environment around the vehicle based on the outputs of various sensors and the like. The environment surrounding the vehicle that is to be recognized includes, for example, the lane shape of the road on which vehicle 1 is traveling, the relative position and relative speed of various obstacles with respect to the vehicle, and information regarding the route that vehicle 1 will travel in the future, as well as information regarding accidents, congestion, and regulations. The environmental recognition sensor 100 is connected to a visible light camera 110, a millimeter-wave radar device 120, a laser scanner device 130, a high-precision map database 140, a positioning device 150, and the like.

[0015] The visible light camera device 110 is an imaging device that captures images of the area around the vehicle (front, rear, sides, etc.) using visible light cameras such as stereo cameras and single-lens cameras. The visible light camera device 110 has a function to perform image processing on the captured image and detect the presence or absence of objects around the vehicle, the relative position and speed of objects with respect to the vehicle, the lane shape, etc. The millimeter-wave radar system 120 is a radar system that uses radio waves in a frequency band of, for example, 30 to 300 GHz, and has the function of detecting the presence or absence of an object and the relative position of the object with respect to the vehicle. The laser scanner device 130 has a function to scan the area around the vehicle by, for example, irradiating it with pulsed near-infrared laser light, and to detect the presence or absence of an object, the relative position of the object to the vehicle, the shape of the object, etc., based on the presence or absence of reflected light and the time difference until the reflected light returns.

[0016] The high-precision map database 140 stores data related to high-precision 3D map data (HD map) within the range where vehicle 1 is expected to travel. This data, for example, includes 3D data of lane markings, shoulder edges, and lane dividers (so-called white lines), with a resolution of, for example, centimeters, and contains information on latitude, longitude, and altitude. The positioning device 150 includes, for example, a receiver for a quasi-zenith satellite system such as GPS, a vehicle-to-infrastructure communication device, a gyro sensor for autonomous navigation, and detects the current position of vehicle 1. The high-precision map database 140 and positioning device 150 work in cooperation with the environment recognition unit 100 and the driver assistance control unit 200 to function as a navigation system that provides route guidance to a predetermined destination and displays information such as lane changes and right / left turns. Such navigation system functionality is also included in the driver assistance control (information presentation control) of the present invention.

[0017] The driver assistance control unit 200 performs driver assistance control based on the environment around the vehicle recognized by the environment recognition unit 100, the driving state of the vehicle 1 recognized based on the output of each unit and sensor described later, and the emotional state of the occupant P. Driving assistance control includes, for example, information presentation control that provides information to the occupant P through images, sounds, vibrations, etc., and vehicle driving assistance control such as braking force control, power output control of the driving power source, and steering control.

[0018] The driver assistance control unit 200 is connected to an input / output device 210 and a communication device 220. The input / output device 210 includes, for example, an image display device that also serves as an input device such as a touch panel display, output devices such as a sound speaker, a vibration generator installed in a part that comes into contact with the occupant such as a seat, and input devices such as a physical switch and a voice microphone. The occupant P can use the input / output device 210 to make various settings related to driver assistance control. The communication device 220 communicates with a base station located outside the vehicle, for example, using a wireless communication line, and transmits and receives various types of data.

[0019] The electric power steering control unit 310 controls the steering device (not shown) that steers the steering wheels (typically the front wheels) of the vehicle 1, providing assist force in response to the steering operation of the occupant P, and steering force during automatic steering. The electric power steering control unit 310 is connected to a steering angle sensor 311, a torque sensor 312, a motor 313, and the like.

[0020] The steering angle sensor 311 is a sensor (steering angle detection unit) that detects the steering angle in the steering system. The torque sensor 312 is a sensor that detects the torque applied to the steering shaft to which a steering wheel (not shown) is connected, which is operated by the occupant P. The electric power steering control unit 310 generates assist force according to the torque detected by the torque sensor 312. Motor 313 is an electric actuator that provides assist force and steering force to the steering system and generates rack thrust. The output of the motor 313 is controlled by the electric power steering control unit 310.

[0021] The power unit control unit 320 comprehensively controls the vehicle 1's power source and its auxiliary equipment. As a power source for driving, for example, an internal combustion engine (ICE), an electric motor, or an engine-electric motor hybrid system can be used. The power unit control unit 320 sets the required torque based on, for example, the amount of accelerator pedal operation (not shown), and controls the driving power source so that the actual torque generated by the driving power source matches the required torque.

[0022] The brake control unit 330 controls the braking force of the brake devices provided on each of the front, rear, left, and right wheels of the vehicle individually (for each wheel). The braking system can be configured, for example, to include a hydraulic disc brake. The brake control unit 330 is connected to the hydraulic control unit 331, vehicle speed sensor 332, acceleration sensor 333, yaw rate sensor 334, and the like.

[0023] The hydraulic control unit 331 is a hydraulic pressure control device that individually adjusts the brake fluid pressure of the wheel cylinders (not shown) of each wheel. The hydraulic control unit 331 includes an electric pump for pressurizing the brake fluid, and pressure boosting valves, pressure reducing valves, and pressure holding valves for controlling the brake fluid pressure in each wheel cylinder.

[0024] The hydraulic control unit 331 is connected to a master cylinder, wheel cylinders, etc. (not shown) via brake fluid piping. The master cylinder pressurizes the brake fluid in response to the driver's operation of the brake pedal (not shown). The brake fluid pressure generated by the master cylinder is transmitted to the wheel cylinder via the hydraulic control unit 331. The hydraulic control unit 331 has the function of overriding the brake fluid pressure generated by the master cylinder to increase or decrease the brake fluid pressure of each wheel cylinder. A wheel cylinder is installed on each wheel and, for example, presses the brake pad against the disc rotor, generating a frictional force (braking force) corresponding to the brake fluid pressure.

[0025] The vehicle speed sensor 332 is installed in the hub portion that rotatably supports each wheel, and is a sensor that generates a vehicle speed signal corresponding to the rotational speed of each wheel. The acceleration sensor 333 is a sensor (acceleration / deceleration detection unit) that detects acceleration acting on the vehicle body in the longitudinal direction and the lateral direction (vehicle width direction). The yaw rate sensor 334 is a sensor that detects the yaw rate, which is the rotational (spinning) angular velocity of the vehicle body around its vertical axis. The electric power steering control unit 310, power unit control unit 320, and brake control unit 330 described above function as a driving state detection unit that detects the driving state of vehicle 1 in cooperation with the environment recognition unit 100.

[0026] The occupant status determination unit 400 determines the conscious state, emotional state, health state, etc., of occupant P (typically the driver). The occupant status group unit 400 is connected to a driver monitoring camera 401, a seat pressure sensor 402, and the like. The driver monitoring camera 401 and seat pressure sensor 402 work in cooperation with the occupant state determination unit 400 to function as an occupant monitoring unit.

[0027] Figure 2 is a schematic diagram showing the arrangement of the driver monitoring camera 401 and the seat pressure sensor 402 in the driving assistance device of the embodiment. Vehicle 1 has a seat S in which occupant P is seated. The seat S includes a seat cushion S1, a backrest S2, a headrest S3, and the like. The seat cushion S1 is the seat surface on which the occupant P's thighs, buttocks, etc., rest. The backrest S2 is the part that contacts the occupant's back and supports the occupant's upper body. The backrest S2 protrudes upward and diagonally backward from near the rear of the seat cushion S1. The headrest S3 is positioned at the rear of the occupant's head and is the part that holds the head in place when the head is reclined. The headrest S3 is positioned to protrude upward from the upper edge of the backrest S2.

[0028] The driver monitoring camera 401 is an imaging device that captures images of the occupant P from the front of the vehicle. The driver monitoring camera 401 is positioned to include the occupant P's face and their hands, while they are gripping the steering wheel SW, within its imaging field of view. The driver monitoring camera 401 includes, for example, a solid-state image sensor such as a CMOS or CCD, an optical system such as a lens group that forms an image of a subject on the solid-state image sensor, a drive circuit for the solid-state image sensor, and an output processing circuit.

[0029] The seat pressure sensor 402 is installed on the seat cushion S1 and measures the distribution of surface pressure received by the upper surface of the seat cushion S1 from the occupant P. Furthermore, the occupant status determination unit 400 may also be configured to include sensors other than those mentioned above. For example, sensors may be provided to acquire various biological information (so-called vital signs) such as the crew's heart rate, body temperature, blood pressure, and blood oxygen saturation.

[0030] The occupant state determination unit 400 has the function of determining the emotions of occupant P based on information about the environment around the vehicle recognized by the environment recognition unit 100, the driving state of the vehicle 1 including information detected by various sensors, and the occupant state detected by the driver monitoring camera 401, seat pressure sensor 402, etc. This point will be explained in detail below. Specifically, the occupant state determination unit 400 has the function of determining unsuitable driving emotions, such as distracted state, anxious state, and restless / angry state. Figure 3 is a flowchart showing the basic configuration for occupant emotion discrimination in the driver assistance system of the embodiment. The following explains each step in order.

[0031] <Step S101: Start monitoring driving and occupant conditions> The occupant status determination unit 400 starts monitoring the driving status and occupant status as described above after a predetermined initialization process. The initialization process is a process that initializes various parameters used for determining the emotions of the occupants, and is usually performed at the start of the driving cycle of vehicle 1 (a series of periods from the main power of vehicle 1 being turned on, the start of driving, arrival at the destination, and the main power being turned off). For example, the occupant status determination unit 400 can determine the start of a driving cycle when the main power of the vehicle 1 is turned on (ignition on), occupant P is wearing a seatbelt, and the vehicle is ready to drive (for example, by shifting the transmission to the driving range).

[0032] For example, the occupant state determination unit 400 sets the initialization values ​​of parameters used for emotion determination, such as the distribution of seat pressure detected by the seat pressure sensor 402 and the distance g between the steering wheel SW and the occupant's fingers (see Figure 4), which is obtained by image processing from images captured by the driver monitoring camera 401, based on the average value over a predetermined period (for example, 5 minutes) when driving at a low speed of 10 km / h or less or when stopped. Furthermore, parameters used for emotion discrimination include, for example, the amount of eye movement and the amount of head (face) orientation movement during normal times. These parameters are also initialized (initialized values ​​are obtained) in a similar manner. The initialization values ​​set here serve as baseline values ​​for parameters indicating the occupant's physical condition. The emotion discrimination described below is basically performed by comparing the difference between the initialization value (baseline value) of each parameter and the threshold value with a predetermined threshold.

[0033] If a predetermined amount of data cannot be collected within a predetermined period, or if measurement cannot be performed when the vehicle is in a horizontal plane, the system will correct or reacquire initialization values ​​to improve accuracy, provided that the predetermined conditions are met. The predetermined conditions include, for example, that the traffic light in front of the vehicle is stopped (red light on), that the vehicle speed is 0 km / h (including extremely low speeds where vehicle speed detection is impossible), that the steering angle detected by the steering angle sensor 311 is approximately 0, and that the acceleration in the longitudinal and lateral directions detected by the acceleration sensor 333 is 0 (meaning the vehicle is stationary on a horizontal plane). Note that this type of parameter initialization may be performed multiple times during a single driving cycle. After the monitoring of the driving condition and occupant condition begins, proceed to step S02.

[0034] <Step S102: Beware of Distracted Attention> The crew state determination unit 400 initiates distraction alert, which focuses on monitoring parameters used to determine the distracted state of crew member P. Focused monitoring includes, for example, increasing the amount of data by improving the time resolution of data acquisition compared to normal (non-alert) conditions. Distraction can be identified, for example, based on factors such as the fixation of the occupant P's field of vision and fluctuations (variations) in the distance to the vehicle ahead. Field of view fixation can be determined, for example, based on the driver's gaze direction detected from images captured by the driver monitoring camera 401.

[0035] For example, if a state in which the direction of gaze does not change by more than a predetermined amount (typically, a state in which the amount of displacement in the direction of gaze is less than or equal to a small threshold) continues for a predetermined period of time or longer, it can be determined that the person is in a state of distraction. Fluctuations in the distance between your vehicle and the vehicle ahead can be determined, for example, based on the change in the relative distance of the vehicle ahead to your vehicle, as detected by the environmental recognition unit 100. For example, if the following distance fluctuates by more than a predetermined value within a predetermined period, it can be determined that the driver is in a state of distraction. Then proceed to step S103.

[0036] <Step S103: Determining the state of distractedness> The crew state determination unit 400 detects whether or not crew member P is in a state of distraction within a predetermined distraction alert period (for example, within 10 minutes) using at least one of the determination methods described above. After determining whether or not the patient is in a state of distraction, the process proceeds to step S104.

[0037] <Step S104: Determining the conditions for alerting about a state of anxiety and confusion> The crew state determination unit 400 determines whether predetermined conditions for determining crew member P's state of anxiety and confusion have been met while the crew member is on alert for distracted state. Conditions for identifying a state of anxiety and confusion (events that suggest a high probability of an anxious and confused state) can be used to determine this, for example, based on the frequency of up-and-down movement (shaking) of crew member P's head (face direction) and fluctuations in the direction of their gaze.

[0038] If the head is moving up and down frequently, it is thought that occupant P is frequently looking at things like the navigation screen or mirrors, and it is presumed that they are not fully aware of their surroundings or are feeling anxious about their surroundings. For example, if head shaking (e.g., reversal of direction of movement) occurs more than a predetermined number of times (e.g., 3 times) within a predetermined time (e.g., 3 seconds), a state of anxiety and confusion should be identified. Similarly, if there are frequent changes in the direction of the crew's gaze, it can be inferred that the crew is either not fully aware of their surroundings or is feeling anxious about their surroundings. For example, if the blurring of the central visual field (the range scanned by the central visual field) is more than three times wider than when waiting at a traffic light, a state of anxiety and confusion should be identified.

[0039] Furthermore, the conditions for identifying a state of anxiety and confusion can be determined based on the driving conditions. For example, if the driver assistance control unit 200 determines that the vehicle is swerving based on the change in the vehicle's lateral position within the lane as recognized by the environmental recognition unit 100, it should also determine that the vehicle is in a state of anxiety and confusion if the change in the distance between the vehicle and the vehicle in front exceeds a predetermined value within a predetermined period. If it is determined that a state of anxiety and confusion should be identified, proceed to step S105; otherwise, proceed to step S107.

[0040] <Step S105: Beware of anxiety and confusion> The crew state determination unit 400 initiates an anxiety / confusion state alert, focusing its monitoring on parameters used to determine the anxiety / confusion state of crew member P. The determination of an anxious or confused state can be made, for example, based on the seat pressure distribution of the seat cushion S1 detected by the seat pressure sensor 402 (a parameter indicating the seating posture of the occupant P), and the distance between the rim of the steering wheel SW and the fingers of the occupant P, which is detected based on the image captured by the driver monitoring camera 401. Regarding seat pressure, for example, if there are fluctuations in the seat pressure distribution (changes in sitting posture) that exceed a predetermined level at a predetermined frequency (e.g., three or more times in 10 minutes), it can be determined that the person is in a state of anxiety and confusion. Furthermore, if the occupant's center of gravity shifts backward by more than a predetermined amount (for example, more than 2% relative to the most recent initialization value), it can be determined that the occupant is in a state of tension and a kind of anxious or confused state.

[0041] Figure 4 shows an example of the position of the occupant's fingers while holding the steering wheel. Figure 4(a) shows a situation where the crew is not under stress. Generally, when not under tension and in a relaxed state, occupant P does not grip the rim of the steering wheel SW tightly. In such cases, a gap g is often observed between the steering wheel SW and the thumb T. Note that while the thumb (T) is used here, the distance between other fingers and the steering wheel switches (SW) could also be used as a parameter.

[0042] Figure 4(b) shows the situation when the crew is under stress. Generally, when under stress (including anxiety, confusion, or anger), occupant P grips the rim of the steering wheel SW tightly, so the distance g becomes smaller or disappears compared to when not under stress. The occupant state determination unit 400 can determine that occupant P is gripping the steering wheel SW tightly and is in a state of anxiety and confusion if the difference between the current value of the gap g detected based on the image captured by the driver monitoring camera 401 and the initial value of the gap g becomes small enough to exceed a preset threshold. Then proceed to step S106.

[0043] <Step S106: Discrimination of Anxiety and Confusion> The crew state determination unit 400 determines whether or not crew member P is in an anxious or confused state within a predetermined anxiety / confusion state alert period (for example, within 10 minutes) using at least one of the determination methods described above. After determining whether or not the person is in a state of anxiety and confusion, proceed to step S107.

[0044] <Step S107: Determining the conditions for alerting about a state of impatience and anger> The crew state determination unit 400 determines whether predetermined conditions for determining crew member P's state of agitation and anger have been met while the system is on alert for anxiety and confusion. For example, if the gap g between the steering wheel SW and the thumb T falls below a predetermined value, the system should determine that the patient is in a state of impatience and anger, and proceed to step S108; otherwise, the series of processes is terminated.

[0045] <Step S108: Beware of state of impatience and anger> The crew state determination unit 400 initiates an impatience and rage state alert, focusing its monitoring on parameters used to determine the impatience and rage state of crew member P. The state of impatience and anger can be determined, for example, based on the simultaneous occurrence of rapid acceleration / deceleration and steering operations exceeding a predetermined level in vehicle 1 (a typical example of reckless driving). The occupant state determination unit 400 can determine that the occupant is in a state of agitation or anger, for example, when the steering angle sensor 311 detects a change in steering angle exceeding a predetermined threshold, and at the same time, the acceleration sensor 333 detects a change in longitudinal acceleration (sudden acceleration or sudden deceleration) exceeding a predetermined threshold. Then proceed to step S109.

[0046] <Step S109: Determining the state of impatience and anger> The crew state determination unit 400 determines, using the determination method described above, whether or not crew member P is in a state of agitation and anger within a predetermined period of alert for anxiety and confusion (for example, within 10 minutes). After determining whether or not the user is in a state of anxiety or anger, the series of processes is terminated. In addition, in the emotion discrimination of this embodiment, it is possible that multiple emotional states from among distracted state, anxious state, and impatient / angry state may be detected simultaneously.

[0047] Generally speaking, it is said that a person's attention span can last for about 15 minutes. In driving a car, which requires the use of multiple abilities, a 10-minute drive equivalent to a 5km journey on a public road often results in some kind of change in emotional state. In particular, during periods of high stress, there is a concern that drivers may become distracted, leading to frequent oversights while driving, or that their driving may deteriorate, resulting in accidents. In contrast, the crew status determination unit 400 manages the sensing system in units of a predetermined warning period (for example, 10 minutes).

[0048] Specifically, if suspicion of a state of anxiety and confusion arises while vigilance for a state of distraction is being maintained for a predetermined period (for example, 10 minutes), vigilance for the state of anxiety and confusion is initiated for a new period (for example, 10 minutes). Subsequently, if suspicion of a state of agitation and anger arises, vigilance for the state of agitation and anger is initiated for a new period (for example, 10 minutes). This allows for accurate determination (estimation) of the emotional state of crew member P, even if their emotions shift in a direction that is inappropriate for the driving operation.

[0049] In addition to the emotion discrimination described above, the occupant state discrimination unit 400 determines that if the frequency of the driver's brake operations detected by the brake control unit 330 is below a predetermined value and extremely infrequent, and if the deceleration of the vehicle 1 detected by the acceleration sensor 333 is below a predetermined value and no deceleration operation is substantially performed, then the occupant P's driving ability is significantly reduced, and the driver support control unit 200 performs a forced stop control to bring the vehicle 1 to a safe location such as the shoulder of the road by automatic driving.

[0050] Furthermore, if the status determination unit 400 detects any of the following events, it will suspect that the occupant P is unable to drive due to an abnormal physical condition (such as a deterioration in physical condition that leads to a decrease in the level of consciousness, or intoxication due to alcohol or drugs), and will execute the aforementioned forced stop control, as well as use the communication device 220 to notify servers of designated organizations such as the police, ambulance services, and medical facilities. (a) Crew member P's field of vision is fixed (e.g., for 3 seconds or more) (b) If, based on the images captured by the driver monitoring camera 401, it is determined that the occupant P has closed eyes (i.e., is deemed unconscious) (c) When the driver assistance control unit 200 detects a predetermined meandering motion (d) When the change in the distance between the vehicle and the vehicle in front exceeds a predetermined threshold for detecting poor physical condition. (e) When the amount of backward or downward movement of the occupant P's center of gravity detected by the seat pressure sensor 402 is greater than or equal to a predetermined threshold for determining poor physical condition.

[0051] Next, we will explain the content of the driver assistance control that corresponds to the results of the emotion discrimination (estimation) of occupant P described above. Figure 5 is a flowchart showing the basic control content of the driver assistance system in the embodiment. The following explains each step in order.

[0052] <Step S201: Initialize emotion discrimination parameters> The occupant state determination unit 400 initializes each parameter used for emotion determination (estimation) in response to the start of the driving cycle of vehicle 1 (for example, when the main power of vehicle 1 is turned on (ignition on)). Then proceed to step S202.

[0053] <Step S202: Start monitoring driving conditions and emotional state> The driver assistance control unit 200 starts monitoring the driving conditions of vehicle 1, including vehicle speed, longitudinal acceleration, distance to the vehicle in front, lateral position within the lane, presence or absence of obstacles, and, if there are obstacles, their relative position and speed relative to vehicle 1. The occupant status determination unit 400 begins monitoring the emotional state of the occupant P (driver) as described above. Then proceed to step S203.

[0054] <Step S203: Driving condition sensing judgment> The driver assistance control unit 200 senses events that require driver assistance regarding the driving state, and determines whether or not the occupant state determination unit 400 sensed any events related to emotional state (determination of at least one of distracted state, anxious state, or agitated / angry state). Examples of events that require driver assistance regarding driving conditions include the following: (a) Vehicle 1 swerving or wobbling (a deviation of the lateral position within the lane that exceeds a specified limit) (b) Lane departure of Vehicle 1 (c) Approaching the distance between vehicle 1 and the vehicle in front of it to a predetermined value or less (d) Detection of obstacles (risk objects) around the vehicle Here, the risk objects can include, for example, other vehicles, pedestrians, bicycles, and various other objects that obstruct the passage of vehicle 1. Furthermore, the objects of risk include other vehicles present to the rear and sides when changing lanes. (e) exceeding the speed limit (f) Various information regarding traffic congestion, restrictions, accidents, etc. along the route that Vehicle 1 is scheduled to travel. If an event occurs that requires driver assistance only in relation to the driving conditions, proceed to step S210; otherwise, proceed to step S204.

[0055] <Step S204: Emotional State Sensing Judgment> The driver assistance control unit 200 determines whether the occupant state determination unit 400 sensed an event related to emotional state (at least one of distracted state, anxious state, or irritable state) and whether it sensed an event requiring driver assistance regarding the driving state. If only events related to emotional states are sensed, proceed to step S220; otherwise, proceed to step S205.

[0056] <Step S205: Driving condition / emotional state sensing and judgment> The driver assistance control unit 200 senses events that require driver assistance due to the driving conditions, and determines whether the occupant state determination unit 400 has sensed events related to emotional states unsuitable for driving (distracted state, anxious state, restless state). If events related to both driving conditions and emotional states are sensed, the process proceeds to step S230; otherwise, the series of processes ends (returns).

[0057] <Step S210: Warning display output> The driver assistance control unit 200 causes the input / output device 210 to output a display to alert the occupant P regarding the identified events among the above-mentioned events that require driver assistance in relation to driving conditions. After that, the series of processes will be terminated.

[0058] <Step S220: Sensing threshold update and warning display output> The driver assistance control unit 200 updates the threshold for sensing events requiring driver assistance regarding the driving state in a way that makes it easier to make a judgment (lower the threshold). Furthermore, the driver assistance control unit 200 causes the input / output device 210 to output a display to alert the occupant P regarding the identified events among the above-mentioned events that require driver assistance in relation to driving conditions. Then proceed to step S221.

[0059] <Step S221: Re-sensing decision based on update threshold> The driver assistance control unit 200 determines, based on the updated threshold in step S220, whether or not an event requiring driver assistance regarding the driving state has been sensed again. If an event requiring driver assistance is sensed again regarding the driving conditions, proceed to step S222; otherwise, terminate the series of processes.

[0060] <Step S222: Forced Stop Warning Display Output> The driver assistance control unit 200 outputs (presents) a warning to the occupant P via the input / output device 210 that the emotional state and driving condition have deteriorated and that the vehicle 1 should be stopped. Then proceed to step S223.

[0061] <Step S223: Re-sensing decision based on the re-update threshold> The driver assistance control unit 200 determines whether an event requiring driver assistance has been sensed again regarding the driving state, using a threshold that has been updated in step S220, further updated (threshold lowered) in a direction that makes the determination easier to obtain. If an event requiring driver assistance is sensed again regarding the driving conditions, proceed to step S224; otherwise, terminate the series of processes.

[0062] <Step S224: Execute forced stop control> The driver assistance control unit 200 performs a forced stop control to bring the vehicle 1 to a safe location such as the shoulder of the road through automatic driving, where the driver assistance control unit 200 controls steering and acceleration / deceleration. After that, the series of processes will be terminated.

[0063] <Step S230: Sensing threshold update / Forced stop warning display output> The driver assistance control unit 200 updates the threshold for sensing events that require driver assistance due to driving conditions in a way that makes it easier to make a judgment (lower the threshold). Furthermore, the driver assistance control unit 200 outputs (presents) a warning to the occupant P via the input / output device 210 that the emotional state and driving condition have deteriorated and that the vehicle 1 should be stopped. Then proceed to step S231.

[0064] <Step S231: Re-sensing decision based on update threshold> The driver assistance control unit 200 determines, based on the updated threshold in step S220, whether or not an event requiring driver assistance control intervention has been sensed again regarding the driving conditions. If an event requiring intervention by driver assistance control is sensed again regarding the driving conditions, proceed to step S232; otherwise, terminate the series of processes.

[0065] <Step S232: Execute forced stop control> The driver assistance control unit 200 performs a forced stop control to bring the vehicle 1 to a safe location such as the shoulder of the road through autonomous driving. After that, the series of processes will be terminated.

[0066] If occupant P stops vehicle 1 in response to the aforementioned forced stop warning, the driver assistance control unit 200 prohibits driving vehicle 1 until it is estimated that occupant P's emotional state has recovered to a state suitable for driving, and then permits driving vehicle 1. For example, if the vehicle speed is 0 km / h and at least one of the following is detected: the door or side door glass is open, the seat S is reclined, or the transmission is kept in a non-driving range for a predetermined time (for example, 5 minutes) or longer, it is assumed that occupant P has taken a rest for a predetermined period of time or longer and that occupant P's emotional state has recovered (refreshed), and driving of vehicle 1 is permitted.

[0067] Furthermore, the same process is performed when the driver assistance control unit 200 forcibly stops the vehicle 1 (automatic stop). Furthermore, if the emotional state deteriorates beyond a predetermined level, the communication device 220 will be used to notify relevant parties such as family members, medical institutions, and specialized organizations.

[0068] Furthermore, in the driving assistance device of the embodiment, different driving assistance is provided depending on the type of emotional state that is identified. Figure 6 is a diagram showing how the content of the driver assistance control in the driver assistance device of the embodiment changes according to the emotional state of the occupant. The following explains each step in order.

[0069] <Step S301: Start monitoring based on the initialization value> The crew state determination unit 400 initializes the parameters related to the determination of crew member P's emotions as described above, and then begins monitoring each parameter. Then proceed to step S302.

[0070] <Step S302: Assessment of Distracted State> The driver assistance control unit 200 determines whether the occupant state determination unit 400 has determined that occupant P is in a state of distraction (step S103). If a state of distraction is identified, proceed to step S310; otherwise, proceed to step S303.

[0071] <Step S303: Assessment of Anxiety and Confusion> The driver assistance control unit 200 determines whether the occupant state determination unit 400 has determined (step S106) that occupant P is in a state of anxiety or confusion. If a state of distraction is identified, proceed to step S320; otherwise, proceed to step S304.

[0072] <Step S304: Assessment of the state of impatience and anger> The driver assistance control unit 200 determines whether the occupant state determination unit 400 has determined (step S106) that occupant P is in a state of agitation or anger. If a state of distraction is detected, proceed to step S330; otherwise, proceed to step S305.

[0073] <Step S305: Normal control> The driver assistance control unit 200 performs normal driver assistance control, assuming that the occupant P's emotional state is normal (suitable for driving). After that, the series of processes will be terminated.

[0074] <Step S310: Information display control - Specialized for pop-up information> The driver assistance control unit 200 performs information presentation control, which presents various types of information to the occupant, using the input / output device 210. The information presented to the crew may include, for example, the following: (a) Warning of the distance to the vehicle in front (warning when the distance is excessively small or fluctuating) (b) Speed ​​control warning (warning in case of speeding) (c) Driving lane information (warnings in case of weaving or lane departure, and guidance to encourage lane changes) (d) Sudden obstacle information (warning when a risk object with a specified or higher probability of collision appears in front of the vehicle) (e) Route environment information (information on road surface conditions including curvature, gradient, position of the sun, and temperature drop on the route the vehicle is scheduled to travel) The driver assistance control unit 200 reduces the output frequency of information related to (a) to (c) and (e) compared to normal operation, and focuses on outputting information related to (d) (specializing in sudden appearance information related to the environment in front of the vehicle and environmental information presentation control). Then proceed to step S311.

[0075] <Step S311: Lowering the intervention threshold for driving support control and vehicle behavior> The driver assistance control unit 200 performs driving assistance control by intervening in the control of the vehicle's steering system, power unit, and brake system according to the environment around the vehicle detected by the environment recognition unit 100 and the driving state of the vehicle detected by sensors. Examples of driving assistance control include the following: (a) Collision damage mitigation braking control when a risk object with a predetermined or higher probability of collision appears in front of the vehicle. (b) Lane departure prevention control that generates a yaw moment in the direction of restoration when the vehicle deviates from its lane or is highly likely to deviate from its lane. The driver assistance control unit 200 lowers the control intervention threshold for lane departure prevention control, which is primarily a control mechanism related to the vehicle's behavior. (It changes the threshold so that lane departure prevention control intervenes earlier.) After that, the series of processes is terminated (return).

[0076] <Step S320: Specialized for information display control and driving line information> The driver assistance control unit 200 reduces the output frequency of information related to (a), (b), and (d) from the information related to (a) to (d) described in step S310 above, compared to normal operation, and focuses on displaying information related to (c) (specializing in driving line information). Then proceed to step S321.

[0077] <Step S321: Driving support control - Decreasing the control intervention threshold for the surrounding environment> The driver assistance control unit 200 lowers the intervention threshold of the collision damage mitigation braking control described in step S311 above, which is primarily a control (environment-responsive driver assistance control) caused by changes in the environment surrounding the vehicle. (The threshold is changed to allow the collision damage mitigation braking control to intervene earlier.) After that, the series of processes will be terminated.

[0078] <Step S330: Specialized for information presentation control and route environment information> The driver assistance control unit 200 reduces the output frequency of information (a) to (c) from the information (a) to (e) described in step S310 above compared to normal operation, and outputs route environment information (e) (typically, the situation at a distance of 1 km or more ahead) at a higher frequency and with greater emphasis compared to normal operation (specializing in route environment information). Furthermore, the information regarding the sudden appearance of an object (d) will be output in the same way as under normal circumstances. In this case, the driving assistance control will be performed in the same way as under normal circumstances. After that, the series of processes will be terminated.

[0079] According to the embodiments described above, the following effects can be obtained. (1) By initializing the baseline values ​​of the parameters used to determine the emotions of occupant P for each driving cycle, it is possible to determine the appropriate emotional state even if there are individual differences among occupant P or daily changes in occupant P's physical condition. (2) By updating the reference values ​​of the parameters multiple times during the same driving cycle, it is possible to perform driving support control that appropriately reflects the changes in the emotions of occupant P, which change moment by moment while driving vehicle 1. (3) After an anxious / confused state or a state of agitation / anger is identified, if the threshold is lowered and these emotional states are identified again, a forced stop warning prompting the vehicle to stop will be issued, and a forced stop control will be implemented to automatically stop the vehicle. This prevents the vehicle 1 from being driven in an emotional state that is significantly unsuitable for driving, thus preventing a dangerous situation from occurring. (4) After issuing a forced stop warning or forced stop control, if it is determined that the occupant has taken a rest for a period of time longer than specified, these measures can be canceled, and the vehicle can be allowed to run after the occupant's emotional state has improved, thereby ensuring the convenience of the vehicle. (5) When a state of agitation and anger in crew member P is detected, the frequency of intervention with information presentation control is reduced compared to when the state is not detected. This prevents crew member P from feeling annoyed and worsening their state of agitation and anger due to the frequent presentation of information to a crew member in an agitated and anger state. Furthermore, even in this case, by presenting route environment information that is easily accepted by occupant P and beneficial to occupant P, appropriate driving support can be provided while suppressing the deterioration of occupant P's emotional state.

[0080] (modified version) The present invention is not limited to the embodiments described above, and various modifications and changes are possible, all of which fall within the technical scope of the present invention. (1) The configuration of the driver assistance system and the vehicle on which it is installed is not limited to the embodiments described above and can be modified as appropriate. For example, the functions of each unit, the types of sensors installed, and the types of events to be sensed are not particularly limited and can be changed as appropriate. (2) The specific details of the driver assistance control are not limited to those of the embodiment, and other functions may be added. Also, some functions in the embodiment may be omitted. (3) The parameters and discrimination logic used to determine the emotional state of the crew members are not limited to the configuration of the embodiment and can be changed as appropriate. For example, in this embodiment, the state of anxiety, confusion, and agitation are determined based on the distance between the occupant's thumb and the rim of the steering wheel. However, the finger used as the reference for measuring the distance is not limited to the thumb; other fingers may also be used. Alternatively, the distance between the palm and the rim may be used as a substitute. Alternatively, instead of such intervals, a contact pressure sensor may be installed on the rim of the steering wheel to estimate the degree to which the occupant grips the steering wheel based on the gripping force (contact pressure) of the occupant's fingers and palm. (4) In this embodiment, the acceleration and deceleration of the vehicle to determine the state of impatience and anger is detected using an acceleration sensor, but the acceleration and deceleration of the vehicle may be detected or estimated by other methods. For example, instead of deceleration, parameters such as braking force, braking amount, hydraulic fluid pressure in hydraulic brakes, absorption torque in regenerative braking, vehicle speed reduction amount, and reduction rate may be used. Alternatively, instead of acceleration, parameters such as accelerator pedal input, target torque of the driving power source, or estimated actual torque may be used. [Explanation of symbols]

[0081] 1 Vehicle 100 Environmental Recognition Units 110 Visible light camera system 120 Millimeter-wave radar system 130 Laser scanner equipment 140 High-precision map database 150 Positioning device 200 Driver support control unit 210 Input / Output Devices 220 Communication Devices 310 Electric Power Steering Control Unit 311 Steering angle sensor 312 Steering angle sensor 313 Motor 320 Power Unit Control Unit 330 Brake control unit 331 Hydraulic Control Unit 332 Vehicle speed sensor 333 Acceleration sensor 334 Yaw rate sensor 400 Crew status determination unit 401 Driver monitoring camera 402 Seat pressure sensor P Passenger T Thumb SW steering wheel g interval S Seat S1 Seat Cushion S2 Backrest S3 Headrest

Claims

1. A vehicle status detection unit that detects the vehicle's status, A crew emotion discrimination unit that determines the emotions of the crew, A driver assistance control unit performs driver assistance control, including at least one of presenting information to the occupant and driving assistance control of the vehicle, in accordance with the output of at least one of the vehicle state detection unit and the occupant emotion discrimination unit. Equipped with, The occupant emotion determination unit determines the level of unsuitability for driving by comparing the difference between a reference value and the current value of a parameter indicating the occupant's physical condition with a predetermined threshold, and updates the reference value with each driving cycle. The driver assistance control unit shall, if the level of the driver's unsuitability for driving deteriorates beyond a predetermined level, perform at least one of the following: present information prompting the vehicle to stop, or perform a forced stop control to automatically stop the vehicle; and if it is determined that the occupant has taken a break for a predetermined period of time or longer, cancel the presentation of information prompting the vehicle to stop, or the forced stop control that has been performed. A driver assistance system characterized by the following.

2. A vehicle status detection unit that detects the vehicle's status, A crew emotion discrimination unit that determines the emotions of the crew, A driver assistance control unit performs driver assistance control, including at least one of presenting information to the occupant and driving assistance control of the vehicle, in accordance with the output of at least one of the vehicle state detection unit and the occupant emotion discrimination unit. Equipped with, The occupant emotion discrimination unit has the function of determining whether the occupant is emotionally unsuitable for driving by comparing the difference between a reference value and the current value of a parameter indicating the occupant's physical condition with a predetermined threshold, and updating the reference value with each driving cycle, and also has the function of determining whether the occupant is in a state of agitation or anger. The driver assistance control unit, when the state of impatience and anger is detected, suppresses the frequency of intervention of at least some of the controls included in the driver assistance control compared to when the state of impatience and anger is not detected. A driver assistance system characterized by the following.

3. The occupant emotion discrimination unit updates the reference value multiple times during the same driving cycle. The driving support device according to claim 1 or 2, characterized by the following:

Images