Driving assistance device

Abstract

To maintain safe driving by performing support according to the feeling of a driver.SOLUTION: The driving assistance apparatus executes an emotion determination process of determining an emotion of the driver based on the acquired detection information, an assistance determination process of determining whether the driver needs driving assistance, an assistance type determination process of determining a type of driving assistance in accordance with the determined emotion when the driver needs driving assistance, and an assistance execution control process of executing the determined type of driving assistance.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a driving assistance device mounted on a vehicle. [Background technology]

[0002] The following Patent Document 1 describes a technique for estimating the emotion of a driver or a passenger and controlling the running of a vehicle based on the estimated emotion. Patent Document 2 listed below describes a driving assistance device that can determine whether a driving state is suitable for a driving environment and feed back the determination result. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-136922 [Patent Document 2] Japanese Patent Application Publication No. 2018-151911 Summary of the Invention [Problem to be solved by the invention]

[0004] A driver's emotional state affects safe driving. For example, there are various emotions such as anxiety, fear, irritation, joy, and laughter, but when these emotions become excessive, it tends to be more difficult to maintain safe driving than when driving calmly. On the other hand, in recent years, various driving assistance technologies have been developed, but it is desirable to provide appropriate driving assistance according to the level and severity of the driver's emotions.

[0005] Therefore, the present invention proposes a technology that can appropriately determine the emotional state of the driver and provide appropriate driving assistance as needed. [Means for solving the problem]

[0006] A driving assistance device according to one embodiment of the present invention includes one or more processors and a storage medium storing a program executed by the one or more processors, the program including one or more instructions that cause the one or more processors to execute an emotion determination process that determines the driver's emotion based on acquired detection information, an assistance determination process that determines whether the driver is in a state requiring driving assistance, an assistance type determination process that, if driving assistance is required, determines the type of driving assistance based on the determined emotion, and an assistance execution control process that executes the determined type of driving assistance. [Effects of the Invention]

[0007] According to the present invention, emotions are judged, and whether the emotional state is such that driving assistance is necessary is determined, and then driving assistance selected according to the emotion is provided. As a result, appropriate driving assistance according to the emotion is provided at an appropriate timing according to the emotional state, which contributes to promoting safe driving. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is an explanatory diagram of a configuration inside a vehicle according to an embodiment of the present invention; [Figure 2] FIG. 2 is an explanatory diagram of a functional configuration of the driving assistance device according to the embodiment. [Figure 3] 4 is a flowchart of a reference value setting process of the driving assistance device according to the embodiment. [Figure 4] 4 is a flowchart of a driving assistance control process according to the first embodiment. [Figure 5] 10 is a flowchart of a driving assistance control process according to a second embodiment. [Figure 6] 1 is an explanatory diagram of an example of emotion-based driving assistance control according to an embodiment; [Figure 7] 1 is an explanatory diagram of an example of emotion-based driving assistance control according to an embodiment; DETAILED DESCRIPTION OF THE INVENTION

[0009] <Vehicle configuration> 1 shows a vehicle 1 equipped with a control system 10 including a driving assistance device 2 according to the present embodiment. The vehicle 1 may be any of various types of vehicles, such as a gasoline engine vehicle, an electric vehicle, a hybrid vehicle, or a fuel cell vehicle.

[0010] The vehicle 1 includes a control system 10 including a driving assistance device 2, a driving mechanism 5, an electrical component 6 including a notification device 7, and various sensors. It should be noted that FIG. 1 shows only a portion of the configuration of the vehicle 1, and it goes without saying that the vehicle 1 is equipped with various configurations for traveling and driving.

[0011] The driving assistance device 2 is configured with one or more processors. Specifically, it is configured with one or more ECUs (Electronic Control Units) in the vehicle 1. The driving assistance device 2 may be configured as a driving assistance ECU specialized for driving assistance functions, or may be realized in another ECU. For example, the ECUs provided in the vehicle 1 may include a driving control ECU, a charge control ECU that controls charging of the on-board battery, an engine control ECU, a display control ECU that controls display of display devices (including meters, etc.) provided in the vehicle 1, an audio output control ECU that controls output of audio output devices, an autonomous driving ECU, an air conditioning control ECU, and various other ECUs. The driving assistance device 2 may be configured in any of these ECUs in the vehicle.

[0012] The ECU 3 shown separately from the driving support device 2 indicates an ECU other than the ECU that functions as the driving support device 2, among the various ECUs described above.

[0013] The driving assistance device 2 and other ECUs 3 are configured with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), non-volatile memory, an input / output interface, etc. The CPU executes various processes according to programs stored in the ROM and non-volatile memory. The RAM stores data and other information necessary for the CPU to execute various processes.

[0014] The driving assistance device 2 and the other ECUs 3 are connected to each other so as to be able to communicate with each other via a bus 4. For example, the bus 4 is configured as a bus for CAN (Controller Area Network (registered trademark)) communication, and allows the devices to send and receive information to and from each other.

[0015] The driving assistance device 2 and each ECU 3 are configured to be able to perform predetermined functions by installing software programs therein.

[0016] The driving assistance device 2 and other ECUs 3 constitute a control system 10 in the vehicle 1, and the operation of each part is controlled by the control system 10. That is, the driving mechanism 5, the electrical equipment part 6, the notification device 7, etc. are controlled.

[0017] The driving mechanism 5 collectively refers to all parts related to the driving of the vehicle 1. Specifically, it collectively refers to mechanisms related to the driving operation of the automobile, such as the engine or motor mechanism, accelerator mechanism, brake mechanism, transmission mechanism, power steering mechanism, and suspension mechanism. Driving operation devices for driving, such as the accelerator pedal, brake pedal, steering, and shift lever, are also included in the driving mechanism 5.

[0018] The electrical equipment section 6 comprehensively refers to all electrical equipment provided in the vehicle 1. Specific examples include an air conditioner, a navigation system, audio and other sound output devices, interior lights, display devices such as an MFD (Multi Function Display), power window devices, an electric seat mechanism, and a vibration mechanism. The vibration mechanism may be, for example, a mechanism that slightly vibrates driving operation means such as a steering wheel or pedals, or a mechanism that slightly vibrates a seat.

[0019] The notification device 7 is a comprehensive term that refers to any device that can notify the driver in some form, such as text, images, or sound. For example, it refers to the display unit of an MFD or navigation device, which is an electrical component, or the audio output unit of an audio or navigation device.

[0020] The driving mechanism 5, the electrical component 6, and the notification device 7 are shown as parts that the driving assistance device 2 causes to perform some operation or process depending on the emotional state of the driver. Specific examples of the driving mechanism 5, the electrical component 6, and the notification device 7 are not limited to those illustrated.

[0021] In FIG. 1, various sensors are shown, including an in-vehicle sensor unit 14, an out-vehicle sensor unit 15, a vehicle sensor unit 16, and a wearable sensor unit 17.

[0022] The interior sensor unit 14 is composed of various sensors that detect the status of occupants in the cabin of the vehicle 1 and the status inside the cabin. For example, the interior sensor unit 14 is composed of various sensors that constitute a DMS (Driver Monitoring System), and specifically, it is assumed that the interior sensor unit 14 includes an interior image sensor for detecting the driver's facial expression, gaze direction (eye movement), blinking, etc., an in-vehicle microphone for acquiring information on voices emitted by the driver and passengers, a pulse sensor for detecting the driver's pulse, a body temperature sensor for detecting the driver's body temperature, a humidity sensor for detecting the driver's sweating state, a temperature sensor for detecting the temperature inside the cabin, a sensor for detecting the pressure and pressure distribution of the seat, etc. Furthermore, for example, a fingerprint sensor provided on the steering wheel, a voiceprint sensor for detecting the driver's voice and analyzing the voiceprint, etc. may also be provided. The various sensors described above are merely examples, and the interior sensor unit 14 may include only some of the sensors described above, or may include sensors other than those described above. The interior sensor unit 14 may also include, for example, a weight sensor or a contact sensor for detecting the presence or absence of a passenger.

[0023] The external sensor unit 15 is composed of various sensors that detect the conditions outside the vehicle 1. For example, the external sensor unit 15 includes an external image capture sensor for detecting objects outside the vehicle, such as other vehicles, such as a preceding vehicle, pedestrians, and lanes, weather outside the vehicle, such as sunny, rainy, or snowy weather, and conditions such as flooding and snow accumulation, an external air temperature sensor for detecting the temperature outside the vehicle, and a gradient sensor, such as a gyro sensor, for example, for detecting the gradient of the road on which the vehicle is traveling. Information on other vehicles outside the vehicle, such as preceding vehicles, can also be used to determine traffic congestion, cutting in, vehicle-to-vehicle distance, and tailgating. The various sensors described above are merely examples, and the vehicle exterior sensor unit 15 may include only some of the sensors described above, or may include sensors other than those described above. For example, the vehicle exterior sensor unit 15 may include millimeter-wave radar or sonar for object detection.

[0024] The vehicle sensor unit 16 is composed of various sensors that detect the driving conditions of the vehicle 1, including the state of each part of the vehicle. For example, the vehicle sensor unit 16 includes a vehicle speed sensor that detects the speed of the vehicle, an acceleration sensor that detects acceleration, an engine rotation speed sensor that detects the engine rotation speed, an accelerator position sensor that detects the accelerator position, a steering angle sensor that detects the steering angle, a yaw rate sensor that detects the yaw rate, tire pressure sensors that constitute a TPMS (Tire Pressure Monitoring System), and a GNSS (Global Navigation Satellite System) sensor, such as a GPS (Global Positioning System) sensor, that receives information necessary for recognizing the position information of the vehicle 1. The vehicle driving operation is based on the driving operation of the driver. In other words, it is the operation of the driving operation devices such as the steering wheel, shift lever, accelerator pedal, and brake pedal. The vehicle sensor unit 16 also includes sensors that detect the amount of operation, number of operations, and frequency of operation of these driving operation devices by the driver. The above-mentioned various sensors are merely examples, and the vehicle sensor unit 16 may include only some of the above-mentioned sensors, or may include sensors other than those mentioned above.

[0025] The vehicle interior sensor unit 14, the vehicle exterior sensor unit 15, and the vehicle sensor unit 16 transmit their respective detection information to the control system 10, that is, the driving assistance device 2 and other ECUs 3.

[0026] The wearable sensor unit 17 refers to a sensor worn by the driver, and does not necessarily have to be equipped on the vehicle 1. For example, it is a body-worn sensing device such as a wristwatch. The wearable sensor unit 17 can detect the driver's pulse rate, respiratory rate, body temperature, sweat rate, skin temperature, brain waves, etc. The wearable sensor unit 17 may take any form, but the wearable sensor unit 17 is assumed to be capable of detecting at least some biological information of the driver and transmitting the detected biological information to the driving assistance device 2.

[0027] The driving assistance device 2 executes driving assistance control based on the detected information from the in-vehicle sensor unit 14, the out-vehicle sensor unit 15, the vehicle sensor unit 16, and the wearable sensor unit 17, as well as information notified from other ECUs 3. In this embodiment, the driving assistance device 2 particularly determines the driver's emotions and provides driving assistance according to the driver's emotional state. The content of the driving assistance is determined according to the determined emotional state.

[0028] 2 shows in block form the functions of the microprocessor serving as the driving assistance device 2. The microprocessor serving as the driving assistance device 2 includes a calculation unit 20 and a storage unit 30, and the calculation unit 20 realizes the functions shown in the figure by means of a program. These functions include an information acquisition unit 21, an allowable information calculation unit 22, a determination unit 23, and an assistance control unit 24. Note that the stimulus control unit 25 is a function related to the processing of the second embodiment described later in FIG. 5, and is not required for the processing of the first embodiment described later in FIG. 4, and will therefore be described later. The programs that cause the calculation unit 20 to realize the functions of the information acquisition unit 21, the allowable information calculation unit 22, the determination unit 23, the assistance control unit 24, and the stimulation control unit 25 are stored in the storage unit 30, for example.

[0029] There is no specific definition for the storage unit 30. For example, the storage unit 30 may be an internal memory of a microprocessor serving as the driving assistance device 2, or may be an external memory.

[0030] The information acquisition unit 21 has a function of acquiring detection information from the above-mentioned in-vehicle sensor unit 14, out-vehicle sensor unit 15, vehicle sensor unit 16, and wearable sensor unit 17 (hereinafter collectively referred to as "various sensors"). The information acquisition unit 21 mainly acquires from the various sensors information to be used for estimating the driver's emotions, such as biometric information such as the driver's pulse, respiratory rate, body temperature, and amount of sweat, information on facial expressions and gaze, information on the driver's operation of driving operation devices such as the steering wheel, brake pedal, and accelerator pedal, and information on the seat pressure distribution of the driver's seat.

[0031] Note that information other than the above may also be used to estimate the driver's emotions. For example, information from the external sensor unit 15, such as information on surrounding vehicles, traffic congestion, weather, road conditions, lighting conditions (morning sun, afternoon sun, etc.), night / daytime information, and information on the area in which the vehicle is traveling, can be used to estimate the driver's emotions.

[0032] The tolerance information calculation unit 22 has a function of calculating, for each individual driver, the tolerance range and degree of the detection information from the various sensors acquired by the information acquisition unit 21. For example, it calculates the upper limit or range of the biological information in the calm emotional state of the individual driver, the range of information on the seat pressure distribution, the range of driving operation, etc. based on learning processing.

[0033] Specifically, the permissible information calculation unit 22 calculates, as permissible information related to the biological information of the driver, for example, Heart rate limit Upper respiratory rate limit Upper limit of skin temperature change - Upper limit of sweat increase Upper limit of eye movement Upper limit of blink frequency These are set based on learning processing. If these are equal to or less than the upper limit values, they are information that serves as an index for estimating that the driver's emotions and condition are normal.

[0034] The tolerance information calculation unit 22 calculates, for example, the tolerance information of the pressure distribution of the seat as follows: Upper limit of difference from ideal pressure distribution Upper limit of the difference between the pressure distribution at the start of operation and the current pressure distribution Upper limit of pressure distribution difference between left and right and front and back - Upper limit of pressure distribution bias and deviation of center of gravity position These are set based on learning processing. If these are equal to or less than the upper limit values, they are information that serves as an index for estimating that the emotional state and condition of the driver are normal.

[0035] The allowable information calculation unit 22 calculates, as allowable information relating to the driving operation device, for example, - Upper limit of steering operation count per unit time - Upper limit of brake and accelerator operation count per unit time - Upper limit of braking and accelerating frequency relative to the distance between the vehicle and the vehicle ahead - Upper limit of braking and accelerating frequency relative to the distance between the vehicle and the vehicle behind These are set based on learning processing. If these are equal to or less than the upper limit values, they are information that serves as an index for estimating that the emotional state and condition of the driver are normal.

[0036] The above is an example, and other detected information, such as upper limits or ranges relating to the external environment of the vehicle, such as traffic congestion or weather, may also be set as the allowable information.

[0037] The tolerance information calculation unit 22 calculates tolerance information that serves as an index for emotion determination as described above, but it is assumed that the tolerance information differs for each individual. Therefore, the tolerance information calculation unit 22 also performs individual determination based on detection information from various sensors of the driver, learns the driving behavior of each individual, and updates the tolerance information for each individual. The storage unit 30 stores the tolerance information as an initial value and the tolerance information for each individual that is updated through learning.

[0038] The determination unit 23 performs an emotion determination process for determining the emotion of the driver based on the acquired detection information, and an assistance determination process for determining whether the driver is in a state requiring driving assistance. That is, the determination unit 23 estimates the emotion of the driver based on the detection information from various sensors. Various known algorithms can be applied as emotion estimation algorithms, but emotion estimation can be performed using not only information that directly indicates the driver's state and behavior, such as biometric information, information on line of sight, facial expression, driving operation information, and seat pressure distribution information, but also information on the external environment of the vehicle, etc., to improve the accuracy of the determination.

[0039] The determination unit 23 also determines whether the estimated emotional state (level of anger, level of laughter, etc.) is at a level that requires driving assistance. In this case, the determination unit 23 refers to the driver's tolerance information and determines whether driving assistance is required based on the presence or absence of items that exceed the upper limit values ​​and the number of items.

[0040] When the judgment unit 23 judges that the driver is in an emotional state requiring driving assistance, the assistance control unit 24 performs control processing to determine the type of driving assistance operation according to the driver's emotions and to execute that driving assistance. For example, the assistance control unit 24 issues instructions to the ECU 3 that controls the traveling mechanism 5 and the electrical equipment unit 6 so that the determined driving assistance operation is performed in the traveling mechanism 5 and the electrical equipment unit 6. The assistance control unit 24 also instructs the ECU 3 that controls the notification device 7 to cause the notification device 7 to perform an operation to notify the driver of the driver's emotion.

[0041] Examples of the types of driving assistance that the assistance control unit 24 determines in accordance with the driver's emotion will be described later, but the following examples are possible examples of the control targets. Engine / motor driving force control Brake / regenerative brake / accelerator braking and driving command control Steering characteristic control Suspension control ADAS (Advanced driver-assistance systems) control

[0042] It is anticipated that these control thresholds will be shifted to higher values ​​or control times to shorter values, thereby increasing the speed of response in the event of a danger occurring.

[0043] <Processing example of the first embodiment> An example of processing by the driving support device 2 according to the first embodiment will be described with reference to Figures 3 and 4. The processing described below is an example of processing executed by the driving support device 2 using the functions of the calculation unit 20 described with reference to Figure 2.

[0044] The driving assistance device 2 performs a process for preparing for the determination in FIG. 3 at the stage when the vehicle 1 starts traveling, for example, immediately after the ignition is turned on or the ready-to-drive state is reached.

[0045] In step S101, the driving assistance device 2 performs personal identification of the driver. For example, the driver is identified using face authentication based on face detection information from the in-vehicle sensor unit 14, iris authentication based on an image of the driver's eyeballs, information from a fingerprint sensor, voiceprint information, or the like. This is personal identification to determine whether or not the learned allowable information has already been stored. For example, the learned allowable information is managed in association with face information or the like for personal identification, and by comparing this with the detected information about the current driver, it can be determined whether or not the current driver is a driver for whom allowable information has already been learned.

[0046] If the driver has already learned the allowable information, the driving assistance device 2 proceeds from step S102 to step S103, reads the allowable information for the driver from the storage unit 30, and sets the allowable information to be used in the process of FIG.

[0047] If the driver has not yet learned the permissible information, the driving assistance device 2 proceeds from step S102 to step S104, reads out general permissible information from the storage unit 30, and sets it as the permissible information to be used in the processing of FIG.

[0048] It is desirable that the identification information obtained when identifying an individual in step S101, such as facial identification information, iris information, fingerprint information, etc., be stored in the memory unit 30 so that it can be linked to the updated acceptable information as a result of subsequent learning processing.

[0049] 4 shows an example of processing performed by the driving support device 2 during driving. The driving support device 2 repeatedly executes the processing in FIG. 4 until it is determined in step S210 that the support processing has ended.

[0050] In step S201, the driving assistance device 2 acquires detection information from various sensors. In step S202, the driving assistance device 2 performs a process for determining the driver's emotions. This process involves the driving assistance device 2 estimating emotions such as joy, surprise, anger, fear, and sadness based on the detection information received from various sensors. As described above, emotions can be estimated primarily using biometric information, gaze information, and driving behavior information. Information about the surrounding environment, such as traffic congestion, tailgating, and weather, can also be used for emotion estimation.

[0051] In step S203, the driving support device 2 determines the driver's condition. This processing example is an example of enabling driving assistance that enhances safety by simultaneously determining not only the driver's emotions but also the driver's physical condition, drowsiness, fatigue, etc. As with emotion determination, condition determination can also be performed using the driver's biological information, gaze information, driving behavior information, etc., so condition determination can be performed in addition to emotion determination. For example, physical fatigue level can be calculated by calculating the center of gravity of the driver's body from seat pressure.

[0052] In step S204, the driving assistance device 2 determines whether the driver's emotions are such that assistance is required. For example, people's emotions, such as joy, surprise, anger, fear, and sadness, vary in degree, and even slight joy or anger does not prevent a person from driving calmly. On the other hand, if emotions such as joy or laughter are too strong, driving may become neglected. The same is true for emotions such as anger and fear; the stronger they become, the more likely they are to impede safe driving.

[0053] Therefore, the driving assistance device 2 compares the detection information obtained from the various sensors for emotion determination with the tolerance information obtained in step S103 or step S104 in Fig. 3 to determine whether the emotional state is at a level requiring driving assistance. For example, each piece of detection information is compared with the upper limit value of the tolerance information exemplified above to determine whether or not there is any detection information that exceeds the upper limit value, the number of pieces of detection information that exceed the upper limit value, or which pieces of detection information exceed the upper limit value, and comprehensively determines whether or not the detection information exceeds the reference value for determining the need for assistance.

[0054] For example, each piece of detection information is judged as to whether it exceeds the set allowable information, or the amount by which it exceeds the allowable information, and is then scored. Then, a predetermined calculation is performed using each score, and it is judged whether the calculation result exceeds a reference value for determining whether or not assistance is required. In this case, each piece of detection information may be weighted. If the calculation result exceeds the reference value, it is determined that driving assistance is required. In this case, the algorithm for making the judgment could also take into account the situation outside the vehicle. For example, if a vehicle cuts in front of the driver, the weight of the score related to anger could be increased. It is also possible to take into account the in-car environment, the relationship with passengers, etc. The driving assistance device 2 may also determine the emotions of passengers, and if the driver and passengers are in a tense situation, the determination criteria may be lowered or the weighting of each score may be changed.

[0055] The driving assistance device 2 also determines the driver's condition in step S203. Therefore, the need for assistance may be determined by taking into account not only the driver's emotional state but also the driver's condition. For example, if poor physical condition or fatigue is determined, the upper limit of the tolerance information that serves as the basis for determination may be lowered to make it easier to provide assistance. Alternatively, tolerance information may also be set for detection information that determines poor physical condition, fatigue, drowsiness, etc., and it may be determined that driving assistance is necessary if the condition exceeds the tolerance information even if the driver's emotions are normal. In other words, the determination of whether driving assistance is necessary may be made from the perspective of emotion, or may be made by a determination calculation that combines emotion and condition, or may be made from both the perspective of emotion and the perspective of condition.

[0056] The driving assistance device 2, for example, determines whether driving assistance is necessary as described above in step S204, and if it determines that assistance is not necessary, proceeds to step S211 to perform learning processing. In this process, the detected information at that time is set as the value for when assistance is not necessary and is used as the data to be learned, and the learning processing is continued to update the allowable information for each individual driver. For example, in the learning processing of step S211, the values ​​of each piece of detected information are stored. Then, at a certain point in time in step S211, learning processing is performed using the detected information values ​​stored up to that point, and the upper limit value of the allowable information at which the driver can perform normal driving is corrected, and the allowable information associated with the individual driver is updated.

[0057] If the driving assistance device 2 determines in step S204 that driving assistance is necessary, it proceeds to step S205 and determines the type of driving assistance depending on the determined emotion. That is, it determines what type of driving assistance control to perform. For example, it determines the parts to be controlled, such as steering assistance control, braking assistance control, or accelerator assistance control, as well as the control amount and control direction (upward / downward, etc.).

[0058] Then, in order to perform the assist control determined in step S206, the driving assist device 2 instructs the ECU 3, which controls the controlled units, to perform the control. For example, when performing brake assist control, the driving assist device 2 instructs the ECU 3, which controls the brake mechanism, to perform the control related to the brake mechanism. As a result, an appropriate type of driving assistance is performed depending on the driver's emotion. Examples of what type of assistance control is performed depending on what emotion will be described later with reference to Figs. 6 and 7.

[0059] In step S207 of Fig. 4, the driving assistance device 2 performs emotion notification control. This control notifies the driver of his / her emotion or emotional state. To this end, the driving assistance device 2 transmits an instruction to the ECU 3 that controls the notification device 7 to cause the notification device 7 to notify the emotion. In response to this instruction, the ECU 3 that controls the notification device 7 controls the notification device 7 to notify the emotion. For example, depending on the judgment of joy, fear, anger, etc., the driver is notified of emotions such as excessive excitement or irritation by text, images such as icons, or voice. This allows the driver to know when their emotions are heightened and encourages them to return to calm driving. In addition to emotions, the driver may also be notified of their physical condition, fatigue, drowsiness, etc.

[0060] The driving assistance device 2 repeats the above-described process of Fig. 4 until assistance is terminated, for example, when driving is terminated.

[0061] <Processing example of the second embodiment> In the second embodiment, the driving assistance device 2 adds processing using the function of the stimulus control unit 25 in FIG. 2 to the first embodiment.

[0062] The stimulus control unit 25 has a function of controlling an operation that gives a slight stimulus or stress to the driver according to the driver's emotions. The stimuli in this case include visual stimuli from displays and lights, auditory stimuli from audio and voice sounds, and stimuli related to the air environment in the vehicle cabin, such as changes in the temperature and airflow of the air conditioner and opening the windows. There are also stimuli related to the seat movement, changes in the headrest angle, slight vibrations of the steering wheel, changes in the position of the lumbar support, etc. Olfactory stimuli such as spraying air fresheners are also possible.

[0063] It is known that the human brain reacts to certain types of stress, such as seeing light or hearing sound. It is also known that opening a window and getting some fresh air, or pressing acupressure points, can eliminate or alleviate drowsiness. It is also known that the body reacts to changes from a steady state. These small amounts of stress are known to contribute to brain arousal, and applying one or a combination of these small amounts of stress can calm emotions and contribute to brain arousal, promoting safe driving.

[0064] Therefore, the driving support device 2 performs the processing shown in Fig. 5. In the second embodiment, the driving support device 2 also performs the processing shown in Fig. 3 and then performs the processing shown in Fig. 5. The processing example in Fig. 5 is obtained by adding steps S208 and S209 to the processing in Fig. 4. Since the other processing is similar, the same step numbers are used to avoid redundant explanation.

[0065] When the driving assistance device 2 determines that the emotional state requires driving assistance, it performs steps S208 and S209 in addition to the processes from step S205 to step S207. In step S208, the driving support device 2 determines the type of stimulus to be given to the driver according to the determined emotion of the driver.

[0066] In step S209, the driving support device 2 instructs the ECU 3 that controls the mechanism that applies the stimulus to control the stimulus in order to apply the determined stimulus. For example, if the air volume of the air conditioner is to be changed, the driving support device 2 instructs the ECU 3 that controls the air conditioner to carry out control of the change in air volume. As a result, an appropriate type of stimulus is applied depending on the driver's emotion. Examples of what type of stimulus is applied depending on what emotion will be described later.

[0067] As shown in Figure 5 above, by providing stimuli according to emotions in conjunction with driving assistance control, the driver is encouraged to return to a safe emotional state and a state of alertness.

[0068] <Specific examples of driving assistance> Below are examples of driving assistance control and types of stimulus provision according to various emotions. Fig. 6 shows an example of driving assistance, and Fig. 7 shows an example of the stimulus to be provided. Each figure also shows examples of control types in the presence of not only emotions but also drowsiness and fatigue.

[0069] If the determined emotion is "joy," there is concern that the driver's driving behavior may be due to oversight or carelessness. In this case, types of driving assistance control include increasing braking response. This is a control that increases the brake pressure, for example, to improve braking response when the brakes are applied. Other types of driving assistance control include decreasing steering response and decreasing driving force response. These are controls that prevent excessive steering or accelerator operation. In the case of "joy," stimuli could be lowering the temperature setting on the air conditioner, increasing the airflow, or lowering the audio volume. In other words, these are stimuli that induce a sense of calm.

[0070] If the determined emotion is "expectation," such as excitement, there is concern that the driver's driving behavior may be due to oversight or carelessness. In this case, the types of driving assistance control include increasing braking responsiveness, decreasing steering responsiveness, and decreasing driving force responsiveness. In the case of "expectation," possible stimuli include lowering the temperature setting of the air conditioner, increasing the airflow, or lowering the audio volume.

[0071] If the emotion determined is "anger," for example, anger or annoyance toward other vehicles, the driver's driving behavior may be feared to be tailgating or rushing. If the emotion is irritation or annoyance, the driver's driving behavior may be feared to be sudden operations such as sudden steering, sudden acceleration, and sudden braking. The types of driving assistance control for these "anger" situations include increasing braking responsiveness, increasing vehicle notification sensitivity to the front and rear, decreasing driving force responsiveness, and increasing suspension rigidity. Possible stimuli include lowering the temperature setting of the air conditioner, increasing the airflow, or lowering the audio volume.

[0072] If the determined emotion is "dislike," such as impatience, confusion, distress, or discomfort, there is concern that the driver's driving behavior may be due to oversight or carelessness. In this case, the types of driving assistance control available include increasing braking responsiveness, decreasing steering responsiveness, and decreasing driving force responsiveness.

[0073] If the determined emotion is "sadness," there is concern that the driver's driving behavior will become dull. In this case, types of driving assistance control include increasing braking response, increasing steering response, increasing driving force response, and increasing suspension rigidity. This is intended to encourage as smooth driving as possible in response to dull driving behavior.

[0074] If the determined emotion is "surprise," there is concern that the driver may be performing unintended driving operations. In this case, the types of driving assistance control that can be implemented include increasing braking response, decreasing steering response, and decreasing driving force response. A possible example of providing a stimulus is to increase the volume of the audio.

[0075] If the determined emotion is "fear," there are concerns that the driver's driving behavior may be oversight, carelessness, slow driving operation, etc. In this case, the types of driving assistance control include increasing braking response, increasing steering response, increasing driving force response, increasing suspension rigidity, etc. A possible example of providing a stimulus is to increase the volume of the audio.

[0076] If the determined emotion is "trust," for example, an excessive sense of ease, there is concern that the driver's driving behavior may be due to oversight or carelessness. In this case, the types of driving assistance control include increasing braking responsiveness, increasing steering responsiveness, increasing driving force responsiveness, increasing suspension rigidity, etc.

[0077] If drowsiness or fatigue is determined by the condition assessment, there is concern that the driver may experience delays in driving operations or unintentional driving operations, etc. In this case, types of driving assistance control may include increasing braking responsiveness, increasing vehicle notification sensitivity to forward and rearward traffic, decreasing steering responsiveness, decreasing driving force responsiveness, etc. Possible stimuli include flashing the meters, MFD, interior, etc., lowering the air conditioner's set temperature or increasing the airflow, slightly lowering the windows, applying vibrations such as steering wheel vibration, pedal vibration, or seat vibration, changing the seat angle, and increasing the audio volume.

[0078] If the emotion is determined to be "indifferent and unconscious," there are concerns about the driver's driving behavior, such as oversight, inattention, delayed driving operations, unintended driving operations, etc. In this case, the types of driving assistance control include increased braking responsiveness, increased sensitivity to vehicle notifications for the front and rear, decreased steering responsiveness, decreased driving force responsiveness, etc. The types of stimuli that can be applied are similar to those used for drowsiness and fatigue.

[0079] It should be noted that FIG. 7 indicates that a "state notification" is performed in any emotion or condition, which means that the emotion or condition is notified by the notification device 7.

[0080] <Effects of the embodiment> As described above, the driving assistance device 2 of the embodiment executes an emotion determination process that determines the driver's emotion based on the acquired detection information, an assistance determination process that determines whether the driver is in a state where driving assistance is required, an assistance type determination process that determines the type of driving assistance according to the determined emotion when driving assistance is required, and an assistance execution control process that executes the determined type of driving assistance. As a result, when it is determined that the driver is not calmly concentrating on driving and is in an emotional state requiring driving assistance, driving assistance appropriate to the driver's emotions is executed, thereby improving safety when the driver's emotions are no longer appropriate for driving. In particular, the processing of the embodiment does not only determine the driver's state of wakefulness. Even when the driver is aware that their consciousness is normal, their driving behavior is affected by the degree of concentration on driving based on their emotions. For example, the driver may be late in braking, or may operate the accelerator or steer more roughly than usual. In this embodiment, assistance according to emotions is possible in response to these situations.

[0081] The driving assistance device 2 according to the embodiment executes a notification control process for issuing a notification to the driver according to the determined emotion when driving assistance is required. By notifying the driver of the determined emotional state, the driver can be made aware that their emotions are not normal, which can encourage the driver to remain calm and concentrate on driving.

[0082] As described in the processing of the second embodiment, when driving assistance is required, the driving assistance device 2 executes a stimulus type determination process that determines the type of stimulus depending on the determined emotion, and a stimulus execution control process that executes a stimulus imparting operation that imparts the determined type of stimulus to the driver. Depending on the driver's emotional state, the system provides appropriate stimuli, such as slight stress depending on the emotion, to the driver. This can stimulate the driver's brain and encourage them to return to an emotional state that allows for safe driving.

[0083] In the assistance determination process, the driving assistance device 2 of the embodiment compares the acquired detection information with the set allowable information, and executes a process of determining whether or not driving assistance is required using the comparison result. Threshold values ​​for various types of detected information in the driver's normal state are set as permissible information, and these are compared with the various detected information while driving. This makes it possible to determine with relatively high accuracy whether the driver is in a normal emotional state for safe driving or in a state where driving assistance is required.

[0084] In the embodiment, the driving assistance device 2 performs processing to update the tolerance information for each individual driver in accordance with the learning processing. Since tolerance varies from person to person, the system learns information detected in a normal emotional state for each individual driver and updates the tolerance information, thereby enabling emotion determination that reflects individual differences. [Explanation of symbols]

[0085] 1 vehicle 2. Driving assistance devices 3 ECU 5. Traveling mechanism 6 Notification Devices 7 Electrical Equipment Department 10. Control System 14 In-vehicle sensor section 15. Outside vehicle sensor 16 Vehicle sensor unit 17 Wearable Sensors 20 Arithmetic section 21 Information Acquisition Department 22 Allowable Information Calculation Unit 23 Judgment section 24 Support control section 25 Stimulation control section 30 Storage section

Claims

1. one or more processors; a storage medium storing a program to be executed by the one or more processors; The program includes one or more instructions, The instructions may cause the one or more processors to: An emotion determination process for determining the emotion of the driver based on the acquired detection information; an assistance determination process for determining whether or not the driver is in a state requiring driving assistance; an assistance type determination process for determining the type of driving assistance according to the determined emotion when driving assistance is required; an assistance execution control process for executing the driving assistance of the determined type; Driving assistance device.

2. The instruction: When driving assistance is required, the one or more processors are caused to execute a notification control process for issuing a notification to the driver according to the determined emotion. The driving assistance device according to claim 1 .

3. The instruction: a stimulus type determination process for determining a type of stimulus according to the determined emotion when driving assistance is required; a stimulus execution control process for executing a stimulus providing operation for providing the determined type of stimulus to the driver; causing the one or more processors to execute The driving assistance device according to claim 1 .

4. The instruction: In the assistance determination process, the one or more processors are caused to execute a process of comparing the acquired detection information with set allowable information and determining whether or not driving assistance is required using a comparison result. The driving assistance device according to any one of claims 1 to 3.

5. The instruction: The one or more processors are caused to execute a process of updating the permission information for each individual driver in accordance with a learning process. The driving assistance device according to claim 4.

Images