Driving assistance control device and driving assistance control method

Abstract

To appropriately support driving according to emotion of a driver.SOLUTION: A driving support control device comprises: a biological information acquiring unit that acquires biological information of a driver of a vehicle in current and past predetermined periods; an emotion determination unit that determines an emotional level indicating a degree of emotion of the driver in the current and past predetermined periods based on the biological information acquired by the biological information acquisition unit; a driving condition prediction unit that predicts a driving condition of the vehicle based on the emotional levels of the driver in the current and past predetermined periods when the current emotional level of the driver is determined to be equal to or higher than a first threshold; a driving support determination unit that determines whether or not a driving support of the vehicle is required based on the result of prediction of the driving condition of the vehicle by the driving condition prediction unit, and determines the content of the driving support of the vehicle when the driving support of the vehicle is required; and a driving support execution unit that executes the driving support determined by the driving support determination unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a driving assistance control device and a driving assistance control method. [Background technology]

[0002] There are known techniques for estimating a user's emotion. For example, Patent Document 1 discloses a technique for reflecting an estimation result of a user's past emotional state when estimating the emotional state of a user based on information transmitted by the user. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-29386 A Summary of the Invention [Problem to be solved by the invention]

[0004] When a driver's emotions change, such as becoming irritated, the driver may engage in dangerous driving, such as tailgating or intentionally cutting in. Dangerous driving may lead to accidents involving the driver's own vehicle or other vehicles. Therefore, it is desirable to appropriately determine the driver's emotional changes and provide driving support according to the determined emotions.

[0005] An object of the present invention is to provide a driving assistance control device and a driving assistance control method that can appropriately assist driving in accordance with the driver's emotions. [Means for solving the problem]

[0006] The driving assistance control device of the present invention includes a biometric information acquisition unit that acquires biometric information of a driver of a vehicle for a current and past predetermined period, an emotion determination unit that determines an emotion level indicating the degree of emotion of the driver for the current and past predetermined periods based on the biometric information acquired by the biometric information acquisition unit, a driving situation prediction unit that predicts a driving situation of the vehicle based on the driver's emotion levels for the current and past predetermined periods when it is determined that the driver's current emotion level is equal to or higher than a first threshold, a driving assistance determination unit that determines whether driving assistance of the vehicle is necessary based on a prediction result of the driving situation of the vehicle by the driving situation prediction unit and determines content of the driving assistance of the vehicle if driving assistance of the vehicle is necessary, and a driving assistance execution unit that executes the driving assistance determined by the driving assistance determination unit.

[0007] The driving assistance control method of the present invention includes the steps of acquiring biometric information of a vehicle driver for a predetermined period of time, currently and in the past, based on the acquired biometric information, determining an emotion level indicating the degree of emotion of the driver for the current and past predetermined periods, and if the current emotion level of the driver is determined to be equal to or higher than a first threshold, predicting a driving situation of the vehicle based on the driver's current and past emotion levels for the predetermined periods, based on the predicted result of the driving situation of the vehicle, determining whether or not driving assistance of the vehicle is necessary, and if driving assistance of the vehicle is necessary, determining the content of the driving assistance of the vehicle, and executing the determined driving assistance. Effect of the Invention

[0008] According to the present invention, driving can be appropriately assisted in accordance with the driver's emotions. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a driving assistance system according to the first embodiment. [Diagram 2] FIG. 2 is a flowchart showing an example of the flow of the driving assistance control process according to the first embodiment. [Diagram 3] FIG. 3 is a diagram for explaining a method for determining an emotion level according to the first embodiment. [Figure 4] FIG. 4 is a diagram for explaining a method for determining a past emotion level according to the first embodiment. [Diagram 5] FIG. 5 is a diagram for explaining patterns of the contents of driving support according to the first embodiment. [Figure 6] FIG. 6 is a diagram for explaining a first pattern of driving assistance content according to the first embodiment. [Figure 7] FIG. 7 is a diagram for explaining a second pattern of driving assistance content according to the first embodiment. [Figure 8] FIG. 8 is a diagram for explaining a third pattern of driving assistance content according to the first embodiment. [Figure 9] FIG. 9 is a diagram for explaining a fourth pattern of driving assistance content according to the first embodiment. [Figure 10] FIG. 10 is a diagram for explaining a fifth pattern of driving assistance content according to the first embodiment. [Figure 11] FIG. 11 is a diagram for explaining the sixth pattern of driving assistance content according to the first embodiment. [Figure 12] FIG. 12 is a diagram for explaining the seventh pattern of driving support content according to the first embodiment. [Figure 13] FIG. 13 is a diagram for explaining an eighth pattern of driving support content according to the first embodiment. [Figure 14] FIG. 14 is a block diagram showing an example of the configuration of a driving assistance system according to the second embodiment. [Figure 15] FIG. 15 is a flowchart showing an example of the flow of the driving assistance control process according to the second embodiment. [Figure 16] FIG. 16 is a block diagram showing an example of the configuration of a driving assistance system according to the third embodiment. [Figure 17] FIG. 17 is a flowchart showing an example of the flow of a driving assistance control process according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to this embodiment, and in the following embodiments, the same components are designated by the same reference numerals, and duplicated explanations will be omitted.

[0011] [First embodiment] (Driving assistance system) An example of the configuration of the driving assistance system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the driving assistance system according to the first embodiment.

[0012] 1, the driving assistance system 1 includes an imaging device 10, a biosensor 12, and a driving assistance device 14. The imaging device 10 and the driving assistance device 14, and the biosensor 12 and the driving assistance device 14 are connected to each other so that various data can be communicated.

[0013] The imaging device 10 includes a first camera 10a and a second camera 10b.

[0014] The first camera 10a is a camera that captures the surroundings of the vehicle. The first camera 10a is, for example, composed of a group of cameras that capture the front, rear, left side, and right side of the vehicle. The first camera 10a may be, for example, composed of a single camera that can capture the entire sky. The first camera 10a may be, for example, composed of a single camera that captures only one direction, that is, the front of the vehicle. The first camera 10a may have, for example, a microphone that collects sound around the vehicle.

[0015] The second camera 10b is a camera that is arranged to face toward the interior of the vehicle and captures the interior of the vehicle. The second camera 10b captures, for example, at least the face (facial expression, eye movement) of the driver of the vehicle. The second camera 10b is, for example, a wide-angle camera that can capture an image in a range of about 180°. The second camera 10b may be, for example, a group of multiple cameras. The second camera 10b may have, for example, a microphone that captures sound in the vehicle.

[0016] The biosensor 12 is a sensor that detects bioinformation of the driver of the vehicle. The biosensor 12 is, for example, a wearable device that is attached to the body of the driver of the vehicle. The biosensor 12 is, for example, attached to the arm, finger, torso, etc. of the driver of the vehicle, but may be attached to other body parts. The biosensor 12 detects the bioinformation of the driver of the vehicle at predetermined intervals, for example, and stores the detection results. The biosensor 12 is, for example, composed of a brain wave sensor, a pulse sensor, a blood pressure sensor, a body temperature sensor, and a sweat sensor, but may be composed of a sensor that detects other bioinformation. Specifically, the biosensor 12 is, for example, composed of a sensor that detects bioinformation that can predict the emotions of the driver of the vehicle.

[0017] The driving assistance device 14 is mounted on a vehicle. The driving assistance device 14 may be a device mounted on the vehicle, or may be a portable device that can be used in the vehicle. The driving assistance device 14 is a device that performs driving assistance according to the current and past emotional levels of a driver who drives the vehicle.

[0018] The driving assistance device 14 includes an input unit 20, a display unit 22, an audio output unit 24, a storage unit 26, a communication unit 28, and a control unit (driving assistance control device) 30.

[0019] The input unit 20 accepts various operations for the driving support device 14. The input unit 20 is configured with various input devices such as, for example, a button, a switch, and a touch panel.

[0020] The display unit 22 displays various images. For example, the display unit 22 can function as a notification unit that notifies the driver of the contents of the driving assistance by displaying the contents of the driving assistance as images. The display unit 22 is, for example, a display including a liquid crystal display (LCD), an organic EL (Electro-Luminescence), a head-up display, or the like.

[0021] The audio output unit 24 outputs various sounds. For example, the audio output unit 24 can function as a notification unit that notifies the driver of the contents of the driving assistance by outputting the contents of the driving assistance by sound. The audio output unit 24 is, for example, configured with a speaker mounted on the vehicle.

[0022] The storage unit 26 is a memory that stores various types of information. The storage unit 26 stores information such as the contents of calculations performed by the control unit 30 and programs. The storage unit 26 includes at least one of a RAM (Random Access Memory), a main storage device such as a ROM (Read Only Memory), and an external storage device such as an HDD (Hard Disk Drive).

[0023] The communication unit 28 transmits and receives data to and from an external device. For example, the communication unit 28 transmits and receives data to and from the imaging device 10 and the biosensor 12. The communication unit 28 is configured to communicate using a method such as Wi-Fi (registered trademark) and Bluetooth (registered trademark).

[0024] The control unit 30 controls the operation of each unit of the driving support device 14. The control unit 30 is realized, for example, by a central processing unit (CPU) or a micro processing unit (MPU) executing a program stored in the storage unit 26 or the like using a RAM or the like as a working area. The control unit 30 may be realized, for example, by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control unit 30 may be realized by a combination of hardware and software.

[0025] The control unit 30 includes a video data acquisition unit 40, a biometric information acquisition unit 42, an emotion determination unit 44, a driving situation prediction unit 46, a driving assistance decision unit 48, and a driving assistance execution unit 50.

[0026] The video data acquisition unit 40 acquires video data from the imaging device 10. For example, the video data acquisition unit 40 acquires video data of the periphery of the vehicle from the first camera 10a. For example, the video data acquisition unit 40 acquires video data of the interior of the vehicle from the second camera 10b.

[0027] The biometric information acquisition unit 42 acquires biometric information of the vehicle driver from the biometric sensor 12. The biometric information acquisition unit 42 acquires, for example, the current biometric information of the vehicle driver and the biometric information of the user for a predetermined period of time in the past. The predetermined period of time in the past also includes a period when the user was not in the vehicle.

[0028] The emotion determination unit 44 determines the emotion level of the driver of the vehicle. For example, the emotion determination unit 44 determines the emotion level of the driver of the vehicle at present and for a predetermined period in the past. For example, the emotion determination unit 44 determines the emotion level of the driver of the vehicle based on the biometric information of the driver of the vehicle acquired by the biometric information acquisition unit 42. For example, the emotion determination unit 44 determines whether the emotion of the driver of the vehicle is joy, anger, sadness, or happiness based on the biometric information of the driver of the vehicle. For example, the emotion determination unit 44 determines an emotion level indicating the degree to which the driver feels "happy" or "happy" as the emotion of joy. For example, the emotion determination unit 44 determines an emotion level indicating the degree to which the driver feels "anger", "worry", or "unpleasant" as the emotion of anger. For example, the emotion determination unit 44 determines an emotion level indicating the degree to which the driver feels "sadness" or "depression" as the emotion of sadness. The emotion determination unit 44 determines, for example, an emotion level indicating the degree to which the driver feels "joy," "satisfaction," or "relief" as the emotion of comfort. In this embodiment, it is preferable that the emotion determination unit 44 determines, for example, emotions such as anger and sadness, which are assumed to be highly likely to induce dangerous driving. For example, the emotion determination unit 44 may determine emotions such as joy and comfort, since it is assumed that excitement, joy, and enjoyment are also likely to distract attention and induce dangerous driving.

[0029] The driving condition prediction unit 46 predicts the driving condition of the vehicle after a predetermined period of time has elapsed. The driving condition prediction unit 46 predicts the driving condition of the vehicle after a predetermined period of time has elapsed, for example, based on the determination result of the emotion level of the driver of the vehicle by the emotion determination unit 44. In this embodiment, the driving condition includes information indicating the possibility of dangerous driving or a dangerous driving state.

[0030] The driving support determination unit 48 determines whether or not to perform driving support. When the driving support determination unit 48 determines to perform driving support, the driving support determination unit 48 determines the content of driving support for the driver of the vehicle. The driving support determination unit 48 determines the content of driving support based on, for example, the result of prediction of the driving situation of the vehicle by the driving situation prediction unit 46.

[0031] The driving assistance execution unit 50 executes driving assistance for the driver of the vehicle. The driving assistance execution unit 50 executes driving assistance based on the result of the determination of the content of the driving assistance by the driving assistance determination unit 48. The driving assistance execution unit 50 includes a vehicle control unit 50a and a notification control unit 50b.

[0032] The vehicle control unit 50a controls the vehicle based on the result of the determination of the content of the driving assistance by the driving assistance determination unit 48. For example, the vehicle control unit 50a controls the engine to limit acceleration, or controls the brake mechanism to assist in deceleration.

[0033] The notification control unit 50b issues a notification based on the result of the determination of the content of the driving support by the driving support determination unit 48. The notification control unit 50b, for example, controls the display unit 22 to display the notification content and controls the audio output unit 24 to output the notification content by audio.

[0034] (Driving assistance control processing) The flow of processing for driving assistance control according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a flowchart showing an example of the flow of processing for driving assistance control according to the first embodiment.

[0035] The biometric information acquiring unit 42 acquires biometric information of the vehicle driver (step S10). Specifically, the biometric information acquiring unit 42 acquires the current and past biometric information of the vehicle driver from the biometric sensor 12 worn by the vehicle driver. Then, the process proceeds to step S12.

[0036] The emotion determination unit 44 determines the current emotion level of the driver of the vehicle (step S12). Specifically, the emotion determination unit 44 determines the emotion level of the driver based on, for example, the current biometric information of the driver acquired by the biometric information acquisition unit 42. The emotion determination unit 44 determines the current emotion level of the driver based on, for example, the difference between reference biometric information indicating a reference value of the biometric information of the driver stored in the storage unit 26 and the biometric information acquired by the biometric information acquisition unit 42. The reference value of the biometric information may be biometric information when the driver is emotionless, without any emotion of joy, anger, sadness, or pleasure. The biometric information acquired by the biometric information acquisition unit 42 in this case may be a weak detection result. FIG. 3 is a diagram for explaining a method of determining an emotion level according to the first embodiment. For example, when the current emotion level of the driver is slightly changed from the reference value, the emotion determination unit 44 determines that the emotion level of the driver is "low". The emotion level "low" is also called a first threshold value of the emotion level. For example, when the driver's emotional level has changed moderately from normal, the emotion determination unit 44 determines that the driver's emotional level is "medium". The emotion level "medium" is also called a second emotion level threshold. The emotion level "medium" is an emotion level that may impair safe driving. For example, when the driver's emotional level has changed significantly from normal, the emotion determination unit 44 determines that the driver's emotional level is "high". In the first embodiment, the emotion determination unit 44 is described as determining whether the driver's emotional level is "normal", "low", "medium", or "high", but is not limited thereto. For example, the emotion determination unit 44 may quantitatively calculate the driver's emotional level using a numerical value. For example, the emotion determination unit 44 may calculate the driver's emotional level based on the driver's brain waves, pulse value, blood pressure, body temperature, and sweat amount acquired by the bioinformation acquisition unit 42. For example, the emotion determination unit 44 may determine the driver's emotional level based on video data capturing the driver's facial expression. Then proceed to step S14.

[0037] The emotion determination unit 44 may determine the emotion level of the driver based on, for example, video data of the driver while driving captured by the second camera 10b. The emotion determination unit 44 may determine the current emotion level of the driver by extracting the driver's facial expression from the video data of the driver. The emotion determination unit 44 may determine the emotion level of the driver by analyzing the driver's gestures from the video data of the driver. The emotion determination unit 44 may determine the emotion level of the driver based on, for example, dictionary data that determines the emotion from the driver's gestures. The emotion determination unit 44 may determine the emotion level of the user based on, for example, biometric information of the user using a machine learning model that predicts the emotion level of the user. The machine learning model may be a model that inputs the biometric information of the user and outputs a prediction result of the type of emotion and the emotion level of the user.

[0038] Returning to FIG. 2, the emotion determination unit 44 determines whether the current emotion level of the driver of the vehicle is equal to or higher than the first threshold (step S14). If it is determined that the current emotion level of the driver is equal to or higher than the first threshold (step S14; Yes), the process proceeds to step S16. If it is not determined that the current emotion level of the driver is equal to or higher than the first threshold (step S14; No), the process proceeds to step S30.

[0039] If the answer to step S14 is Yes, the emotion determination unit 44 determines the emotion level of the driver of the vehicle in a predetermined period in the past (step S16). Specifically, the emotion determination unit 44 determines the emotion level of the driver based on, for example, the driver's biometric information acquired by the biometric information acquisition unit 42 for a predetermined period in the past. FIG. 4 is a diagram for explaining a method for determining a past emotion level according to the first embodiment. FIG. 4 shows the detection result of the driver's biometric information by the biometric sensor 12. FIG. 4 shows a change in the detection result of the biometric information of the driver of the vehicle in a predetermined period in the past. In FIG. 4, the time t8 is the present, and the period from time t1 to time t8 is the predetermined period in the past. The predetermined period in the past is, for example, from the present to about 12 hours ago, but is not limited thereto. The predetermined period in the past may be arbitrarily set according to, for example, the personality of the driver of the vehicle. For example, if the driver is an anxious person, the predetermined period in the past may be up to about 12 hours ago. For example, if the driver is an optimistic person, the predetermined period in the past may be up to about 30 minutes ago.

[0040] In the example shown in FIG. 4, the emotion determination unit 44 determines that the emotion level of the vehicle driver is "low" for example, during a period from time t1 to time t2. The emotion determination unit 44 determines that the emotion level of the vehicle driver is "normal" for example, during a period from time t2 to time t3. The emotion determination unit 44 determines that the emotion level of the vehicle driver is "low" for example, during a period from time t3 to time t4. The emotion determination unit 44 determines that the emotion level of the vehicle driver is "medium" for example, during a period from time t4 to time t5. The emotion determination unit 44 determines that the emotion level of the vehicle driver is "high" for example, during a period from time t5 to time t6. The emotion determination unit 44 determines that the emotion level of the vehicle driver is "medium" for example, during a period from time t6 to time t7. The emotion determination unit 44 determines that the emotion level of the vehicle driver is "low" during the period from time t7 to time t8, for example. The emotion determination unit 44 calculates the number of times the emotion level is determined to be "low", the number of times the emotion level is determined to be "medium", and the number of times the emotion level is determined to be "high" during the period from time t1 to time t8. As will be described later, the driving support determination unit 48 determines the content of the driving support based on the number of times each emotion level is determined. The emotion determination unit 44 may, for example, equally divide the period from time t1 to time t8 at a predetermined time interval, and calculate the number of times the emotion level is determined to be "low", the number of times the emotion level is determined to be "medium", and the number of times the emotion level is determined to be "high" for each equally divided time. There is no particular limit to the predetermined time interval. For example, the emotion level may be determined in detail by equally dividing the period from time t1 to time t8 into 100 parts.

[0041] The emotion determination unit 44 may, for example, calculate the area of ​​the portion exceeding the threshold lines indicating "low", "medium", and "high" for each of the emotion levels "low", "medium", and "high" to determine the emotion level of the driver. Specifically, the emotion determination unit 44 may, for example, calculate the area of ​​the diagonal line portion for each of the emotion levels "low", "medium", and "high" between time t1 and time t2, between time t3 and time t4, between time t4 and time t5, between time t5 and time t6, between time t6 and time t7, and between time t7 and time t8 to determine the emotion level of the driver. The method by which the emotion determination unit 44 determines the emotion level of the driver in the past predetermined period is the same as the method in step S12, and therefore the description will be omitted. Then, the process proceeds to step S18.

[0042] Returning to FIG. 2, the emotion determination unit 44 determines whether or not the driver of the vehicle had an emotion level equal to or higher than the second threshold during a predetermined period in the past (step S18). Specifically, the driving assistance decision unit 48 determines, for example, whether or not the driver's emotion level was "medium" or higher during a predetermined period in the past. If it is determined that the driver had an emotion level equal to or higher than the second threshold during the predetermined period in the past (step S18; Yes), the process proceeds to step S20. If it is not determined that the driver had an emotion level equal to or higher than the second threshold during the predetermined period in the past (step S18; No), the process proceeds to step S22.

[0043] If the answer to step S18 is Yes, the driving situation prediction unit 46 predicts the driving situation of the vehicle based on the emotional level of the driver of the vehicle at present and for a predetermined period in the past (step S20). Specifically, the driving situation prediction unit 46 predicts a driving risk indicating the possibility of compromising safety, for example, based on the emotional level of the driver of the vehicle at present and for a predetermined period in the past. More specifically, the driving situation prediction unit 46 predicts the driving risk in three stages, for example, "low", "medium", and "high". A driving risk of "low" indicates that the driver is unlikely to compromise safety. A driving risk of "medium" indicates that the driver is likely to compromise safety at a medium level. A driving risk of "high" indicates that the driver is likely to compromise safety. Then, the process proceeds to step S26.

[0044] If it is determined as No in step S18, the driving assistance decision unit 48 determines whether or not driving assistance is necessary (step S22). Specifically, the driving assistance decision unit 48 determines that driving assistance is necessary, for example, when the driver's current emotion level is "medium" or higher. That is, the driving assistance decision unit 48 determines that driving assistance is not necessary, when an emotion level of "medium" or higher has not been detected within a predetermined past period and the current emotion level is "low" or lower. If it is determined that driving assistance is necessary (step S22; Yes), the process proceeds to step S24. If it is not determined that driving assistance is necessary (step S22; No), the process proceeds to step S30.

[0045] If the result of the determination in step S22 is Yes, the driving situation prediction unit 46 predicts the driving situation based on the current emotional level of the driver of the vehicle (step S24). Specifically, the driving situation prediction unit 46 predicts the driving risk based on, for example, the current emotional level of the driver of the vehicle. Then, the process proceeds to step S26.

[0046] After step S20 or step S24, the driving support determination unit 48 determines the content of the driving support (step S26). Specifically, the driving support determination unit 48 determines the content of the driving support based on the prediction result by the driving situation prediction unit 46. The driving support determination unit 48 determines, for example, the notification content to be notified to the driver and the vehicle control content as the content of the driving support. The driving support determination unit 48 determines, for example, the notification content and the vehicle control content in two stages, "medium" and "high". When the notification content is "medium", an example is, but is not limited to, a warning to the driver. When the notification content is "high", an example is, but is not limited to, a rest warning to the driver. Examples of the vehicle control content include, but are not limited to, engine control, brake control, and warning processing control for the driver when an object is detected outside the vehicle. When the vehicle control content is "medium" and "high", the degree of the strength of the control changes. FIG. 5 is a diagram for explaining the pattern of the content of the driving support according to the first embodiment. As shown in the pattern information 60, in the first embodiment, there are eight possible patterns of the contents of the driving assistance, from the first pattern to the eighth pattern. The eight patterns of the contents of the driving assistance will be described below, but the contents of the driving assistance are not limited to these. In Fig. 6 to Fig. 13, the horizontal axis represents time and the vertical axis represents emotion level, and the number of times that the emotion level states of "low", "medium", and "high" were detected and the time are shown in a bar graph.

[0047] FIG. 6 is a diagram for explaining a first pattern of driving assistance contents according to the first embodiment. The first pattern shows the content of driving assistance when the determination result of the driver's current emotion level is "low" and the emotion level has been determined to be "high" once in the past within a predetermined period T. In this case, the driving situation prediction unit 46 predicts the driving risk to be "medium". The driving assistance determination unit 48 determines the content of driving assistance based on the determined current emotion level, the number of determined past emotion levels, and the predicted driving risk. In the first pattern, the driving assistance determination unit 48 determines the level of the notification content to be "medium" and the level of the vehicle control content to be "medium" among the driving assistance.

[0048] FIG. 7 is a diagram for explaining a second pattern of the driving assistance content according to the first embodiment. The second pattern shows the content of the driving assistance when the determination result of the driver's current emotion level is "low" and the emotion level has been determined to be "medium" once in the past within a predetermined period T. In this case, the driving situation prediction unit 46 predicts the driving risk as "low". In the second pattern, the driving assistance decision unit 48 determines the degree of the notification content of the driving assistance as "none" and the degree of the vehicle control content as "none". That is, in the case of the second pattern, the driving assistance decision unit 48 determines that the vehicle control will not be performed because the driver is unlikely to compromise safety.

[0049] FIG. 8 is a diagram for explaining a third pattern of the driving support content according to the first embodiment. The third pattern shows the content of the driving support when the determination result of the driver's current emotion level is "low" and the emotion level has been determined to be "medium" twice in the past within the past predetermined period T. In this case, the driving situation prediction unit 46 predicts the driving risk as "medium". In the fourth pattern, the driving support determination unit 48 determines the degree of the notification content of the driving support to be "medium" and the degree of the vehicle control content to be "medium". That is, in the case of the third pattern, when the emotion level has been in a "medium" state multiple times within the predetermined period T, the driving support determination unit 48 determines that driving support is necessary according to the number of times the emotion level has been determined in the past. Note that the number of times the emotion level has been in a "medium" state within the predetermined period T, which is determined to be necessary for driving support, may be set according to the driver.

[0050] FIG. 9 is a diagram for explaining a fourth pattern of the driving support content according to the first embodiment. The fourth pattern shows the content of the driving support when the determination result of the driver's current emotion level is "low" and it is determined that the emotion level has been in a "medium" state for a predetermined time or more in the past within the past predetermined period T. In this case, the driving situation prediction unit 46 predicts the driving risk as "medium". In the fourth pattern, the driving support determination unit 48 determines the level of the notification content of the driving support to be "medium" and the level of the vehicle control content to be "medium". That is, in the fourth pattern, the driving support determination unit 48 determines that driving support is necessary when the emotion level has been in a "medium" state for a predetermined time or more within the predetermined period T. Note that the duration of the emotion level in the "medium" state within the predetermined period T, which is determined to be necessary for driving support, may be set according to the driver.

[0051] FIG. 10 is a diagram for explaining a fifth pattern of the driving support content according to the first embodiment. The fifth pattern shows the content of the driving support when the determination result of the driver's current emotion level is "low" and it is determined that the emotion level has been in a "high" state for a predetermined time or more in the past within the past predetermined period T. In this case, the driving situation prediction unit 46 predicts the driving risk as "high". In the fifth pattern, the driving support determination unit 48 determines the level of the notification content of the driving support as "high" and the level of the vehicle control content as "high". That is, in the fifth pattern, the driving support determination unit 48 determines that the driving support is particularly necessary when the emotion level has been in a "high" state for a predetermined time or more within the predetermined period T. In the fifth pattern, the strength of the driving support is stronger. Note that the duration of the emotion level in the "high" state within the predetermined period T, which is determined to be particularly necessary for the driving support, may be set according to the driver. Furthermore, the duration of the "high" emotion level state may be the same as or different from the duration of the "medium" emotion level state in the fourth pattern. For example, safety may be improved by making the duration of the "high" emotion level state shorter than the duration of the "medium" emotion level state in the fourth pattern.

[0052] FIG. 11 is a diagram for explaining the sixth pattern of the driving support content according to the first embodiment. The sixth pattern shows the content of the driving support when the determination result of the driver's current emotion level is "low" and the emotion level has been determined to be "high" twice in the past within the past predetermined period T. In this case, the driving situation prediction unit 46 predicts the driving risk as "high". In the sixth pattern, the driving support determination unit 48 determines the level of the notification content of the driving support to be "high" and the level of the vehicle control content to be "high". That is, in the sixth pattern, the driving support determination unit 48 determines that the driving support is particularly necessary when the emotion level has been in a "high" state multiple times within the predetermined period T.

[0053] FIG. 12 is a diagram for explaining the seventh pattern of the driving assistance content according to the first embodiment. The seventh pattern shows the driving assistance content when the determination result of the driver's current emotion level is "medium" and the emotion level has been determined to be "high" once in the past within a predetermined period T. In this case, the driving situation prediction unit 46 determines the driving risk to be "high". In the seventh pattern, the driving assistance determination unit 48 determines the level of the notification content to be "high" and the level of the vehicle control content to be "high" among the driving assistance.

[0054] FIG. 13 is a diagram for explaining an eighth pattern of driving assistance content according to the first embodiment. The eighth pattern shows the content of driving assistance when the judgment result of the driver's current emotion level is "medium" and the emotion level has not been judged to be in any of the states of "high", "medium" and "low" in the past within the past predetermined period T. In this case, the driving situation prediction unit 46 predicts the driving risk to be "medium". In the eighth pattern, the driving assistance decision unit 48 determines the level of the notification content of the driving assistance to be "medium". Medium " and the degree of vehicle control content " Medium That is, when the current emotion level of the driver is determined to be "medium," the driving assistance decision unit 48 determines that driving assistance is necessary regardless of the emotion level over a predetermined period of time in the past.

[0055] Return to FIG. 2. After step S26, the driving assistance execution unit 50 executes driving assistance (step S28). Specifically, the driving assistance execution unit 50 executes driving assistance in accordance with the content of the driving assistance determined by the driving assistance determination unit 48 in step S26. More specifically, the vehicle control unit 50a controls the vehicle in accordance with the content of the driving assistance determined by the driving assistance determination unit 48 in step S26. The notification control unit 50b controls at least one of the display unit 22 and the audio output unit 24 in accordance with the content of the driving assistance determined by the driving assistance determination unit 48 in step S26 to output the notification content. Then, the process proceeds to step S30.

[0056] After step S28, if it is determined as No in step S14 or if it is determined as No in step S22, the control unit 30 determines whether or not to end the process (step S30). Specifically, the control unit 30 determines to end the process when it receives an operation to end the process or an operation to turn off the power. In addition, the control unit 30 also determines to end the process when the driver's current emotions and the emotions in the past for a predetermined period become normal after the driving assistance is executed. If it is determined to end the process (step S30; Yes), the process of FIG. 2 ends. If it is not determined to end the process (step S30; No), the process proceeds to step S10.

[0057] As described above, the first embodiment determines the content of the driving assistance based on the current emotion level of the driver of the vehicle and the emotion level of the driver in a predetermined period of time in the past. This allows the first embodiment to appropriately assist driving according to the emotion of the driver, thereby improving the safety of the vehicle.

[0058] [Modification of the first embodiment] In the above-described first embodiment, the driving assistance system 1 has been described as providing driving assistance to the driver when the driver drives the vehicle. The driving assistance system 1 may provide parking assistance when the driver parks the vehicle.

[0059] The driving assistance device 14 may perform parking assistance if the driver's emotion level when parking the vehicle is equal to or higher than the first threshold and the emotion level has been determined to be equal to or higher than the second threshold during a predetermined period in the past.

[0060] Specifically, the emotion determination unit 44 determines the driver's current emotion level and the emotion level of the driver over a predetermined period of time when the driver parks the vehicle. The driving assistance determination unit 48 determines the content of parking assistance when the emotion determination unit 44 determines that the driver's current emotion level is equal to or higher than a first threshold and that the emotion level over a predetermined period of time in the past is equal to or higher than a second threshold. The driving assistance execution unit 50 executes the parking assistance determined by the driving assistance determination unit 48. As a result, the modified example of the first embodiment can appropriately assist parking in accordance with the driver's emotion, thereby improving the safety of the vehicle when parking.

[0061] [Second embodiment] An example of the configuration of the driving assistance system according to the second embodiment will be described with reference to Fig. 14. Fig. 14 is a block diagram showing an example of the configuration of the driving assistance system according to the second embodiment.

[0062] As shown in FIG. 14, a driving assistance system 1A differs from the driving assistance system 1 shown in FIG.

[0063] The cause determination unit 52 determines the cause of the change in the emotion level of the driver of the vehicle to the first threshold or more. For example, the cause determination unit 52 determines the cause of the change in the emotion level of the driver based on the video data acquired by the video data acquisition unit 40 from the first camera 10a or the second camera 10b. For example, when the first camera 10a or the second camera 10b detects a sound, the cause determination unit 52 may identify the cause of the change in the emotion level to the first threshold or more based on the detected sound. When the cause determination unit 52 determines the emotion level based on the sound, it is preferable that the driver is in the vehicle from a predetermined time in the past to the present. For example, the cause determination unit 52 associates the emotion level of the driver that is equal to or higher than the first threshold determined by the emotion determination unit 44 with the cause of the change in the emotion level, and stores the result in the storage unit 26 as cause information indicating the cause of the event in which the driver's emotion level became equal to or higher than the first threshold.

[0064] (Driving assistance control processing) The flow of processing for driving assistance control according to the second embodiment will be described with reference to Fig. 15. Fig. 15 is a flowchart showing an example of the flow of processing for driving assistance control according to the second embodiment.

[0065] The video data acquisition unit 40 acquires video data from the imaging device 10 (step S40). The video data acquisition unit 40 acquires, for example, video data of the interior of the vehicle from the first camera 10a. The video data acquisition unit 40 acquires, for example, video data of the periphery of the vehicle from the second camera 10b. Then, the process proceeds to step S42.

[0066] The processes from step S42 to step S46 are the same as the processes from step S10 to step S14 shown in FIG. 2, respectively, and therefore will not be described.

[0067] After the determination in step S46 is Yes, the cause determination unit 52 determines the cause of the driver's emotion level becoming equal to or higher than the first threshold (step S48). Specifically, the cause determination unit 52 determines the cause of the driver's emotion level becoming equal to or higher than the first threshold, for example, based on video data of the vehicle's surroundings. More specifically, when the cause determination unit 52 obtains video data of another vehicle cutting in when the driver's emotion level is determined to be equal to or higher than the first threshold, the cause determination unit 52 determines that the other vehicle cutting in is the cause of the emotion level becoming equal to or higher than the first threshold. That is, when the cause determination unit 52 obtains video data including an event that may affect the driver's emotion when the driver's emotion level becomes equal to or higher than the first threshold, the cause determination unit 52 determines that the event is the cause of the driver's emotion level becoming equal to or higher than the first threshold. Then, the process proceeds to step S50.

[0068] The processes from step S50 to step S54 are the same as the processes from step S16 to step S20 shown in FIG. 2, respectively, and therefore will not be described.

[0069] If it is determined that the result of step S52 is No, the cause determination unit 52 determines whether or not the emotion level has become equal to or higher than the first threshold value due to the same cause as the cause identified in step S48 in the past for a predetermined period of time (step S56). Specifically, the cause determination unit 52 determines whether or not the emotion level has become equal to or higher than the first threshold value due to the same cause as the cause identified in step S48, for example, based on the cause information stored in the storage unit 26. The predetermined period in step S56 may be set arbitrarily, for example, to 30 minutes, 1 hour, 6 hours, or 12 hours, depending on the driver's personality, etc. If it is determined that the emotion level has become equal to or higher than the first threshold value due to the same cause as the cause identified in step S48 (step S56; Yes), the process proceeds to step S58. If it is not determined that the emotion level has become equal to or higher than the first threshold value due to the same cause as the cause identified in step S48 (step S56; No), the process proceeds to step S62.

[0070] If it is determined as Yes in step S56, the driving support decision unit 48A decides the content of the driving support according to the cause identified in step S48 (step S58). Specifically, the driving support decision unit 48 decides the content of the driving support for calming the emotion according to the cause, for example. Then, the process proceeds to step S60.

[0071] The driving support execution unit 50A executes driving support according to the cause determined in step S58 (step S60). Specifically, the driving support execution unit 50A executes driving support to calm the driver's emotions. More specifically, the notification control unit 50Ab causes the audio output unit 24 to output a voice such as, for example, "A situation that is likely to change your emotions has occurred. Please be careful." Then, the process proceeds to step S70.

[0072] The processes from step S62 to step S70 are the same as the processes from step S22 to step S30 shown in FIG. 2, respectively, and therefore will not be described.

[0073] As described above, the second embodiment identifies the cause of the change in the current and past emotion levels of the driver of the vehicle, and performs driving assistance based on the identified cause. As a result, the second embodiment can appropriately perform driving assistance according to the cause of the change in the driver's emotion, thereby improving safety.

[0074] [Modification of the second embodiment] In the above-described second embodiment, the driving assistance system 1A has been described as providing driving assistance to the driver when the driver drives the vehicle. The driving assistance system 1A may provide parking assistance when the driver parks the vehicle.

[0075] The driving assistance device 14A may perform parking assistance if the driver's emotional level when parking the vehicle is equal to or higher than a first threshold and if the emotional level has been equal to or higher than the first threshold for the same reason when parking during a specified period in the past.

[0076] Specifically, the emotion determination unit 44 determines the current emotion level of the driver when the driver parks the vehicle. The cause determination unit 52 determines whether the emotion level has been equal to or higher than the first threshold value due to the same cause in the past for a predetermined period of time based on the cause information stored in the storage unit 26. The driving support determination unit 48A determines the content of parking support when the emotion determination unit 44 determines that the current emotion level of the driver is equal to or higher than the first threshold value and the cause determination unit 52 determines that the emotion level has been equal to or higher than the first threshold value in the past for the same cause in the past for a predetermined period of time. The driving support execution unit 50A executes the parking support determined by the driving support determination unit 48A. As a result, the second embodiment can appropriately execute parking support according to the cause of the change in the driver's emotion, thereby improving safety.

[0077] [Third embodiment] An example of the configuration of the driving assistance system according to the third embodiment will be described with reference to Fig. 16. Fig. 16 is a block diagram showing an example of the configuration of the driving assistance system according to the third embodiment.

[0078] As shown in FIG. 16, the driving assistance system 1B differs from the driving assistance system 1 shown in FIG. 1 in that the driving assistance device 14B includes a GNSS (Global Navigation Satellite System) receiver 32 and the control unit 30B includes a position information acquisition unit 54 and a position information determination unit 56.

[0079] The GNSS receiver 32 receives GNSS signals including information for identifying position information from GNSS satellites (not shown). The GNSS receiver 32 can be realized by, for example, a GNSS receiver circuit or a GNSS receiver device.

[0080] The position information acquisition unit 54 acquires the GNSS signal from the GNSS receiving unit 32. The position information acquisition unit 54 calculates the current position information of the driver (driving support device 14B) based on the GNSS signal. For example, the position information acquisition unit 54 calculates the current position of the driver (driving support device 14B) in Earth coordinates. For example, the position information acquisition unit 54 associates the emotion level of the driver that is equal to or higher than the first threshold determined by the emotion determination unit 44 with the location information where the emotion level has changed, and stores the emotion change location information indicating the location where the driver's emotion level became equal to or higher than the first threshold in the storage unit 26. Note that, if the predetermined period in the past is before getting into the vehicle, the position information acquisition unit 54 may acquire the location information from a wearable device worn by the driver.

[0081] The position information determination unit 56 determines whether or not the current position is a position where the driver's emotion has changed. For example, the position information determination unit 56 determines whether or not the current position is a position where the driver's emotion has changed based on the position information acquired by the position information acquisition unit 54 and the emotion change position information stored in the storage unit 26.

[0082] (Driving assistance control processing) The flow of processing of driving assistance control according to the third embodiment will be described with reference to Fig. 17. Fig. 17 is a flowchart showing an example of the flow of driving assistance control processing according to the third embodiment.

[0083] The position information acquisition unit 54 acquires position information of the driver (driving support device 14B) (step S80). Specifically, the position information acquisition unit 54 calculates the position information of the driver (driving support device 14B) based on the GNSS signal acquired from the GNSS receiving unit 32. Then, the process proceeds to step S82.

[0084] The processes from step S82 to step S86 are the same as the processes from step S10 to step S14 shown in FIG. 2, respectively, and therefore will not be described.

[0085] If the result of the determination in step S86 is Yes, the position information acquisition unit 54 identifies the current position (step S88). Specifically, the position information acquisition unit 54 identifies, for example, the position information of the point where the driver's emotion level is equal to or higher than the first threshold. Then, the process proceeds to step S90.

[0086] The processes from step S90 to step S94 are the same as the processes from step S16 to step S20 shown in FIG. 2, respectively, and therefore will not be described.

[0087] Specifically, the position information determination unit 56 determines whether or not the driver's emotion level at the current position has been equal to or higher than the first threshold within a predetermined period of time in the past (step S96). Specifically, the position information determination unit 56 determines whether or not the driver's emotion level at the current position has been equal to or higher than the first threshold within a predetermined period of time in the past, based on the current position information acquired by the position information acquisition unit 54 and the emotion change position information stored in the storage unit 26. If it is determined that the driver's emotion level at the current position has been equal to or higher than the first threshold within a predetermined period of time in the past (step S96; Yes), the process proceeds to step S98. If it is not determined that the driver's emotion level at the current position has been equal to or higher than the first threshold within a predetermined period of time in the past (step S96; No), the process proceeds to step S102.

[0088] If it is determined as Yes in step S96, the driving support determination unit 48B determines the content of the driving support according to the current position identified in step S88 (step S98). Specifically, the driving support determination unit 48B determines the content of the driving support for calming the emotion according to the current position, for example. Then, the process proceeds to step S100.

[0089] The driving support execution unit 50B executes driving support according to the cause determined in step S98 (step S100). Specifically, the driving support execution unit 50B executes driving support to calm the driver's emotions. More specifically, the notification control unit 50Bb causes the audio output unit 24 to output a voice such as, for example, "You are traveling in an area where emotions are likely to change. Please be careful." Then, the process proceeds to step S110.

[0090] The processes from step S102 to step S110 are the same as the processes from step S22 to step S30 shown in FIG. 2, respectively, and therefore will not be described.

[0091] As described above, the third embodiment identifies the location information of the location where the current and past emotion levels of the driver of the vehicle have changed, and performs driving assistance based on the identified location information. As a result, the third embodiment can perform driving assistance appropriately according to the location when the vehicle is traveling in a location where the driver's emotion is likely to change, thereby improving safety.

[0092] [Modification of the third embodiment] In the above-described third embodiment, the driving assistance system 1B has been described as providing driving assistance to the driver when the driver drives the vehicle. The driving assistance system 1B may provide parking assistance when the driver parks the vehicle.

[0093] The driving assistance device 14B may perform parking assistance if the driver's emotional level when parking the vehicle is equal to or greater than a first threshold and has been equal to or greater than the first threshold when parking in the current parking location within a predetermined period of time in the past.

[0094] Specifically, the emotion determination unit 44 determines the current emotion level of the driver when the driver parks the vehicle. The position information determination unit 56 determines whether or not the emotion level of the driver has become equal to or higher than the first threshold value within a predetermined period of time in the past when parking at the current position, based on the emotion change position information stored in the storage unit 26. The driving assistance determination unit 48B determines the content of parking assistance when the emotion determination unit 44 determines that the current emotion level of the driver is equal to or higher than the first threshold value and the position information determination unit 56 determines that the emotion level of the driver has become equal to or higher than the first threshold value within a predetermined period of time in the past at the current position. The driving assistance execution unit 50B executes the parking assistance determined by the driving assistance determination unit 48B. As a result, the third embodiment can appropriately execute parking assistance according to the location where the driver's emotion has changed, thereby improving safety.

[0095] Although the embodiments of the present invention have been described above, the present invention is not limited to the contents of these embodiments. The above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments. [Explanation of symbols]

[0096] 1,1A,1B Driving Support System 10. Imaging device 10a First Camera 10b Second Camera 12 Biometric sensors 14, 14A, 14B Driving support device 20 Input section 22 Display section 24 Audio output section 26 Memory section 28 Communications Department 30, 30A, 30B Control unit (driving assistance control device) 32 GNSS receiver 40 Video data acquisition unit 42 Biometric information acquisition unit 44 Emotion Judgment Department 46 Driving Condition Prediction Unit 48, 48A, 48B Driving support decision unit 50, 50A, 50B Driving Support Execution Department 50a Vehicle control unit 50b Notification control unit 52 Cause determination section 54 Location information acquisition unit 56 Location information determination unit

Claims

1. A biometric information acquisition unit that acquires current and past biometric information of a vehicle driver for a predetermined period of time; an emotion determination unit that determines an emotion level indicating a degree of emotion of the driver in the present and the predetermined past period based on the biological information acquired by the biological information acquisition unit; a driving situation prediction unit that predicts a driving situation of the vehicle based on the current and past emotion levels of the driver during the predetermined period when the current emotion level of the driver is determined to be equal to or higher than a first threshold; a driving assistance determination unit that determines whether or not driving assistance of the vehicle is necessary based on a result of the prediction of the driving situation of the vehicle by the driving situation prediction unit, and determines the content of the driving assistance of the vehicle when the driving assistance of the vehicle is necessary; a driving assistance execution unit that executes the driving assistance determined by the driving assistance determination unit; A driving assistance control device comprising:

2. the driving assistance decision unit changes the content of the driving assistance for the vehicle depending on the number of times that the emotion level has been determined to be equal to or higher than a second threshold value, which is higher than the first threshold value, within the predetermined period in the past. The driving assistance control device according to claim 1 .

3. the driving assistance determination unit changes content of the driving assistance for the vehicle depending on a duration during which an emotion level equal to or higher than a second threshold continued within the predetermined period in the past. The driving assistance control device according to claim 1 or 2.

4. A video data acquisition unit that acquires video data of the driver, The emotion determination unit determines an emotion level indicating a degree of emotion of the driver during the predetermined period in the past based on the video data. The driving assistance control device according to any one of claims 1 to 3.

5. acquiring current and past biometric information of a driver of a vehicle; determining an emotion level indicating a degree of emotion of the driver in the present and the predetermined past period based on the acquired biological information; When the current emotion level of the driver is determined to be equal to or higher than a first threshold, predicting a driving situation of the vehicle based on the current and past emotion levels of the driver during the predetermined period; A step of determining whether or not driving assistance for the vehicle is necessary based on a prediction result of a driving situation of the vehicle, and determining the content of the driving assistance for the vehicle when the driving assistance for the vehicle is necessary; Executing the determined driving assistance; A driving assistance control method comprising:

Images