Notification system, vehicle control device, notification method, and program
The notification system addresses the issue of desensitization to vehicle alerts by varying the mobile terminal's accessibility and display based on alert urgency, ensuring critical notifications are acknowledged.
Patent Information
- Application Number
- JP2023190387
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-01
- Filing Date
- 2023-11-07
- Publication Date
- 2024-11-14
AI Technical Summary
In conditional automated driving scenarios, drivers may become desensitized to important vehicle notifications due to receiving all notifications uniformly, making it difficult to notice critical alerts.
A notification system that varies the controllable range of a driver's mobile terminal based on the urgency of the notification information, restricting access and display until a predetermined condition is met, ensuring critical alerts are prioritized and acknowledged.
Prevents drivers from missing important vehicle notifications by adjusting the mobile terminal's functionality and display based on urgency, enhancing the driver's awareness of critical alerts.
Smart Images

Figure 2024160674000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a notification system, a vehicle control device, a notification method, and a program. [Background technology]
[0002] Patent Document 1 discloses a driver monitoring system that monitors a vehicle driver by communicating between a mobile terminal and a vehicle control device that controls the vehicle. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2020-102252 A Summary of the Invention [Problem to be solved by the invention]
[0004] According to conditional automated driving, it is expected that the driver of the vehicle will be temporarily released from the obligation to pay attention to the road ahead and will be able to perform tasks other than driving operations, such as operating a mobile device such as a smartphone. In this case, the driver is required to be able to return to driving operations at any time upon request from the vehicle's system. In this case, it is considered effective to convey notifications and requests from the vehicle's system to the driver using a mobile device. However, if all notifications are notified to the driver in the same way, the driver may find the notifications annoying and lose interest in the notifications, which may make it difficult for the driver to notice important notifications regarding the vehicle.
[0005] The present invention has been made in consideration of the above circumstances, and aims to provide a notification system, a vehicle control device, a notification method, and a program that can prevent a vehicle driver from having difficulty noticing important notifications regarding the vehicle. [Means for solving the problem]
[0006] The first aspect of the notification system is a notification system including a mobile terminal capable of receiving and notifying notification information regarding a vehicle, and a vehicle control device that generates and transmits the notification information to a driver of the vehicle, and includes a notification control unit that controls the mobile terminal to notify the notification information by varying the range that the driver can control until a specified condition is satisfied, depending on the urgency of the notification information.
[0007] According to the notification system of the first aspect, the notification information is notified to the mobile terminal by changing the range that the driver can control until a predetermined condition is satisfied, depending on the urgency of the notification information. Therefore, the driver can recognize the importance of the notification information, and it is possible to prevent the driver from becoming unaware of important notifications.
[0008] A notification system of a second aspect is the notification system of the first aspect, wherein the urgency is determined depending on whether or not the notification information is information requiring a response by the driver.
[0009] According to the notification system of the second aspect, it is possible to suppress the driver from becoming less aware of notifications that require a response from the driver.
[0010] The notification system of the third aspect is the notification system of the first or second aspect, wherein the urgency is determined according to the degree of danger of the event.
[0011] According to the notification system of the third aspect, it is possible to suppress the driver from becoming less aware of notifications relating to events with a high degree of risk.
[0012] A fourth aspect of the notification system is a notification system according to any one of the first to third aspects, wherein the notification control unit controls the mobile terminal so that, for notification information where the urgency is at or above a certain level, the driver cannot operate the mobile terminal until the driver operates the vehicle, and then notifies the notification information.
[0013] According to the notification system of the fourth aspect, it is possible to more effectively prevent the driver of the vehicle from becoming unaware of important notifications regarding the vehicle.
[0014] A fifth aspect of the notification system is a notification system according to any one of the first to fourth aspects, in which the notification control unit controls the notification information such that, for notification information where the urgency is a certain level or higher, a confirmation button is displayed on the display device of the mobile terminal, and the mobile terminal cannot be operated except for pressing the confirmation button until the driver presses the confirmation button, thereby notifying the notification information.
[0015] According to the notification system of the fifth aspect, it is possible to more effectively prevent the driver of the vehicle from becoming unaware of important notifications regarding the vehicle.
[0016] A sixth aspect of the notification system is a notification system of any one of the first to fifth aspects, in which the notification control unit provides notification on the mobile terminal according to the priority order between first notification information generated on the mobile terminal and second notification information which is notification information related to the vehicle.
[0017] According to the notification system of the sixth aspect, it is possible to more effectively prevent the driver of the vehicle from becoming unaware of important notifications related to the vehicle and important notifications generated on the mobile terminal.
[0018] A vehicle control device of a seventh aspect includes an acquisition unit that acquires notification information to be sent to a mobile device held by a driver of a vehicle and notified to the driver, and a notification control unit that controls the notification information to be notified to the mobile device by changing the range that the driver can control until a predetermined condition is satisfied, depending on the urgency of the notification information.
[0019] The notification method of the eighth aspect is a notification method executed by a computer constituting a notification system including a mobile terminal capable of receiving and notifying notification information regarding a vehicle, and a vehicle control device that generates and transmits the notification information to a driver of the vehicle, in which the computer executes a process to control the mobile terminal to notify the notification information by varying the range that the driver can control until a predetermined condition is satisfied, depending on the urgency of the notification information.
[0020] The program of the ninth aspect is a program to be executed by a computer constituting a notification system including a mobile terminal capable of receiving and notifying notification information regarding a vehicle, and a vehicle control device that generates and transmits the notification information to a driver of the vehicle, and causes the computer to execute a process of controlling the mobile terminal to notify the notification information by changing the range that the driver can control until a predetermined condition is satisfied, depending on the urgency of the notification information. Effect of the Invention
[0021] According to the present disclosure, it is possible to prevent a vehicle driver from having difficulty noticing important notifications regarding the vehicle. [Brief description of the drawings]
[0022] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a configuration of a notification system. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of the vehicle control device. [Diagram 3] FIG. 11 is a diagram illustrating an example of urgency data. [Figure 4] FIG. 13 is a diagram illustrating an example of an urgency level. [Diagram 5] 11 is a diagram showing an example of notification information, an event, a user action, and an urgency level. FIG. [Figure 6] FIG. 13 is a diagram illustrating an example of a relationship between events. [Figure 7] FIG. 11 is a diagram illustrating an example of the urgency of a notification on the mobile terminal side. [Figure 8] FIG. 13 is a diagram showing an example in which urgency is divided into six stages. [Figure 9] FIG. 2 is a block diagram showing an example of a hardware configuration of a mobile terminal. [Figure 10] FIG. 2 is a block diagram showing an example of a functional configuration of a vehicle control device. [Figure 11] 4 shows an example of information provided from a mobile terminal to a vehicle. [Figure 12] FIG. 11 is a diagram illustrating an example of a notification state of a mobile terminal. [Figure 13] FIG. 11 is a diagram illustrating an example of a notification state of a mobile terminal. [Figure 14] FIG. 11 is a diagram showing an example of a notification state of an in-vehicle display device. [Figure 15] FIG. 11 is a diagram illustrating an example of a notification state of a mobile terminal. [Figure 16] FIG. 2 is a block diagram showing an example of a functional configuration of a mobile terminal. [Figure 17] 5 is a flowchart showing an example of a notification control process executed by the vehicle control device. [Figure 18] 11 is a flowchart illustrating an example of a notification process executed on the mobile terminal. [Figure 19] FIG. 1 is a diagram illustrating an example of directionality of notification information exchanged in a notification system. [Figure 20] FIG. 11 is a diagram showing a first example of stratification of urgency. [Figure 21] FIG. 21 is a diagram showing first notification information corresponding to each of the areas A-1 to A-5 in FIG. 20, an example of an event for which the first notification information is notified, and an example of a user action required in response to the notified first notification information. [Figure 22] FIG. 13 is a diagram showing a second example of urgency stratification. [Figure 23] 23 is a diagram showing second notification information corresponding to each of the areas B-1 to B-3 in FIG. 22, an example of an event for notifying the second notification information, and an example of a user action for notifying the second notification information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0023] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0024] (First embodiment) First, the configuration of a notification system 1 according to this embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the notification system 1 includes a vehicle control device 12 provided inside a vehicle 10, and a mobile terminal 14 carried by the driver and brought into the vehicle 10 by the driver. An example of the vehicle control device 12 is a computer such as an ECU (Electronic Control Unit). An example of the mobile terminal 14 is a computer such as a smartphone, a tablet terminal, or a smartwatch. The vehicle control device 12 according to this embodiment is applied to an autonomous vehicle that allows switching between autonomous driving and manual driving.
[0025] The mobile terminal 14 is capable of receiving and notifying notification information related to the vehicle 10. The vehicle control device 12 generates notification information for the driver of the vehicle 10 and transmits the notification information to the mobile terminal 14. This notification information is information to be notified to the driver while autonomous driving is being performed.
[0026] Next, a hardware configuration of the vehicle control device 12 according to this embodiment will be described with reference to Fig. 2. The vehicle control device 12 includes a CPU (Central Processing Unit) 20, a ROM (Read Only Memory) 21, a RAM (Random Access Memory) 22, a wireless communication I / F (Interface) 23, an in-vehicle communication I / F 24, and a communication unit 25. The CPU 20, the ROM 21, the RAM 22, the wireless communication I / F 23, the in-vehicle communication I / F 24, and the communication unit 25 are connected via a bus 27 so as to be able to communicate with each other.
[0027] The CPU 20 is a central processing unit that executes various programs and controls each part. That is, the CPU 20 reads out a program from the ROM 21 and executes the program using the RAM 22 as a working area.
[0028] The ROM 21 serving as a storage unit stores various programs and various data. In this embodiment, the ROM 21 stores a control program 30 and urgency data 32. The RAM 22 serves as a working area and temporarily stores the programs and data.
[0029] The wireless communication I / F 23 is an interface for connecting to the mobile terminal 14 by wireless communication. In the present embodiment, a configuration in which the vehicle control device 12 and the mobile terminal 14 communicate directly will be described as an example, but the disclosed technology is not limited to this. For example, the vehicle control device 12 and the mobile terminal 14 may communicate with each other via a server computer or the like provided on a network.
[0030] The in-vehicle communication I / F 24 is an interface for connecting to in-vehicle devices such as the display device 15, the external sensor 100, the in-vehicle sensor 101, the map data 102, the external HMI (Human Machine Interface) device 103, the haptic device 104, the audio device 105, the input device 106, and the autonomous driving ECU 107. The interface uses a communication standard based on the CAN (Controller Area Network) protocol.
[0031] The communication unit 25 is an interface for connecting to a network when communicating with an external server such as the cloud server 16. For this interface, a communication standard such as LTE (Long Term Evolution) is used. The vehicle control device 12 can also communicate with the mobile terminal 14 via the cloud server 16.
[0032] The display device 15 displays the message under the control of the vehicle control device 12. An example of the display device 15 is a liquid crystal display provided on the instrument panel.
[0033] The external sensor 100 includes various sensors such as a camera, a radar, and a millimeter wave radar that acquire various information about the environment surrounding the vehicle 10. The in-vehicle sensor 101 includes various sensors such as a driver monitoring camera that acquires various information about the interior of the vehicle. The map data 102 is provided from a navigation device or the like. The exterior HMI device 103 outputs various signals to people outside the vehicle such as pedestrians and cyclists. The haptic device 104 provides feedback to the occupant's tactile sense by applying force, vibration, movement, etc. to the occupant of the vehicle 10, such as a device that vibrates a seat belt. The audio device 105 includes an audio output device such as a speaker. The input device 106 is a device for the occupant of the vehicle 10 to perform input, such as various switches and touch sensors.
[0034] The autonomous driving ECU 107 controls autonomous driving of the vehicle 10. Note that autonomous driving in this embodiment means, as an example, autonomous driving in which the vehicle 10 is the main driver, and means autonomous driving of level 3 or higher defined by the Society of Automotive Engineers (SAE). The autonomous driving ECU 107 is configured to be capable of autonomous driving processing of level 3 or higher. The autonomous driving ECU 107 is connected to an actuator system 108. The actuator system 108 includes a powertrain system 110, a brake system 111, and a steering system 112. The autonomous driving ECU 107 controls the powertrain system 110, the brake system 111, and the steering system 112.
[0035] The powertrain system 110 includes a powertrain capable of generating a driving force to the drive wheels of the vehicle 10, and an ECU that controls the operation of the powertrain. The brake system 111 includes a plurality of braking devices provided on each wheel of the vehicle 10. The braking devices include, for example, hydraulic brakes such as disc brakes that generate braking force using hydraulic pressure. The steering system 112 includes a steering device capable of changing the steering angle of the steered wheels of the vehicle 10, and an ECU that controls the operation of the steering device.
[0036] Fig. 3 shows an example of the urgency data 32. As shown in Fig. 3, the urgency data 32 stores the urgency of the notification information (hereinafter simply referred to as "urgency") in association with a combination of the urgency of the time until the driver takes action and the risk of the event. This urgency represents the urgency of the driver's action while autonomous driving is being performed. In this embodiment, the higher the urgency of the time, the higher the urgency, and the higher the risk of the event, the higher the urgency.
[0037] For example, as shown in FIG. 3, when the time urgency is low and the event risk is low, the urgency is low. When the time urgency is low and the event risk is medium, the urgency is low. When the time urgency is low and the event risk is high, the urgency is medium. Thus, in this embodiment, as an example, when the time urgency is low and the event risk is low, the urgency is low, similar to the time urgency and the event risk. Also, in this embodiment, as an example, when the time urgency is low and the event risk is medium, the urgency is low by applying the degree of time urgency. Also, in this embodiment, as an example, when the time urgency is low and the event risk is high, the urgency is medium by taking a middle value between the time urgency and the event risk.
[0038] For example, as shown in FIG. 3, when the time urgency is medium and the event risk is low, the urgency is low. When the time urgency is medium and the event risk is medium, the urgency is medium. When the time urgency is medium and the event risk is high, the urgency is high. Thus, in this embodiment, as an example, when the time urgency is medium and the event risk is low, the urgency is low by applying the event risk level. When the time urgency is medium and the event risk is medium, the urgency is medium, like the time urgency and the event risk. When the time urgency is medium and the event risk is high, the urgency is high by applying the event risk level.
[0039] For example, as shown in FIG. 3, when the time urgency is high and the event risk is low, the urgency is medium. When the time urgency is high and the event risk is medium, the urgency is high. When the time urgency is high and the event risk is high, the urgency is high. In this manner, in the present embodiment, as an example, when the time urgency is high and the event risk is low, the urgency is medium between the time urgency and the event risk. When the time urgency is high and the event risk is medium, the urgency is high by applying the degree of time urgency. When the time urgency is high and the event risk is high, the urgency is high like the time urgency and the event risk.
[0040] In the example of FIG. 3, the urgency is divided into three stages, but the urgency may be divided into two stages or four or more stages. In the example of FIG. 3, the time urgency is divided into three stages, but the time urgency may be divided into two stages or four or more stages. In the example of FIG. 3, the risk of an event is divided into three stages, but the risk of an event may be divided into two stages or four or more stages. The urgency may be associated only with the risk of an event. The urgency may be determined depending on whether or not the notification information related to the vehicle is information that requires a response from the driver. For example, the urgency of notification information that requires a response from the driver is set to "medium" or "high", and the urgency of notification information that does not require a response from the driver is set to "low".
[0041] As an example, as shown in FIG. 4, the risk of an event is determined according to weather conditions, obstacles, the state of each part constituting the vehicle 10, information on the road, and the like. The vertical axis of FIG. 4 indicates the risk of the event, and the horizontal axis of FIG. 4 indicates the time urgency. For example, in FIG. 4, the risk of the event is set in the order of increasing to an event that does not affect the running of the vehicle 10, an event that deteriorates the automatic driving performance of the vehicle 10, an event that makes it impossible to continue the automatic driving of the vehicle 10, and an accident event of the vehicle 10. In the case of a failure of each part constituting the vehicle 10, the time urgency is determined according to the degree of the failure. In addition, there are also events in which the time urgency is set according to the time until the occurrence of an event such as a collision with an obstacle, a change in weather, and reaching a fork in the road.
[0042] The risk level and time urgency of the event are determined according to the acquired vehicle status and surrounding conditions, and the urgency is associated on a map as shown in FIG. 4 as an example. The urgency of the event also changes from time to time according to changes in the vehicle status and surrounding conditions. For example, the time urgency of a fallen obstacle, a change in weather, a road fork, etc. increases as the vehicle travels and approaches the target (i.e., time is running out). For example, assume that the vehicle control device 12 detects the occurrence of an event (e.g., heavy rain or thick fog) that makes it impossible to continue the automatic driving of the vehicle 10. In this case, as an example, whether or not there is a time margin of about 10 to 20 seconds until the event occurs is set as one of the boundaries between the medium and high urgency. In other words, a value of about 10 to 20 seconds until the event occurs is set as one of the boundaries between the medium and high urgency. As an example, whether or not there is a time margin of about 60 to 180 seconds until the event occurs is set as one of the boundaries between the low and medium urgency. That is, a value of about 60 to 180 seconds until the occurrence of the event is one of the boundaries between low and medium urgency.
[0043] FIG. 5 shows an example of notification information classification, events, and user actions and urgency for the notification information.
[0044] As shown in FIG. 5, the classification of the notification information according to type 1A is a driving change warning. The event item for which the notification information is notified is an "imminent event" such as going outside an operational design domain (ODD) or encountering a dangerous situation. For example, when the vehicle control device 12 detects going outside the operational design domain, the notification information is notified. Then, the user, specifically, the driver, needs to immediately respond to the notification information in order to resume the manual driving mode. The urgency of the notification information is high.
[0045] As shown in FIG. 5, the classification of the notification information according to type 1B is a notification of a driver changeover. The event item for which the notification information is notified is a "predicted event" such as a system failure or malfunction, or a sudden change in weather. For example, when the vehicle control device 12 detects a system failure, the notification information is notified. In response to the notification information, the driver needs to take action to resume the manual driving mode by TBD (to be determined), specifically, by the time a driver changeover is determined. The urgency of the notification information is medium to high.
[0046] As shown in FIG. 5, the classification of the notification information according to type 2A is system state transition (Minimal Risk Maneuver (MRM)). The event item for which the notification information is notified is transition to MRM. For example, when the vehicle control device 12 decides to transition to MRM, the notification information is notified. The action required of the driver in response to the notification information is "none." In other words, the driver does not need to take any action when the notification information is notified. The urgency of the notification information is medium to high. Alternatively, the driver may be requested to take some action when the notification information is notified.
[0047] As shown in FIG. 5, the classification of the notification information according to type 2B is system state transition (Minimal Risk Condition (MRC)). The event item for which the notification information is notified is transition to MRC. For example, when the vehicle control device 12 decides to transition to MRC, the notification information is notified. The action required of the driver in response to the notification information is "none". In other words, the driver does not need to take any action when the notification information is notified. The urgency of the notification information is medium to high.
[0048] As shown in FIG. 5, the classification of the notification information according to type 3 is a function proposal from the system. The event item for which the notification information is notified is a function proposal from the system, such as overtaking or passing (lane change). For example, when the vehicle control device 12 proposes to execute overtaking, the notification information is notified. In response to the notification information, the driver needs to approve the execution of the proposed function. The urgency of the notification information is low to medium.
[0049] As shown in FIG. 5, the classification of the notification information according to type 4 is a notification of the vehicle state. The event item for which the notification information is notified is the remaining battery or fuel. For example, when the vehicle control device 12 detects a predetermined remaining battery level, the notification information is notified. The notification information is only a notification, and the action required of the driver in response to the notification information is "none." In other words, the driver does not need to take any action when the notification information is notified. The urgency of the notification information is low to medium.
[0050] Moreover, Fig. 6 shows an example of the relationship between events. For example, as shown in Fig. 6, "system failure notification" and "transition notification to MRM" are mutually related. Specifically, "transition notification to MRM" is connected by an arrow extending from "system failure notification". In this case, after notifying "system failure notification" as notification information, if the degree of time urgency becomes high, the vehicle control device 12 notifies "transition notification to MRM" as related notification information.
[0051] Moreover, Fig. 7 shows an example of the urgency of notification on the mobile terminal 14 side corresponding to Fig. 4. In Fig. 7, the urgency of notification on the mobile terminal 14 side is represented by a filled black rectangle. In Fig. 7, the urgency of notification on the vehicle 10 side is represented by a filled black circle. Also, the dashed straight lines in Fig. 7 indicate the boundaries of the urgency levels.
[0052] As an example, in FIG. 7, low (no action required) and low are provided as the urgency of notifications on the mobile terminal 14 side. The urgency increases in the order of low (no action required) and low. For example, low urgency (no action required) includes normal notifications from the mobile terminal 14 such as low battery level and schedule notifications. No action is required from the driver for low urgency (no action required). For example, low urgency includes an incoming call to the mobile terminal 14. It is considered that the driver responds to an incoming call to the mobile terminal 14 with low urgency by answering the call or hanging up the phone.
[0053] As an example, in FIG. 7, low, medium, and high are provided as urgency levels of notification on the vehicle 10 side. The urgency levels increase in the order of low, medium, and high. For example, a low urgency level includes a branch presence notification that notifies the presence of a branch on the travel route of the vehicle 10. For example, a medium urgency level includes a sensor performance degradation notification that the performance of a sensor of the vehicle 10 has deteriorated or will deteriorate. For example, a high urgency level includes an unavoidable collision obstacle detection notification that notifies the presence of an obstacle with which a collision with the vehicle 10 is unavoidable.
[0054] Moreover, an example in which the urgency levels corresponding to Fig. 7 are divided into six levels is shown in Fig. 8. In Fig. 8, the urgency levels increase in the order of low → medium 3 → medium 2 → medium 1 → high 2 → high 1. In this way, the levels of urgency are not particularly limited and can be set appropriately.
[0055] Next, a hardware configuration of the mobile terminal 14 according to the present embodiment will be described with reference to Fig. 9. The mobile terminal 14 includes a CPU 40, a ROM 41, a RAM 42, a wireless communication I / F 43, and a touch panel 44. The CPU 40, the ROM 41, the RAM 42, the wireless communication I / F 43, and the touch panel 44 are connected via a bus 47 so as to be able to communicate with each other.
[0056] The CPU 40 is a central processing unit that executes various programs and controls each part. That is, the CPU 40 reads out a program from the ROM 41 and executes the program using the RAM 42 as a working area.
[0057] The ROM 41 serving as a storage unit stores various programs and various data. In this embodiment, the ROM 41 stores an output program 50. The RAM 42 serves as a working area and temporarily stores the programs and data.
[0058] The wireless communication I / F 43 is an interface for connecting via wireless communication to the vehicle control device 12. The touch panel 44 includes a liquid crystal display as a display device and a touch sensor as an input device.
[0059] Next, a functional configuration of the vehicle control device 12 according to this embodiment will be described with reference to Fig. 10. As shown in Fig. 10, the vehicle control device 12 includes an acquisition unit 60, a derivation unit 62, and a notification control unit 64. The CPU 20 executes a program stored in the ROM 21 to function as the acquisition unit 60, the derivation unit 62, and the notification control unit 64. The notification control unit 64 may be provided in the mobile terminal 14 or the cloud server 16. The notification control unit 64 may be provided in more than one of the vehicle control device 12, the mobile terminal 14, and the cloud server 16.
[0060] The acquisition unit 60 acquires notification information related to the vehicle 10. For example, the acquisition unit 60 acquires various types of information as notification information, such as information from the external sensor 100, information from the in-vehicle sensor 101, peripheral information from the navigation device and map data 102, and information that can be acquired via the communication unit 25 and the mobile terminal 14. Alternatively, the acquisition unit 60 acquires, via the in-vehicle communication I / F 24, notification information generated by another ECU, such as the autonomous driving ECU 107, that has acquired the information.
[0061] FIG. 11 shows an example of information provided from the mobile terminal 14 to the vehicle 10. As shown in FIG. 11, the classification of the information includes the state of the occupant (driver) and information input. Here, the items of the driver's state include camera information showing at least one video of the driver, the interior of the vehicle, and the exterior of the vehicle, and biometric information such as the driver's body temperature, blood pressure, heart rate, and skin electrical resistance. Based on the above camera information provided from the mobile terminal 14, the ADS (Advanced Drive System) of the vehicle 10, specifically, the vehicle control device 12, can estimate the driving state of the driver, such as whether the driving authority can be delegated to the driver or not. Further, based on the above biometric information provided from the mobile terminal 14, the vehicle control device 12 can estimate the health state of the driver and take reactions such as "prompt for rest" or "emergency stop the vehicle 10". Note that the estimation of the driving state and the health state of the driver by the vehicle control device 12 is performed by appropriately using known processes. Also, among the classification of the information, the items of information input include destination search and destination setting of the vehicle 10, etc.
[0062] Returning to FIG. 10, the derivation unit 62 derives the urgency of the time until the driver takes action during the execution of the automatic driving based on the notification information acquired by the acquisition unit 60. For example, when the notification information includes a time limit such as "There are still YY seconds until XX", the derivation unit 62 uses the time limit to derive the urgency of the time. The derivation unit 62 according to the present embodiment derives one of low, medium, and high as the urgency of the time based on the time limit and two threshold values TH1, TH2 (TH1 < TH2). The derivation unit 62 derives "high" as the urgency of the time when the time limit is equal to or less than the threshold value TH1, derives "medium" as the urgency of the time when the time limit exceeds the threshold value TH1 and is equal to or less than the threshold value TH2, and derives "low" as the urgency of the time when the time limit exceeds the threshold value TH2. For example, the threshold value TH1 is set to 10 seconds, and the threshold value TH2 is set to 60 seconds.
[0063] Also, for example, when the notification information includes a distance such as "XX is YY km away," the derivation unit 62 derives the predicted arrival time by dividing the distance by the vehicle speed. In this case, the derivation unit 62 derives the urgency of time using the predicted arrival time, similar to the above-mentioned time limit. Note that the derivation unit 62 may output the distance to an ECU that controls a car navigation system and have the ECU derive the predicted arrival time.
[0064] For example, if the notification information is information indicating "5 seconds left until outside the operation design area", a time urgency of "high" is derived. Also, for example, if the notification information is information indicating "15 seconds left until outside the operation design area", a time urgency of "medium" is derived. Also, for example, if the notification information is information indicating "75 seconds left until outside the operation design area", a time urgency of "low" is derived. The vehicle 10 entering outside the operation design area means that the vehicle 10 switches from automatic driving to manual driving.
[0065] The derivation unit 62 also derives the risk of the event based on the notification information acquired by the acquisition unit 60. The risk of the event according to the present embodiment indicates how difficult it is to continue the autonomous driving of the vehicle 10. For example, the derivation unit 62 extracts words by performing natural language processing on the notification information, and derives the risk of the event associated with the words. The derivation unit 62 may derive the risk of the event by inputting the notification information to a trained model previously obtained by machine learning. The trained model is, for example, a neural network model that receives the notification information as an input and outputs the risk of the event. The trained model is also obtained by machine learning using a set of the notification information and the correct answer data of the risk of the event as learning data.
[0066] For example, if the notification information notifies of a weather change that will deteriorate the performance of autonomous driving, such as "the weather will change from cloudy to rainy," then the risk level of the event is derived as "low." Also, if the notification information notifies of a weather change that will make it difficult to continue autonomous driving, such as "rain will suddenly get stronger," then the risk level of the event is derived as "medium." For example, if the notification information notifies of a road condition that will make it difficult to continue autonomous driving, such as "the road surface is frozen at the point where you have just exited the tunnel you are currently driving through," then the risk level of the event is derived as "high."
[0067] Furthermore, when the notification information notifies of a malfunction of the vehicle 10, the risk of the event is derived according to the severity of the malfunction.
[0068] In this embodiment, notification information with a "low" level of urgency is information that allows automatic driving to continue even if the driver does not respond to the notification information, such as information provided by the vehicle 10 to the driver, such as overtaking a vehicle traveling in front. In addition, in this embodiment, notification information with a "medium" level of urgency is information that the driver can respond to when sitting in the driver's seat with normal attention. In addition, in this embodiment, notification information with a "high" level of urgency is information that is difficult for the driver to respond to.
[0069] Examples of the notification information include information regarding the remaining battery charge of the vehicle 10, information regarding obstacles that are difficult to avoid, information regarding a failure in one of the redundant systems, information regarding a decrease in sensor performance, etc. Examples of the notification information also include information regarding fallen objects, information regarding natural disasters such as earthquake alerts, information regarding branching of driving paths, information regarding changes in the autonomous driving level, information regarding the remaining amount of fuel in the vehicle 10, etc.
[0070] The derivation unit 62 then derives the urgency corresponding to the combination of the derived urgency of time and the risk of the event, with reference to the urgency data 32. Note that the derivation unit 62 may derive the urgency by summing up the scores of the urgency of time and the risk of the event, or may derive the urgency by weighting either one of them and then summing them up.
[0071] The notification control unit 64 performs control to notify the mobile terminal 14 of the notification information by varying the range that the driver can control until a predetermined condition is satisfied, according to the urgency derived by the derivation unit 62. Furthermore, the notification control unit 64 notifies the notification information by at least one of displaying on the touch panel 44 of the mobile terminal 14, outputting a sound from the mobile terminal 14, and vibrating the mobile terminal 14.
[0072] The notification control unit 64 controls the mobile terminal 14 to be in a state where the driver cannot operate the mobile terminal 14 until the driver operates the vehicle, and notifies the notification information, for notification information derived by the derivation unit 62 for which the urgency level is equal to or higher than a certain level ("medium" in this embodiment). Furthermore, the notification control unit 64 controls the mobile terminal 14 to be in a state where a confirmation button is displayed on the touch panel 44 of the mobile terminal 14, and notifies the notification information, for notification information derived by the derivation unit 62 for which the urgency level is equal to or higher than a certain level, by controlling the mobile terminal 14 to be in a state where the driver cannot operate the mobile terminal 14 except for pressing the confirmation button, until the driver presses the confirmation button.
[0073] Furthermore, the notification control unit 64 increases the occupancy rate of the notification information on the touch panel 44 of the mobile terminal 14 for notification information with a certain level of urgency or more, compared to notification information with a level of urgency below a certain level. For notification information with a level of urgency below a certain level, the notification control unit 64 controls the mobile terminal 14 to be in a state where it can be operated without the driver operating the vehicle, and notifies the notification information. For notification information with a level of urgency below a certain level, the notification control unit 64 controls the mobile terminal 14 to be in a state where a confirmation button is not displayed on the touch panel 44, and the driver can operate the mobile terminal 14 in various ways, and notifies the notification information.
[0074] In other words, examples of the range in which the driver can control mobile terminal 14 before the condition that the driver operates the vehicle or the condition that the driver presses the confirmation button are satisfied include the operable range of mobile terminal 14 and the occupancy rate of notification information on touch panel 44 of mobile terminal 14, etc.
[0075] Specifically, as shown in Fig. 12 as an example, the notification control unit 64 performs control to notify the mobile terminal 14 of notification information with a "low" urgency level, such that the notification information is displayed on a part of the touch panel 44 of the mobile terminal 14 and operations other than checking the notification information are possible on the mobile terminal 14 even before the driver checks the notification information. Fig. 12 shows an example in which the notification information is displayed on the upper part of the touch panel 44 of the mobile terminal 14, but the disclosed technology is not limited to this aspect. For example, the notification information may be displayed in the center or lower part of the touch panel 44 of the mobile terminal 14.
[0076] 13 as an example, the notification control unit 64 performs control so that, for notification information with a "medium" or "high" urgency, the notification information is displayed on the entire surface of the touch panel 44 of the mobile terminal 14, and until the driver presses the confirmation button, operations other than pressing the confirmation button are not possible, and the notification information is notified to the mobile terminal 14. FIG 13 shows an example in which operations other than pressing the confirmation button are not possible on the mobile terminal 14 until the driver confirms the notification information by pressing the confirmation button.
[0077] The notification control unit 64 may control to notify the display device 15 of the notification information. As shown in FIG. 14 as an example, the notification control unit 64 may control to display an image indicating the notification content on the display device 15 in addition to a character string indicating the notification content. The notification control unit 64 may count down the remaining time on the notification screen as time passes. Since the display area of the touch panel 44 of the mobile terminal 14 is often smaller than the display area of the display device 15, the notification control unit 64 may reduce the amount of notification on the touch panel 44 of the mobile terminal 14 compared to the display device 15. In this case, as shown in FIG. 15 as an example, the notification control unit 64 may control to notify the mobile terminal 14 of an icon indicating the notification content without notifying the mobile terminal 14 of a character string indicating the notification content. In this case, the notification control unit 64 may change the background color of the notification screen depending on the urgency. For example, the notification control unit 64 may set the background color to yellow when the urgency is "medium" and set the background color to red when the urgency is "high".
[0078] Furthermore, the notification control unit 64 may perform control to notify the notification information with a larger notification amount as the urgency level increases. Examples of the notification amount include the number of devices that output the notification information, the volume when notifying the notification information by sound, the sound duration when notifying the notification information by sound, the vibration amount when notifying by vibration, and the vibration duration when notifying by vibration.
[0079] For example, when the urgency is "medium" or higher, the notification control unit 64 may perform control to notify the notification information with a larger amount of notification as the urgency becomes higher. Specifically, when the urgency is "medium", the notification control unit 64 performs control to display the notification information on the entire surface of the touch panel 44 of the mobile terminal 14. On the other hand, when the urgency is "high", the notification control unit 64 performs control to display the notification information on the entire surface of the touch panel 44 of the mobile terminal 14 and output the notification information by voice from the speaker of the mobile terminal 14.
[0080] The operation of the mobile terminal 14 and the notification to the mobile terminal 14 may be performed on the application screen via a specific application installed in the mobile terminal 14 by the user. When autonomous driving of level 3 or higher is started, the notification control unit 64 may ignore the setting of the mobile terminal 14 main body side not allowing notifications of the application and control the mobile terminal 14 main body to allow notifications. Alternatively, the mobile terminal 14 main body may be controlled so that notifications of the application cannot be disabled.
[0081] Furthermore, by including in advance in the application data for the screen to be displayed on the touch panel 44 according to each situation and having the application downloaded to the mobile terminal 14, the amount of communication on the mobile terminal 14 can be reduced and the time lag when notifying the mobile terminal 14 can be made smaller.
[0082] Furthermore, even if notification is permitted in mobile terminal 14, there may be cases where mobile terminal 14 is in silent mode (i.e., a state in which an alarm sound is not output when a notification is made to mobile terminal 14). In such cases, when the level of urgency is high or is medium or higher, notification control unit 64 may control mobile terminal 14 main body to output an alarm sound regardless of whether silent mode is set on the mobile terminal 14 side.
[0083] The notification control unit 64 also controls the notification information to be notified to the mobile terminal 14 carried by the driver by changing the notification end condition according to the urgency. For example, the notification control unit 64 controls the notification of notification information whose urgency is less than a certain level, a preset number of times such as once or twice. On the other hand, the notification control unit 64 controls the notification information whose urgency is equal to or higher than a certain level, to be repeatedly notified until the driver performs a vehicle operation such as gripping the steering wheel with his / her hands. Furthermore, the notification control unit 64 controls the notification information whose urgency is equal to or higher than a certain level, to be repeatedly notified until the driver presses a confirmation button displayed on the touch panel 44 of the mobile terminal 14. That is, the notification end condition according to this embodiment includes the number of notifications, a condition that the driver performs a vehicle operation, and a condition that the driver presses a confirmation button displayed on the touch panel 44 of the mobile terminal 14.
[0084] The notification control unit 64 generates notification control information including the range in which the driver can control the mobile terminal 14, the output method of the notification information such as display, sound, and vibration on the mobile terminal 14, and an end condition. Then, the notification control unit 64 controls the notification of the notification information by transmitting the notification information and the notification control information to the mobile terminal 14 via the wireless communication I / F 23.
[0085] Next, a functional configuration of the mobile terminal 14 according to this embodiment will be described with reference to Fig. 16. As shown in Fig. 16, the mobile terminal 14 includes a receiving unit 70 and an output unit 72. The CPU 40 executes a program stored in the ROM 41, thereby functioning as the receiving unit 70 and the output unit 72.
[0086] The receiving unit 70 receives the notification information and the notification control information transmitted from the vehicle control device 12 via the wireless communication I / F 43.
[0087] The output unit 72 notifies the driver of the notification information by outputting the notification information in accordance with the notification control information received by the receiving unit 70. That is, the notification is provided to the driver until the end condition included in the notification control information is satisfied.
[0088] Next, the operation of the vehicle control device 12 according to this embodiment will be described with reference to Fig. 17. The notification control process shown in Fig. 17 is executed by the CPU 20 executing a program stored in the ROM 21. The notification control process shown in Fig. 17 is executed, for example, when notification information is input to the vehicle control device 12 from another ECU via the in-vehicle communication I / F 24 while autonomous driving is being performed.
[0089] 17, the acquisition unit 60 acquires notification information via the in-vehicle communication I / F 24. In step S12, the derivation unit 62 derives the degree of urgency of the time until the driver takes action while autonomous driving is being performed, based on the notification information acquired in step S10, as described above. In addition, the derivation unit 62 derives the risk of the event, based on the notification information acquired in step S10, as described above.
[0090] In step S14, the derivation unit 62 refers to the urgency data 32 and derives an urgency corresponding to the combination of the urgency of the time and the risk of the event derived in step S12. In step S16, as described above, the notification control unit 64 controls the notification of the notification information by transmitting the notification information and the notification control information to the mobile terminal 14 via the wireless communication I / F 23 according to the urgency derived in step S14. As a result, the notification control unit 64 controls the notification of the notification information to the mobile terminal 14 by changing the range that the driver can control until a predetermined condition is satisfied according to the urgency derived by the derivation unit 62. In addition, the notification control unit 64 controls the notification of the notification information to the mobile terminal 14 carried by the driver by changing the notification end condition according to the urgency. When the process of step S16 ends, the notification control process ends. For example, when notification information indicating a "high" urgency is notified in step S16, the autonomous driving ECU 107 or the like performs a Minimal Risk Maneuver (MRM), and the state of the vehicle 10 transitions to a Minimal Risk Condition (MRC).
[0091] Next, the operation of the mobile terminal 14 according to this embodiment will be described with reference to Fig. 18. The notification process shown in Fig. 18 is executed by the CPU 40 executing a program stored in the ROM 41. The notification process shown in Fig. 18 is executed, for example, when notification information and notification control information are transmitted from the vehicle control device 12 to the mobile terminal 14 in the above step S16.
[0092] In step S20 of Fig. 18, the receiving unit 70 receives the notification information and notification control information transmitted from the vehicle control device 12 via the wireless communication I / F 43. In step S22, the output unit 72 outputs the notification information according to the notification control information received in step S20, as described above. In step S24, the output unit 72 determines whether or not the end condition included in the notification control information received in step S20 is satisfied. If this determination is negative, the process returns to step S22, and if this determination is positive, the notification process ends.
[0093] As described above, according to the present embodiment, it is possible to prevent the driver of the vehicle 10 from becoming less aware of important notifications regarding the vehicle 10.
[0094] Second embodiment Next, a second embodiment of the notification system 1 according to the present invention will be described while omitting or simplifying parts that overlap with the first embodiment.
[0095] FIG. 19 is a diagram showing an example of the directionality of notification information exchanged in notification system 1. As shown in FIG. An arrow A shown in Fig. 19 indicates notification information from the vehicle control device 12 to the driver of the vehicle 10. Here, the notification information from the vehicle control device 12 to the driver is the notification information described in the first embodiment. In the second embodiment, the notification information from the vehicle control device 12 to the driver is described as "first notification information."
[0096] An arrow B shown in FIG. 19 indicates notification information directed from the driver to the vehicle control device 12. Here, in the second embodiment, the notification information directed from the driver to the vehicle control device 12 is described as "second notification information." For example, the second notification information includes biometric information of the driver acquired via the mobile terminal 14 and the in-vehicle sensor 101, etc. The biometric information is information about the driver, such as the driver's body temperature, blood pressure, pulse, skin electrical resistance, respiratory rate, and behavior. The second notification information also includes operation information indicating the operation contents of the driver on the mobile terminal 14 and the input device 106, etc. Furthermore, the second notification information includes driver information indicating the driver's state, such as whether or not the driving authority can be delegated to the driver. For example, the vehicle control device 12 acquires the driver's utterance contents collected by the microphone included in the in-vehicle sensor 101 as the second notification information, or acquires the above operation information and driver information transmitted by the mobile terminal 14 as the second notification information. As a result, the vehicle control device 12 can grasp the driver's state based on the acquired second notification information. Then, the vehicle control device 12 changes the urgency by setting at least one of the time urgency and the event danger based on the driver's state that has been grasped. Examples of changes in the driver's state and urgency that are grasped by the vehicle control device 12 will be described later.
[0097] Fig. 20 is a diagram showing a first example of stratification of urgency. Here, the urgency in the second embodiment represents the urgency of the response by the driver or the vehicle control device 12 while autonomous driving is being performed. As in the first embodiment, the urgency increases as the time urgency increases and as the risk of the event increases. Fig. 20 shows the stratification of the urgency of the response by the driver while autonomous driving is being performed.
[0098] The urgency of the area A-1 shown in FIG. 20 is classified as high. In the area A-1, both the urgency of time and the risk of the event are high. In the area A-1, the first notification information notified to the driver is a notification regarding MRM, such as executing MRM or running MRM. The first notification information is notified to the driver when the driving environment conditions, which are the premise for the vehicle control device 12 to control the automatic driving, suddenly do not meet, or a malfunction occurs in a critical system, such as the actuator system 108 for controlling the automatic driving. Although the urgency of the driver's response in the area A-1 is high, the vehicle control device 12 does not request the driver's response because there is not enough time for the driver to respond to the driving change request to transfer the main driving responsibility from the vehicle control device 12 to the driver. In this case, the vehicle control device 12, for example, executes MRM to try to park the vehicle 10 on the roadside, and generates a notification regarding MRM indicating that MRM is being executed as the first notification information and transmits it to the mobile terminal 14. This allows the driver to understand that MRM is being executed.
[0099] The urgency level of the area A-2 shown in FIG. 20 is classified as medium 1. In the area A-2, the first notification information notified to the driver is a driving change request or a predetermined action request. The first notification information is notified to the driver when the vehicle 10, which no longer satisfies the driving environment condition, is capable of manual driving. The above-mentioned "driving environment condition is no longer satisfied" refers to, for example, when the vehicle 10 enters outside the operation design area, and when a failure of a predetermined in-vehicle device is detected. In addition, the above-mentioned "when manual driving of the vehicle 10 is possible" refers to a case where there is no abnormality in the in-vehicle device required for manual driving, regardless of whether the driver can respond to the driving change request. In this case, the vehicle control device 12 generates, for example, a driving change request as the first notification information and transmits it to the mobile terminal 14. As a result, the driver who can respond to the driving change request can understand that the vehicle 10 needs to be switched to manual driving.
[0100] As described above, "the driving environment conditions are no longer satisfied" includes a case where a failure of a specific in-vehicle device is detected. Here, the failure of the specific in-vehicle device includes a failure of an in-vehicle device that disables a specific system function of the vehicle 10. For example, if the in-vehicle device in which a failure is detected is a driver monitoring camera included in the in-vehicle sensor 101, the vehicle control device 12 may not be able to determine whether the driver is able to respond to the driving change request transmitted to the mobile terminal 14.
[0101] In the above-mentioned area A-2, the vehicle control device 12 may generate a predetermined action request as the first notification information instead of the driving changeover request and transmit it to the mobile terminal 14. The predetermined action request is, for example, a request for the driver to perform a predetermined action such as gripping the steering wheel with hands and lifting the head.
[0102] The classification of the urgency of the area A-3 shown in FIG. 20 is medium 3. Here, medium 3 is a lower urgency than medium 1. In addition, the time urgency of the area A-3 is low. In the area A-3, the first notification information notified to the driver is a prior notification regarding the end of the automatic driving function. The automatic driving function is a function of the vehicle 10 for performing automatic driving of automatic driving level 3 or higher with the vehicle control device 12 as the driving subject. The first notification information is notified to the driver when the end of the automatic driving function after a certain time (for example, several minutes) is detected. The situation in which the first notification information is notified to the driver is almost always when the vehicle 10 approaches outside a preset operation design area. In this situation, there is enough time before the driving subject is transferred to the driver. Therefore, the vehicle control device 12 generates a prior notification indicating that the automatic driving function will end after a certain time as the first notification information and transmits it to the mobile terminal 14. This allows the driver to understand that the automatic driving function will end after a certain time.
[0103] The urgency level of the area A-4 shown in FIG. 20 is classified as medium 1. In the area A-4, the first notification information notified to the driver is a request for approval to the driver. The first notification information is notified to the driver at a predetermined notification timing for the automatic driving function. In this case, the vehicle control device 12 requests the driver to approve the execution of the automatic driving function before starting to execute the predetermined automatic driving function. For example, the vehicle control device 12 generates an approval request for the execution of the predetermined automatic driving function as the first notification information and transmits it to the mobile terminal 14. This allows the driver to know the automatic driving function to be executed, and if necessary, to approve the execution of the automatic driving function.
[0104] The above-mentioned predetermined automatic driving function may be a part of the automatic driving function input in advance by the driver according to at least one of the driver's preferences and characteristics. In addition, the vehicle control device 12 may execute the predetermined automatic driving function in the situation of the area A-4 without obtaining the driver's approval, generate a notification indicating that the automatic driving function is being executed as the first notification information, and transmit the notification to the mobile terminal 14.
[0105] The urgency classification of the area A-5 shown in FIG. 20 is low. In the area A-5, both the time urgency and the risk of the event are low. In the area A-5, the first notification information notified to the driver is useful information that is not urgent but useful to the driver, such as the remaining fuel level or charge level, and information regarding a relatively distant routing decision. The first notification information is notified to the driver by acquiring useful information. In this case, the vehicle control device 12 generates useful information indicating, for example, the remaining fuel level as the first notification information and transmits it to the mobile terminal 14. This allows the driver to know the remaining fuel level.
[0106] FIG. 21 shows first notification information corresponding to each of the areas A-1 to A-5 in FIG. 20, examples of events for which the first notification information is notified, and an example of an action required of the user, specifically, the driver, in response to the notified first notification information.
[0107] As shown in FIG. 21, an example of an event for which the first notification information corresponding to the area A-1 is notified is, for example, "the brake system or the steering system is disabled." For example, the vehicle control device 12 detects that the brake system 111 included in the actuator system 108 connected to the autonomous driving ECU 107 has been disabled. In this case, the vehicle control device 12 transmits a notification regarding the MRM to the mobile terminal 14 as the first notification information. The action required of the driver in response to the first notification information is "none." In other words, the driver does not need to take any action when the first notification information is notified.
[0108] As shown in FIG. 21, an example of an event for which the first notification information corresponding to the area A-2 is notified is, for example, "detection of approach of emergency vehicle." For example, the vehicle control device 12 detects that an emergency vehicle is approaching the vehicle 10 based on information from the external sensor 100. In this case, the vehicle control device 12 transmits a driving change request or a predetermined action request to the mobile terminal 14 as the first notification information. The driver's action required in response to the first notification information may be, for example, switching from automatic driving to manual driving by performing a predetermined operation on the input device 106, and resuming the manual driving mode. In this case, the driver may perform degenerate driving such as stopping the vehicle 10 on the road shoulder in the resumed manual driving mode in order to reach the MRC.
[0109] As shown in FIG. 21, an example of an event for which the first notification information corresponding to the area A-3 is notified is, for example, "the driving environment conditions will not be satisfied in the near future," such as detection of an obstacle on a road 10 km away or a drastic change in weather within 10 minutes. For example, the vehicle control device 12 detects that the weather will change drastically within 10 minutes based on information from the cloud server 16. In this case, the vehicle control device 12 transmits a prior notification regarding the end of the autonomous driving function to the mobile terminal 14 as the first notification information. Then, the action of the driver required in response to the first notification information is, for example, preparing to resume the manual driving mode by gripping the steering wheel with hands.
[0110] As shown in FIG. 21, an example of an event for which the first notification information corresponding to the area A-4 is notified is, for example, a "proposal from the vehicle control device 12" such as overtaking or passing. For example, the vehicle control device 12 proposes to the driver overtaking the preceding vehicle as a predetermined automatic driving function. In this case, before starting to execute overtaking the preceding vehicle, the vehicle control device 12 transmits a request for the driver's approval for the execution of the automatic driving function related to overtaking the preceding vehicle to the mobile terminal 14 as first notification information. Then, the action of the driver required in response to the first notification information is, for example, a response to the proposal from the vehicle control device 12, such as performing a predetermined operation on the mobile terminal 14.
[0111] As shown in FIG. 21, an example of an event for which the first notification information corresponding to the area A-5 is notified is, for example, "detection of a branch 10 km ahead." For example, the vehicle control device 12 detects that there is a branch 10 km ahead based on surrounding information from the navigation device, the map data 102, etc. In this case, the vehicle control device 12 transmits useful information to the mobile terminal 14 as the first notification information. The action required of the driver in response to the first notification information is "none." In other words, the driver does not need to take any action when the first notification information is notified.
[0112] Fig. 22 is a diagram showing a second example of the stratification of the urgency levels. Fig. 22 shows the stratification of the urgency levels of the response by the vehicle control device 12 while autonomous driving is being performed.
[0113] The urgency level of the area B-1 shown in FIG. 22 is classified as high 1. The second notification information in the area B-1 notifies the vehicle control device 12 that the driving authority cannot be delegated to the driver. In other words, the second notification information notifies the vehicle control device 12 that the driver cannot perform the dynamic driving task of "cognition, judgment, and operation" that the driver should perform when driving. The second notification information is notified to the vehicle control device 12 by a driver who has an acute illness or consciousness disorder or the like performing a predetermined operation on the mobile terminal 14 and the input device 106 or the like. Here, the time urgency and the event risk in the area B-1 change depending on the cause of the inability to delegate the driving authority to the driver, specifically, the state of the driver. For example, when the vehicle control device 12 determines that the driver's state is an acute illness such as a heart attack or a stroke, it sets both the time urgency and the event risk to high. Furthermore, when the vehicle control device 12 determines that the driver's condition is drowsiness, fatigue, a digestive attack, etc., it sets the risk level of the event to high. In the above case, the vehicle control device 12 sets the urgency level higher when the driver's condition is an acute illness than when the driver's condition is drowsiness, fatigue, a digestive attack, etc. The method by which the vehicle control device 12 determines the driver's condition will be described later.
[0114] The classification of the urgency of the area B-2 shown in FIG. 22 is high 2. Here, high 2 is a lower urgency than high 1. The second notification information in the area B-2 notifies the vehicle control device 12 that the driving authority cannot be delegated to the driver. The second notification information is notified to the vehicle control device 12 by the mobile terminal 14 estimating that the driving authority cannot be delegated to the driver. For example, the mobile terminal 14 estimates whether the driving authority cannot be delegated to the driver based on the driver's biometric information and operation information. After estimating that the driving authority cannot be delegated to the driver, the mobile terminal 14 periodically estimates whether the driving authority can be delegated to the driver based on the driver's biometric information and operation information. Then, when the mobile terminal 14 estimates that the driving authority can be delegated to the driver, it notifies the vehicle control device 12 of the second notification information indicating that the driving authority can be delegated to the driver.
[0115] The urgency level of the area B-3 shown in FIG. 22 is classified as medium. In the area B-3, the second notification information notified to the vehicle control device 12 is the driver's approval or disapproval of the approval request from the vehicle control device 12. The second notification information is notified to the vehicle control device 12 by the driver performing a predetermined response operation to the approval request on the mobile terminal 14, the input device 106, etc. For example, the mobile terminal 14 notifies the vehicle control device 12 of operation information indicating the driver's approval or disapproval of the approval request as the second notification information. Note that, without being limited to this, the vehicle control device 12 may acquire the driver's speech indicating approval or disapproval of the approval request, which is collected by a microphone included in the vehicle interior sensor 101, as the second notification information.
[0116] FIG. 23 shows second notification information corresponding to each of the areas B-1 to B-3 in FIG. 22, examples of events for which the second notification information is notified, and examples of actions of a user, specifically, a driver, when notifying the second notification information.
[0117] As shown in FIG. 23, an example of an event for which the second notification information corresponding to the area B-1 is notified is, for example, the driver suffering from a "severe physical illness" such as a stroke or a heart attack. For example, when the driver experiences symptoms of a stroke such as paralysis and numbness in the limbs, the driver performs a predetermined operation on an emergency switch (not shown) included in the input device 106. When the predetermined operation is performed, the emergency switch is a switch that notifies the vehicle control device 12 as second notification information that the driving authority cannot be delegated to the driver. This allows the vehicle control device 12 to understand that the driving authority cannot be delegated to the driver.
[0118] Here, when the vehicle control device 12 acquires the second notification information from the emergency switch, the vehicle control device 12 grasps the driver's state based on the following information. First, the vehicle control device 12 acquires the driver's biometric information via the mobile terminal 14 and the in-vehicle sensor 101, etc., and acquires operation information indicating the operation content of the driver via the mobile terminal 14 and the input device 106, etc. Next, the vehicle control device 12 inputs the acquired driver's biometric information and operation information into a trained model that has been trained to output the driver's state when the driver's biometric information and operation information are input. Then, the vehicle control device 12 acquires the driver's state output from the trained model and grasps the driver's state. The trained model is stored in, for example, the ROM 21.
[0119] The trained model is generated using a known machine learning algorithm such as a neural network. The training data to be input to the machine learning algorithm can be created by manually linking combinations of biometric information and operation information corresponding to various driver conditions such as stroke or heart attack.
[0120] As shown in FIG. 23, an example of an event in which the second notification information corresponding to the area B-2 is notified is, for example, "the mobile terminal 14 and the in-vehicle sensor 101, etc. detect predetermined biological information of the driver, such as body temperature, blood pressure, pulse, skin electrical resistance, respiratory rate, and behavior." For example, the mobile terminal 14 detects that the driver's pulse rate per minute is 100 or more or less than 50. In this case, the mobile terminal 14 notifies the vehicle control device 12 of the second notification information indicating that the driving authority cannot be delegated to the driver. Then, the action of the driver when notifying the second notification information is "none." In other words, the driver does not need to take any action to notify the vehicle control device 12 of the second notification information.
[0121] As shown in FIG. 23, an example of an event for which the second notification information corresponding to the area B-3 is notified is, for example, a "response to a proposal from the vehicle control device 12," such as overtaking or passing. For example, the mobile terminal 14 is assumed to have acquired the first notification information indicating an approval request for the execution of an automatic driving function related to overtaking a preceding vehicle. In this case, the driver performs a predetermined response operation indicating approval or disapproval of the approval request. Then, the mobile terminal 14 notifies the vehicle control device 12 of the operation information indicating the approval or disapproval by the driver as the second notification information. This allows the vehicle control device 12 to grasp the content of the driving response to the approval request.
[0122] In the above embodiment, when the notification information is information with a certain level of urgency or higher, the notification control unit 64 may perform control to notify the notification information such that a notification sound dedicated to notifications from the vehicle 10 is output from the mobile terminal 14. In this case, the driver can easily recognize that the notification is an important notification from the vehicle 10.
[0123] In the above embodiment, when the notification information is information with a certain level of urgency or higher, the notification control unit 64 may perform control to call out the driver's name by voice before notifying the notification information. In this case, the driver's name can be obtained by acquiring the registered name of the mobile terminal 14 or by having the driver input it in advance.
[0124] In the above embodiment, the notification control unit 64 may perform notification on the mobile terminal 14 according to the priority order between the first notification information generated on the mobile terminal 14 and the second notification information that is notification information related to the vehicle 10 (reconciliation of notification information). For example, the notification control unit 64 may prioritize notification information with a certain level of urgency or higher, a call to the mobile terminal 14, a notification related to a natural disaster such as an earthquake alert, a notification of a warning to a specific area, a notification with a level of urgency lower than a certain level, and a normal notification other than the above notifications to the mobile terminal 14. More specifically, notification information with a medium level of urgency or higher or a high level of urgency is related to the safety of the occupants, and therefore may be prioritized higher than a call to the mobile terminal 14, a notification related to a natural disaster such as an earthquake alert, and a notification of a warning to a specific area.
[0125] In the above embodiment, when the notification information is information with a degree of urgency less than a certain level, the notification control unit 64 may perform control to notify the notification information by changing the notification mode according to the state of the driver and the operation state of the mobile terminal 14 by the driver. In this case, when the driver is looking at the touch panel 44 of the mobile terminal 14, such as when the driver is playing a game using the mobile terminal 14, the notification control unit 64 may perform control to notify the notification information by performing control to display the notification information on the touch panel 44 of the mobile terminal 14. When the driver is listening to a sound output from the mobile terminal 14, such as when the driver is playing music using the mobile terminal 14, the notification control unit 64 may perform control to notify the notification information by performing control to output the notification information as sound from the speaker of the mobile terminal 14. When the driver is not using the mobile terminal 14, such as when the driver is sleeping, the notification control unit 64 may perform control to notify the notification information by vibrating the mobile terminal 14 or vibrating the seat belt of the driver's seat. The notification control unit 64 can determine the driver's condition and the driver's operation status of the mobile terminal 14 by acquiring information from the mobile terminal 14, analyzing images captured by a camera that captures the driver, such as a driver monitoring camera, and by a device worn by the driver that acquires the driver's biometric information, such as a smart watch.
[0126] In the above embodiment, the notification control unit 64 may notify the notification information to a notification device mounted on the vehicle 10 at the same timing as the notification to the mobile terminal 14. Examples of the notification device in this case include the display device 15, a speaker mounted on the vehicle 10, and a seat belt of the driver's seat. That is, the notification control unit 64 may notify the notification information by displaying on the display device 15, outputting sound from a speaker mounted on the vehicle, vibrating the seat belt of the driver's seat, and the like, at the same timing as the notification to the mobile terminal 14. In this case, the notification control unit 64 may end the notification from the notification device mounted on the vehicle 10 when a notification end condition is satisfied.
[0127] In the above embodiment, when the driver has a plurality of mobile terminals 14, the notification control unit 64 may perform control to notify two or more of the plurality of mobile terminals 14 of the notification information.
[0128] In the above embodiment, the mobile terminal 14 may include at least a part of the functional units included in the vehicle control device 12.
[0129] In addition, the processing performed by the CPU by reading the software (program) in the above embodiment may be performed by various processors other than the CPU. Examples of the processor in this case include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for performing a specific process. Each process may be performed by one of these various processors, or may be performed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, and a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.
[0130] Furthermore, in the above embodiment, various data are stored in the ROM or storage, but this is not limiting. For example, a recording medium such as a CD (Compact Disk), a DVD (Digital Versatile Disk), or a USB (Universal Serial Bus) memory may be used as the storage unit. In this case, various programs and data are stored in these recording media.
[0131] The notification system 1 according to the embodiment has been described above, but it goes without saying that the present invention can be embodied in various forms without departing from the gist of the present invention. [Explanation of symbols]
[0132] 1. Notification System 10 Vehicles 12 Vehicle control device 14 Mobile devices 30 Control Program 50 Output Program 60 Acquisition Department 62 Derivation part 64 Notification control section 70 Receiving section 72 Output section
Claims
1. A notification system including a mobile terminal capable of receiving and notifying notification information related to a vehicle, and a vehicle control device that generates and transmits the notification information to a driver of the vehicle, A notification control unit that controls the mobile terminal to notify the notification information by varying a controllable range of the driver until a predetermined condition is satisfied according to an urgency of the notification information. Includes a notification system.
2. The urgency is determined based on whether the notification information is information that requires a response from the driver. The notification system of claim 1 .
3. The urgency is determined according to the danger of the event. A notification system according to claim 1 or 2.
4. The notification control unit controls the mobile terminal so that the driver cannot operate the mobile terminal until the driver operates the vehicle, with respect to notification information whose urgency is equal to or higher than a certain level, and controls the notification information to be notified. A notification system according to claim 1 or 2.
5. The notification control unit controls the mobile terminal to display a confirmation button on a display device of the mobile terminal for notification information having a certain degree of urgency or higher, and controls the mobile terminal to be in a state in which operations other than pressing the confirmation button are not possible until the driver presses the confirmation button, thereby notifying the notification information. A notification system according to claim 1 or 2.
6. The notification control unit performs notification on the mobile terminal according to a priority order between first notification information generated on the mobile terminal and second notification information related to the vehicle. A notification system according to claim 1 or 2.
7. An acquisition unit that acquires notification information to be transmitted to a mobile terminal held by a driver of a vehicle, the notification information being to be notified to the driver; a notification control unit that controls the mobile terminal to notify the notification information by changing a controllable range of the driver according to an urgency of the notification information until a predetermined condition is satisfied; A vehicle control device comprising:
8. A notification method executed by a computer constituting a notification system including a mobile terminal capable of receiving and notifying notification information related to a vehicle and a vehicle control device that generates and transmits the notification information to a driver of the vehicle, According to the urgency of the notification information, the range in which the driver can control the mobile terminal is changed and the notification information is notified until a predetermined condition is satisfied. A notification method for computer-implemented processing.
9. A program to be executed by a computer constituting a notification system including a mobile terminal capable of receiving and notifying notification information related to a vehicle and a vehicle control device that generates and transmits the notification information to a driver of the vehicle, According to the urgency of the notification information, the range in which the driver can control the mobile terminal is changed and the notification information is notified until a predetermined condition is satisfied. A program that causes a computer to execute a process.
Citation Information
Patent Citations
Alarm apparatus, alarm system, and portable terminal
JP2016090274A
Information transmission device and program
JP2017102664A
Support system, portable terminal, and on-vehicle device
JP2017200812A
Automatic drive information transmission method
JP2019172180A
Drive assist method, drive assist program, and vehicle control device
WO2018230527A1
