Information presentation control device and function control device

Abstract

The HCU (10), serving as an information prompting control device, is used in a vehicle equipped with an autonomous driving function capable of performing driving actions in place of a driver, and controls an information prompting device (20) that prompts information to the driver. The HCU includes: a permitted behavior discrimination unit (X4) that discriminates permitted behaviors that the driver is allowed to perform when the autonomous driving function is in operation, among behaviors other than driving that are different from driving behaviors; and an information prompting content control unit (X8) that causes the information prompting device to display information related to the discrimination result of the permitted behavior.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to Japanese Patent Application No. 2020-51518, filed on March 23, 2020, and Japanese Patent Application No. 2021-30063, filed on February 26, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The disclosure of this specification relates to support for the implementation of behaviors that differ from driver-driven operations in vehicles equipped with autonomous driving capabilities.

[0004] Background

[0005] Patent Document 1 discloses a method for displaying an application image on an information prompting device when an autonomous driving function is determined to be in operation in a vehicle equipped with an autonomous driving function that can replace a driver.

[0006] Patent Document 1: Japanese Patent Application Publication No. 2017-222271

[0007] When the autonomous driving function of the vehicle described in Patent Document 1 is in operation, the driver can view and operate the aforementioned application images as an action other than driving.

[0008] On the other hand, when autonomous driving functions are in operation, there may be behaviors that should be permitted and behaviors that should not be permitted, which are outside of driving. Furthermore, the permissible range of such behaviors may vary depending on various conditions, such as the vehicle's driving environment and the driver's state. However, accurately identifying these various conditions and correctly determining the extent to which non-driving behaviors are permitted can be difficult for the driver. Summary of the Invention

[0009] One of the purposes of this specification is to provide information prompts and function control devices that enable drivers to take appropriate actions when the automatic driving function is in operation.

[0010] One of the methods disclosed here is for vehicles equipped with autonomous driving functions that can replace the driver in performing driving actions, and for information prompting control devices that control information prompting information to the driver, comprising:

[0011] The permitted behavior identification unit identifies permitted behaviors that the driver is allowed to perform, among those behaviors other than driving that can be performed by the driver while the autonomous driving function is operating; and

[0012] The information prompt content control unit enables the information prompt device to display information related to the identification result of the permitted behavior.

[0013] In this way, information related to the determination result of the permitted behavior identified by the permitted behavior recognition unit is displayed via an information display device. When the autonomous driving function is in operation, where the driver's need to be aware of the vehicle's driving environment and other conditions is reduced, the driver can obtain information related to the determination result of the permitted behavior via the information display device. Therefore, by referring to this information, the driver can easily take appropriate action when the autonomous driving function is operating.

[0014] Additionally, one of the methods disclosed here is a functional control device for vehicles equipped with autonomous driving functions capable of performing driving actions in place of a driver, and for controlling the controlled device, comprising:

[0015] The permitted behavior identification unit identifies permitted behaviors that the driver is allowed to perform, among those behaviors other than driving that can be performed by the driver while the autonomous driving function is operating; and

[0016] The function restriction section restricts the functions of the control object device that enables actions other than driving.

[0017] Based on this method, the functions of the control device that can perform actions other than those permitted by the permitted actions identified by the permitted actions discrimination unit are restricted. When operating autonomous driving functions, where the necessity for the driver to understand the vehicle's driving environment and other conditions is reduced, functional restrictions can prevent the driver from unintentionally performing actions other than permitted actions. Therefore, it is easier for the driver to take appropriate actions when the autonomous driving function is operating.

[0018] Furthermore, the reference numerals in parentheses in the claims illustratively show the correspondence between parts of the embodiments described later, and do not limit the scope of the technology. Attached Figure Description

[0019] Figure 1 It is a diagram representing the overall structure of the vehicle network.

[0020] Figure 2 This is a diagram showing the general structure of the HCU.

[0021] Figure 3 This is a flowchart representing the HCU's processing.

[0022] Figure 4 This is a timing diagram illustrating an example of the operation of an autonomous driving ECU and HCU.

[0023] Figure 5 This is a timing diagram illustrating another example of the operation of the autonomous driving ECU and HCU.

[0024] Figure 6 This is a flowchart illustrating the processing of the HCU in the second embodiment.

[0025] Figure 7 This is a diagram showing the general configuration of the autonomous driving ECU in the third embodiment.

[0026] Figure 8 This is a flowchart illustrating the processing of the autonomous driving ECU in the third embodiment.

[0027] Figure 9 This is a timing diagram illustrating an example of the operation of the autonomous driving ECU and HCU in the fourth embodiment.

[0028] Figure 10 This is a timing diagram illustrating another example of the operation of the autonomous driving ECU and HCU in the fourth embodiment.

[0029] Figure 11 This is a diagram showing the overall structure of the vehicle network according to the fifth embodiment.

[0030] Figure 12 This is a timing diagram illustrating an example of the operation of an autonomous driving ECU and HCU.

[0031] Figure 13 Is with Figure 14 A flowchart of the information prompting process of the HCU in the fifth embodiment is shown together.

[0032] Figure 14 Is with Figure 13 A flowchart of the information prompting process of the HCU in the fifth embodiment is shown together. Detailed Implementation

[0033] Hereinafter, several embodiments will be described based on the accompanying drawings. Additionally, there are cases where repeated descriptions are omitted by using the same reference numerals for corresponding components in each embodiment. In cases where only a portion of the configuration is described in each embodiment, the other portions of the configuration can be constructed using the configurations of other previously described embodiments. Furthermore, not only combinations of configurations explicitly shown in the descriptions of each embodiment, but also configurations of multiple embodiments can be partially combined with each other, even if not explicitly shown, as long as the combination does not particularly cause obstacles.

[0034] (First Implementation)

[0035] like Figure 1As shown, the information prompting control device in the first embodiment of this disclosure is an HCU (Human Machine Interface Control Unit) 10. The HCU 10, together with the information prompting device 20, constitutes an information prompting system IPS. The HCU 10 is an electronic control device that controls the information prompting device 20, and comprehensively implements information prompting functions as well as behavior implementation support functions such as supporting the implementation of second tasks and other non-driving behaviors. The HCU 10 is used in a vehicle 1, for example, mounted on a vehicle 1.

[0036] HCU10 is connected to the communication bus 99 of the vehicle network installed in vehicle 1 in a communicative manner. HCU10 is one of multiple nodes installed in the vehicle network installed in vehicle 1. The communication bus 99 of the vehicle network is connected to nodes such as DSM31, seat sensor 32, perimeter monitoring sensor 33, locator 34, external communication unit 35, driving control ECU 36, driving support ECU 37, and autonomous driving ECU 40.

[0037] The DSM (Driver Status Monitor) 31 is a status detection device that detects the driver's status. The DSM 31 is configured to include a near-infrared light source, a near-infrared camera, and a control unit that controls them. The DSM 31 positions the near-infrared camera towards the headrest of the driver's seat, for example, on the upper surface of the steering column or the upper surface of the instrument panel. The DSM 31 captures images of the driver's head illuminated by the near-infrared light source using the near-infrared camera. The control unit performs image analysis on the images captured by the near-infrared camera. The control unit extracts status information such as the driver's eye position, gaze direction, pupil blur, and driver posture from the captured images, and provides this extracted driver status information to the HCU 10 via the communication bus 99. Alternatively, the control unit can provide the captured images themselves to the HCU 10 via the communication bus 99.

[0038] The seat sensor 32 is a sensor that detects the driver's seat position. The seat sensor 32 has functions such as detecting whether the driver is seated in the driver's seat, detecting whether the driver is wearing a seatbelt, and detecting the tilt angle of the driver's seat. The seat sensor 32 provides the seat position information to the HCU 10, etc., via the communication bus 99.

[0039] The surrounding monitoring sensor 33 is an autonomous sensor that monitors the surrounding environment of vehicle 1. The surrounding monitoring sensor 33 can detect moving objects such as pedestrians, bicycles, animals other than humans, and other vehicles, as well as road markings such as fallen objects, guardrails, curbs, road signs, lane markings, and roadside structures within its detection range around the vehicle. The surrounding monitoring sensor 33 provides the detection information of objects around the vehicle to the driver support ECU 37, the autonomous driving ECU 40, and the HCU 10 via the communication bus 99.

[0040] The peripheral monitoring sensor 33 may include, for example, a camera unit and a millimeter-wave radar. The camera unit may be configured to include either a monocular camera or a compound-eye camera. The camera unit is mounted on the vehicle 1 to capture images of the area in front of, to the sides of, and behind the vehicle 1. The camera unit outputs at least one of the captured data and the analysis results of the captured data as detection information. The millimeter-wave radar emits millimeter-wave or quasi-millimeter-wave radiation around the vehicle. The millimeter-wave radar outputs detection information generated by processing reflected waves received from moving and stationary objects. The peripheral monitoring sensor 33 may also include lidar and sonar, either as an addition to or alternate between the camera unit and the millimeter-wave radar.

[0041] The locator 34 is configured to include a GNSS (Global Navigation Satellite Systems) receiver and an inertial sensor. The locator 34 combines the positioning signal received by the GNSS receiver, the measurement results from the inertial sensor, and the vehicle speed information output to the communication bus 99 to sequentially measure the vehicle 1's position and direction of travel. The locator 34 outputs the vehicle 1's position and orientation information based on the positioning results as locator information to the communication bus 99.

[0042] The locator 34 also includes a map database 34a. The map database 34a is primarily composed of a large-capacity, non-volatile storage medium that stores numerous 3D and 2D map data. The 3D map data is so-called high-precision map data, containing information necessary for advanced driver support and autonomous driving, such as the 3D shape information of roads and detailed information about each lane. The locator 34 reads map data of the surrounding area from the map database 34a and provides it, along with locator information, to the driver support ECU 37 and autonomous driving ECU 40. Alternatively, location information, orientation information, and map data can be provided to the driver support ECU 37 and autonomous driving ECU 40 by a user terminal such as a smartphone or a navigation device, instead of the locator 34.

[0043] The external communication unit 35 is a communication module (Data Communication Module) installed in vehicle 1. The external communication unit 35 transmits and receives radio waves between vehicle 1 and surrounding base stations via wireless communication according to communication standards such as LTE (Long Term Evolution) and 5G. Through the installation of the external communication unit 35, vehicle 1 becomes a connected car connected to the Internet. The external communication unit 35 obtains various data from a detection server located in the cloud. Examples of this data include the latest map data of the road on which vehicle 1 travels, traffic information data, weather information data, and playback data of content such as movies played by the information display device 20. Traffic information data includes, for example, road information such as traffic volume, traffic accidents, and traffic restrictions due to construction.

[0044] The driving control ECU 36 is an electronic control device that includes a microcomputer as its main component. Based on the detection signals from wheel speed sensors located at the wheel hubs of each wheel, the driving control ECU 36 generates vehicle speed information indicating the current driving speed of vehicle 1 and outputs it sequentially to the communication bus 99. In addition, the driving control ECU 36 controls the operation of the driving actuators.

[0045] The driving actuator is configured to include a vehicle steering control device, a vehicle drive device, and a vehicle braking device for performing driving operations. Driving operations include vehicle steering, vehicle drive, and vehicle braking. The vehicle steering control device is a device that controls the steering angle applied to, for example, the front wheels of vehicle 1. The vehicle drive device is a device that uses power supplied from a power source of vehicle 1 to drive, for example, the front wheels of vehicle 1. The vehicle braking device is a device that uses braking methods such as friction braking and regenerative braking to brake, for example, the front wheels of vehicle 1.

[0046] The driving support ECU 37 and the autonomous driving ECU 40 together constitute the autonomous driving system ADS in vehicle 1. With the installation of the autonomous driving system ADS, vehicle 1 has autonomous driving capabilities.

[0047] The Driver Support ECU 37 is an electronic control device that provides driver support functions to assist the driver's driving operations. The Driver Support ECU 37 is capable of providing advanced driver support at approximately Level 2 of automated driving, or partial automated driving control. The Driver Support ECU 37 is primarily composed of a computer, including a processing unit, RAM (Random Access Memory), a storage unit, input / output interfaces, and a bus connecting them. The Driver Support ECU 37 executes computer programs through its processing unit and outputs control signals to the Driving Control ECU 36, and has multiple functional units for providing advanced driver support. Specifically, the Driver Support ECU 37 has an ACC (Adaptive Cruise Control) functional unit that implements ACC, an LTA (Lane Tracing Assist) functional unit that implements LTA, and an LCA (Lane Change Assist) functional unit, among others.

[0048] The autonomous driving ECU 40 is an electronic control device that enables autonomous driving functions that can act as a substitute for the driver's driving operations. The autonomous driving ECU 40 enables Level 3 autonomous driving of the vehicle 1 system, with the vehicle 1 acting as the main control entity, only within a pre-defined operational design domain (hereinafter referred to as the defined domain).

[0049] The defined area here includes the driving environment conditions that are prerequisites for the normal operation of autonomous driving based on the autonomous driving ECU 40. As an example, there are cases where the defined area is set based on the road traffic laws and regulations of the country or region where vehicle 1 travels, and cases where the defined area is set based on technical constraints such as the infrastructure condition and road shape of the road on which vehicle 1 travels. The above-mentioned division of the defined area based on road conditions and geographical conditions, for example, is linked to the map data mentioned above and included in the information of map database 34a.

[0050] The autonomous driving ECU 40 is configured as a computer as its main body, including a processing unit 41, RAM 42, storage unit 43, input / output interface 44, and buses connecting them. The processing unit 41 is hardware integrated with RAM 42 for computational processing. The processing unit 41 is configured to include at least one CPU (Central Processing Unit) and GPU (Graphics Processing Unit) or other computing cores. The processing unit 41 may also include RISC (Reduced Instruction Set Computer), FPGA (Field-Programmable Gate Array), and other dedicated IP cores. The processing unit 41 performs various processes to implement the functions of the functional units described later by accessing RAM 42. The storage unit 43 is configured, for example, to include at least one non-volatile storage medium such as semiconductor memory. Various computer programs (e.g., autonomous driving programs) executed by the processing unit 41 are stored in the storage unit 43.

[0051] The autonomous driving ECU 40, by executing a computer program stored in the storage unit 43 through the processing unit 41, has multiple functional units for realizing autonomous driving. Specifically, the autonomous driving ECU 40 is as follows: Figure 2 As shown, it has a status management unit Y1 and a driving control unit Y2.

[0052] The State Management Unit Y1 manages the state of autonomous driving based on locator information and map information obtained from locator 34, detection information obtained from surrounding monitoring sensors 33, traffic information and weather information obtained through the external communication unit 35, and retreat requests from HCU 10. Specifically, in the first embodiment, the State Management Unit Y1 manages the autonomous driving level, which is the state of autonomous driving, and switches it as needed. In the first embodiment, the autonomous driving level is taken as a discrete value, an integer between 0 and 3. As the autonomous driving level increases, it means that the scope of the system's control over driving behavior increases. Here, driving behavior is defined as the concept of applying surrounding monitoring to the aforementioned driving operations. The autonomous driving level switched by the State Management Unit Y1 is based on the autonomous driving levels specified by the Automotive Technology and Research Center (ATC).

[0053] For example, at Level 0 of the autonomous driving system, the driver becomes the implementer of all driving actions, performing what is known as manual driving. At Level 1, the implementer of any one of the driving actions—steering, driving, or braking—is Vehicle 1 (specifically, the driving support ECU 37), while the driver becomes the implementer of all other driving actions. At Level 2, the implementer of steering, driving, and braking is Vehicle 1 (specifically, the driving support ECU 37). In this case, the driver becomes the implementer of surrounding monitoring and must remain in a position where they can immediately perform driving operations, such as holding the steering wheel, to monitor the implementation of driving operations by Vehicle 1 and be able to intervene at any time.

[0054] At Level 3 of autonomous driving, vehicle 1 is the primary entity responsible for all driving actions, including steering, driving, braking, and monitoring of the surrounding environment. The autonomous driving ECU 40 on vehicle 1 performs these actions. In this scenario, the driver does not constantly monitor the autonomous driving ECU 40's actions and can engage in actions other than driving within safe limits. These safe limits can be defined by regulations or based on actual driving safety protocols.

[0055] Activities outside of driving that a driver is permitted to perform include watching content, playing computer games, reading emails and sending assignments, browsing websites, operating mobile phones or smartphones, eating, applying makeup, reading, and sleeping. Content includes, for example, videos such as movies, music, and audiobooks. These are sometimes referred to as secondary tasks, minor tasks, secondary activities, or other activities. However, these terms can refer to the activities themselves, to generally safe activities outside of driving, or to individually safe activities outside of driving depending on various circumstances. The following will further explain the definition of safe activities outside of driving, depending on various circumstances. The scope of what is considered a permissible activity may change over time depending on various circumstances.

[0056] The Status Management Unit Y1 switches between different levels of autonomous driving as needed. At Level 0, the Driving Support ECU 37 and Autonomous Driving ECU 40 do not actually intervene in driving behavior; instead, they directly control the Driving Control ECU 36 and the driving actuators based on the driver's actions. At Level 1 or 2, the Driving Support ECU 37 supports the driver's actions while controlling the Driving Control ECU 36 and the driving actuators. At Level 3, the Driving Control ECU 36 and the driving actuators are controlled based on calculations performed by the Autonomous Driving ECU 40.

[0057] When vehicle 1 is traveling within the aforementioned restricted area, the status management unit Y1 sets the maximum level of autonomous driving to Level 3. Conversely, when vehicle 1 travels outside the restricted area, the status management unit Y1 generally sets the maximum level of autonomous driving to Level 2, and under exceptional conditions, sets the maximum level to Level 3. Such exceptional conditions include, for example, traffic congestion on the road where vehicle 1 is traveling, and conditions where vehicle 1 is traveling at a speed lower than a prescribed speed. Based on the above, the restricted area shown in map database 34a, and the traffic congestion zone that constitutes the low-speed travel, become the permitted area for at least a portion of the second task.

[0058] The Status Management Unit Y1 may downgrade the autonomous driving level from Level 3 to Level 2 or below, even when Vehicle 1 is driving within a restricted area. For example, the Status Management Unit Y1 may abruptly downgrade the autonomous driving level to Level 2 or below if the autonomous driving function deteriorates from its operating conditions or if it anticipates such a deterioration. For instance, if environmental conditions deteriorate due to severe weather such as heavy rain, snowfall, dense fog, or sandstorms, reducing the detection accuracy of the surrounding monitoring sensors 33, the autonomous driving function may deteriorate from its operating conditions. Additionally, the autonomous driving function may deteriorate from its operating conditions if a traffic accident occurs around the vehicle.

[0059] When the Status Management Unit Y1 decides to downgrade the automated driving level from Level 3 to Level 2 or below due to the planned end of the permitted area or the departure from the conditions of use, it requests the HCU 10 to display a warning of a driving change to the driver via the Information Display Device 20. If the driver responds to the driving change, the Status Management Unit Y1 actually downgrades the automated driving level from Level 3 to Level 2 or below. If the driver does not respond to the driving change even after the warning, even if there is a specified waiting time, the Status Management Unit Y1 can decide to apply MRM (Minimal Risk Management) control to the vehicle 1. MRM control is vehicle control that minimizes risk in an emergency; specifically, it refers to the control of searching for and setting a retreat location and moving and stopping the vehicle at that retreat location.

[0060] When the state management unit Y1 sets the autonomous driving level to level 3, the driving control unit Y2 acts as the implementer of driving behavior, controlling the driving control ECU 36. Specifically, the driving control unit Y2 identifies the road shape and the positions of other vehicles based on locator information and map information obtained from the locator 34, and detection information obtained from the surrounding monitoring sensors 33. Based on the identification results of the road shape and the positions of other vehicles, the driving control unit Y2 generates a predetermined driving line for the vehicle 1. Through cooperation with the driving control ECU 36, the driving control unit Y2 guides the vehicle 1 to travel along the generated predetermined driving line.

[0061] When the above-mentioned MRM control is implemented, the driving control unit Y2 causes vehicle 1 to issue warnings such as sirens to the surrounding area and gradually reduces the speed of vehicle 1, so that vehicle 1 stops at a safer place such as the side of the road.

[0062] Next, the details of the information display device 20 and HCU10 included in the information display system IPS will be explained in turn.

[0063] Information display device 20 is a device that displays information to the driver. For example... Figure 1As shown, the information display device 20 is configured to include multiple displays 21 and multiple speakers 22. The multiple displays 21 are, for example, mounted on a dashboard. The multiple displays 21 may include, for example, a graphic meter and a center information display (CID). Furthermore, the displays 21 may also include a head-up display (HUD). Each display 21 is a device that provides visual information to the driver and is configured to display images. At least a portion of the multiple displays 21 (e.g., CIDs) combine display functionality and operation functionality such as accepting touch operations.

[0064] Multiple speakers 22 are disposed, for example, near the display screen in the dashboard, on the door panels, or in the rear quarter panel. Each speaker 22 is a device that provides auditory information to the driver. Specifically, each speaker 22 converts the input electrical signal into a physical signal using a voice coil and a diaphragm, and is able to produce sound. The speakers 22 can be set independently corresponding to each display 21, or they can be set together with some or all of the displays 21, or they can be set completely independently of the displays 21.

[0065] HCU10 is an electronic control device that works in conjunction with the autonomous driving ECU40 and the information prompting device 20 to uniformly control information prompts for the driver and other passengers. HCU10 is configured as a computer containing a processing unit 11, RAM 12, storage unit 13, input / output interface 14, and a bus connecting them. The processing unit 11 is hardware integrated with RAM 12 for computational processing. The processing unit 11 is configured to include at least one CPU, GPU, RISC, or other computational core. The processing unit 11 may also be configured to include an FPGA or other dedicated IP core. RAM 12 may also include video RAM for image generation. The processing unit 11 performs various processes to implement the functions of the functional units described later by accessing RAM 12. The storage unit 13 is configured, for example, to include at least one non-volatile storage medium such as a semiconductor memory. The storage unit 13 stores various computer programs (e.g., information prompting programs) that are executed by the processing unit 11.

[0066] The HCU10 has multiple functional units, which execute programs stored in the storage unit 13 by the processing unit 11. Specifically, the HCU10 is as follows: Figure 2 As shown, it includes a driving environment estimation unit X1, a wakefulness estimation unit X2, a pressure estimation unit X3, a permissible behavior identification unit X4, a passenger monitoring unit X5, an alarm judgment unit X6, a retreat request unit X7, and an information prompt content control unit X8.

[0067] The driving environment estimation unit X1 estimates the driving environment of the road on which the vehicle 1 is traveling based on detection information obtained from the surrounding monitoring sensor 33, locator information obtained from the locator 34, map data, and traffic information obtained through the external communication unit 35. The estimation of the road driving environment can be further enhanced by using information such as road shape, the position of other vehicles, and the predetermined driving line information identified by the driving control unit Y2 of the autonomous driving ECU 40.

[0068] The driving environment is a concept that includes road shape and driving scenario. Road shape includes road width, road slope, road curvature, and the shape of road lane markings. Driving scenario includes the setting of restricted areas and traffic volume. The driving environment information estimated by the driving environment estimation unit X1 is provided to the permitted behavior recognition unit X4. The driving environment estimation unit X1 updates the driving environment information in a timely manner based on the latest information.

[0069] The estimated driving environment includes the driving environment of the road on which vehicle 1 is currently traveling. In the first embodiment, the estimated driving environment also includes the driving environment of the road on which vehicle 1 is scheduled to travel.

[0070] The alertness estimation unit X2 estimates the driver's alertness based on driver state information obtained from DSM31. Driver alertness is an index representing the driver's drowsiness through a rating scale. Higher alertness ratings indicate that the driver is closer to a sleep state. The alertness rating is taken as a discrete numerical value, an integer between 0 and 3.

[0071] For example, if it is confirmed that the driver's gaze moves quickly and frequently, the blinking cycle is stable, and the driver's actions are active, the alertness estimation unit X2 will estimate the alertness as level 0.

[0072] After confirming the driver's open lips, slow eye movement, slow and frequent blinking, frequent slumping, and hands on face, the alertness estimation unit X2 estimates the alertness as level 1.

[0073] If the driver is found to be consciously blinking, making useless bodily movements such as shaking their head and moving their shoulders up and down, frequently yawning or taking deep breaths, and blinking and moving their eyes slowly, the alertness estimation unit X2 estimates the alertness level as Level 2.

[0074] Upon confirming that the driver's head is tilted forward, tilted backward, or has its eyelids closed for more than several seconds, the alertness estimation unit X2 estimates the alertness level as Level 3. This alertness information, estimated by the alertness estimation unit X2, is then provided to the permitted behavior recognition unit X4. The alertness estimation unit X2 updates the alertness information in a timely manner based on the latest data.

[0075] The stress estimation unit X3 estimates the driver's stress based on the driver's state information acquired from the DSM31. Specifically, the stress estimation unit X3 estimates whether the driver is experiencing stress. The estimation of the presence or absence of driver stress can be achieved, for example, by analyzing the driver's blurred vision. Furthermore, by installing biosensors that detect the driver's heart rate in the driver's seat, steering wheel, etc., and using the heart rate information detected by these biosensors, the estimation accuracy of the stress estimation unit X3 can be improved. The stress information estimated by the stress estimation unit X3 is provided to the permissible behavior recognition unit X4. The stress estimation unit X3 updates the stress information in a timely manner based on the latest information.

[0076] The permitted behavior identification unit X4 identifies permitted behaviors for the driver based on driving environment information obtained from the driving environment estimation unit X1, alertness information obtained from the alertness estimation unit X2, and pressure information obtained from the pressure estimation unit X3. The permitted behavior identification unit X4 identifies the currently permitted behaviors for the driver. The permitted behavior identification result of the permitted behavior identification unit X4 may change according to changes in these driving environment information, alertness information, and pressure information, i.e., changes in various conditions. During the operation of the autonomous driving function, the permitted behavior identification unit X4 updates the identification result in a timely manner based on the latest information provided, corresponding to changes in driving environment information, alertness information, or pressure information.

[0077] Specifically, the discrimination theory of the permissible behavior discrimination unit X4 is constructed based on the premise that, when the autonomous driving function is operating at Level 3, the driver can maintain a "minimum safe posture and be able to perform driving handover in an emergency." That is, within the scope of the driver's ability to safely take over driving in the event of a sudden driving handover in Level 3 autonomous driving, behaviors other than driving are permitted as permissible behaviors.

[0078] In identifying permitted actions, the permitted action identification unit X4 sets multiple identification viewpoints. Furthermore, the permitted action identification unit X4 temporarily calculates a separate identification result for each identification viewpoint. Subsequently, the permitted action identification unit X4 definitively identifies the actions among the separately permitted actions calculated for each identification viewpoint that are commonly identified as permitted actions across all identification viewpoints as permitted actions.

[0079] In the first embodiment, which defines the scope of permissible behavior in stages according to the driver requirements described below, the permissible behavior identification unit X4 extracts the determination result of the narrowest scope of permissible behavior (i.e., the lowest permissibility) from the provisional results of each identification viewpoint. The permissible behavior identification unit X4 takes the determination result of the narrowest scope of permissible behavior as the final identification result. The permissible behavior identification unit X4 provides the information of the determined identification result to the information prompt content control unit X8.

[0080] Driver requirements are indicators that categorize non-driving behaviors in stages based on the aforementioned discrimination theory. Higher driver requirement levels indicate a wider range of permitted behaviors (in other words, a higher degree of permissibility). Driver requirements are represented as discrete integers ranging from 0 to 4. A driver requirement level of 0 indicates that non-driving behaviors are not permitted and are prohibited.

[0081] Driver qualification level 1 behaviors are those that allow a driver to easily return to a driving state. Examples include: operating a mobile phone, smartphone, or CID with one hand; maintaining a stable posture without leaning to the left or right; and maintaining full visual awareness of the position of the eyes on multiple displays 21. Furthermore, examples include: immediately recognizing driving transition requests using a low to medium volume sound from speaker 22; and maintaining a relatively stable posture without reclining the driver's seat. In short, behaviors that allow full use of all five senses even when returning to driving are driver qualification level 1 behaviors.

[0082] Driver qualification level 2 behaviors are those actions that enable a driver to return to a driving state. Examples include using a mobile phone, smartphone, or CID device with both hands; tilting the body to the left or right by approximately ±20 degrees; and maintaining eye position that allows visual confirmation of the location of all but one of the multiple displays 21. Furthermore, actions such as using a sound from the speaker 22 at a medium to high volume, or reclining the driver's seat at approximately 45 degrees, can also be considered driver qualification level 2 behaviors.

[0083] Driver requirement level 3 behaviors are those that enable a driver to return to driving in an emergency. Examples include placing an object on one's knees, tilting the body to the left or right by approximately ±130 degrees, and maintaining the position of the eyes (excluding two) on multiple monitors 21.

[0084] Driver qualification level 4 behaviors are those that may prohibit the implementation of level 3 autonomous driving. Examples include sleeping, drinking alcohol, leaving the seat, unfastening the seatbelt, reclining the driver's seat to approximately 90 degrees, and wearing headphones. Furthermore, when the autonomous driving function is operating at level 3, the permitted behavior recognition unit X4 typically does not recognize driver qualification level 4 behaviors as permitted behaviors.

[0085] The following is a detailed explanation of the permissible behavior discrimination unit X4. The permissible behavior discrimination unit X4 has four discrimination perspectives. The first discrimination perspective is the road shape in the driving environment. The permissible behavior discrimination unit X4 discriminates permissible behavior based on road shape information obtained from the driving environment estimation unit X1. Specifically, the permissible behavior discrimination unit X4 determines the road shape of the road on which the vehicle 1 travels based on factors such as road width, road slope, road curvature, and the shape of road lane markings.

[0086] Furthermore, the permissible behavior recognition unit X4 recognizes behaviors up to driver qualification level 3 as permissible when the road is straight. When the road is curved, it recognizes behaviors up to driver qualification level 2. Specifically, when the road on which vehicle 1 travels is curved and the length of the curve exceeds a predetermined distance, the permissible behavior recognition unit X4 narrows the range of permissible behaviors, recognizing behaviors up to driver qualification level 2 as permissible. Additionally, when the road requires lane changing, the permissible behavior recognition unit X4 recognizes behaviors up to driver qualification level 1 as permissible. In other words, under the assumption of driver-in-control, the higher the difficulty of the driving behavior immediately after driver takeover, the narrower the range of permissible behaviors becomes.

[0087] The second discrimination point is the driving scenario within the driving environment. The permissible behavior discrimination unit X4 discriminates permissible behavior based on information about the driving scenario obtained from the driving environment estimation unit X1. Specifically, the permissible behavior discrimination unit X4 determines the driving scenario of vehicle 1 based on factors such as the setting of a restricted area and traffic volume.

[0088] Furthermore, when the permitted behavior recognition unit X4 is driving within a restricted area during non-traffic congestion, it recognizes behaviors up to driver qualification level 3 as permitted behaviors. When the permitted behavior recognition unit X4 is driving within a restricted area during traffic congestion, it recognizes behaviors up to driver qualification level 3 as permitted behaviors. When the permitted behavior recognition unit X4 is driving outside the restricted area during traffic congestion, it recognizes behaviors up to driver qualification level 2 as permitted behaviors. In other words, the higher the probability of a sudden driving alternation with the driver, the narrower the range of permitted behaviors becomes.

[0089] Furthermore, when the driving scenario involves moving from within a restricted area to outside the restricted area, the permitted behavior recognition unit X4 determines the permitted behavior based on the distance or arrival time from vehicle 1 to the boundary of the restricted area. The shorter the distance or arrival time to the boundary, the narrower the range of permitted behavior is for the permitted behavior recognition unit X4.

[0090] The third distinguishing factor is alertness. The permissible behavior distinguishing unit X4 distinguishes permissible behaviors based on the driver's alertness obtained from the alertness estimation unit X2. When the driver's alertness level is 0, the permissible behavior distinguishing unit X4 distinguishes behaviors up to driver requirement level 3 as permissible behaviors. When the driver's alertness level is 1, the permissible behavior distinguishing unit X4 distinguishes behaviors up to driver requirement level 2 as permissible behaviors. When the driver's alertness level is 2, the permissible behavior distinguishes behaviors at driver requirement level 1 as permissible behaviors. In other words, the lower the driver's ability to respond to sudden driving transitions, the narrower the range of permissible behaviors becomes.

[0091] Furthermore, if the driver's alertness level is 3, it is estimated that the driver is unable to perform emergency driving handover, so the behavior identification unit X4 is allowed to output a command to the alarm judgment unit X6 to issue an alarm.

[0092] The fourth distinguishing factor is pressure. The permissible behavior distinguishing unit X4 distinguishes permissible behaviors based on the driver's pressure obtained from the pressure estimation unit X3. When the driver is not under pressure, the permissible behavior distinguishing unit X4 distinguishes behaviors up to driver competence level 3 as permissible behaviors. When the driver is under pressure, the permissible behavior distinguishes behaviors up to driver competence level 2 as permissible behaviors. That is, under the assumption of driver takeover, the higher the probability of operational error in driving immediately after takeover, the narrower the range of permissible behaviors becomes.

[0093] Of the above provisional identification results, the results for the lower-level identification perspectives regarding driver requirements will be adopted as the definitive identification results. This is because by allowing only behaviors that meet the safety benchmarks of each identification perspective, the appropriateness of the information prompts related to permitted behaviors described later is improved.

[0094] Furthermore, the behavior recognition unit X4 is allowed to roughly predict the range of permissible behaviors along the driving path up to the destination based on multiple recognition viewpoints from which the future can be predicted. In the first embodiment, the driving environment of the road on which the vehicle 1 is scheduled to travel can be estimated, so the driving environment corresponds to the future predictable recognition viewpoint.

[0095] The Passenger Monitoring Unit X5 monitors the driver's behavior based on driver status information acquired from the DSM31 and driver seating information from the seat sensor 32. The Passenger Monitoring Unit X5 can comprehensively analyze the image analysis results from the images captured by the DSM31 and the seating information to determine the driver's behavior. The determination of driver behavior can also utilize a learned artificial intelligence model built primarily using a neural network. The Passenger Monitoring Unit X5 provides the monitoring results of the driver's behavior to the Alarm Judgment Unit X6 and the Information Prompt Content Control Unit X8.

[0096] The alarm determination unit X6 compares the monitoring results of the driver's behavior obtained from the passenger monitoring unit X5 with the identification results of permitted behaviors obtained from the permitted behavior identification unit X4, and determines whether to issue an alarm to the driver. For example, if the driver continues to engage in behavior other than driving that is identified as not permitted behavior (hereinafter, non-permitted behavior) by the permitted behavior identification unit X4 for more than the prescribed alarm setting time, the alarm determination unit X6 decides to issue an alarm to the driver.

[0097] Furthermore, if a command to issue an alarm is input from the permitted behavior recognition unit X4 as described above, the alarm determination unit X6 decides to issue an alarm to the driver. The alarm determination unit X6 then provides the alarm issuance information to the information prompt content control unit X8.

[0098] If, after the alarm has been triggered, the driver continues to engage in unauthorized behavior for an extended period beyond the pre-defined retreat time, the retreat request unit X7 requests the automatic driving ECU 40 to force the vehicle 1 to retreat. The retreat mentioned here is, for example, the MRM control described above.

[0099] The information prompt content control unit X8 controls the content that the information prompt device 20 displays. The information prompt content control unit X8 obtains a discrimination result from the permitted behavior discrimination unit X4 and outputs a prompt request to the information prompt device 20 to display information related to the discrimination result. The information prompt content control unit X8 obtains the determined discrimination result and causes the information prompt device 20 to display information related to the determined discrimination result.

[0100] The information prompt content control unit X8 enables the information prompt device 20 to display information related to the identification result indicating the behavior that has been identified as permissible. For example, if the behavior is identified as permissible up to driver qualification level 2, the information indicating that the driver's seat reclined to 45 degrees is permissible.

[0101] For example, if the behavior up to driver qualification level 3 is determined to be permissible, the information prompt content control unit X8 prompts the information prompt device 20 to display information indicating the main idea of ​​playing a movie on the display 21 (e.g., CID). In such a case, if a functional restriction is imposed on the information prompt device 20 to enable the permissible behavior of using the information prompt device 20, the information prompt content control unit X8 can request the removal of the restriction. The information prompt content control unit X8 requests the removal of the restriction on the information prompt device 20 or the device that imposes the restriction on the information prompt device 20. For example, if the display 21 imposes a functional restriction on the projection of a movie, the information prompt content control unit X8 requests the removal of the functional restriction so that the display 21 can perform the projection of a movie.

[0102] Additionally, the information prompt content control unit X8 enables the information prompt device 20 to display information related to the identification result indicating the behavior identified as non-permitted. For example, if the behavior is identified as permitted up to driver qualification level 2, the information indicating the behavior is that the driver's seat reclined more than 45 degrees.

[0103] When the scope of permitted behaviors (i.e., driver qualification levels) changes in accordance with the updated determination of permitted behaviors, the information prompt content control unit X8 enables the information prompt device 20 to display information related to the determination of permitted behaviors. The greater the change in the scope of permitted behaviors in this timely prompt, the more the information prompt content control unit X8 raises the tone of the prompt. The greater the change, the more the prompt is toned up. For example, if the upper limit of driver qualification levels changes from 3 to 1, the change is 2; if the upper limit of driver qualification levels changes from 2 to 3, the change is 1.

[0104] When adjusting the tone (or tone down) for an image-based display, options include increasing saturation, brightness, display size, and making lines thicker. Conversely, when adjusting the tone (or tone down) for a sound-based notification, options include increasing volume, using higher frequencies, and enhancing sound emphasis.

[0105] When providing information related to the discrimination result, the information prompt content control unit X8 selects the device in the information prompting device 20 for information prompting based on the monitoring results of the driver's behavior obtained from the passenger monitoring unit X5. That is, considering the nature of the driver's behavior other than driving, the information prompting device 20 is selected to be the device that is more easily noticed for information prompting.

[0106] For example, when the driver is watching a movie, information prompts are displayed on the monitor 21. More specifically, when the driver is watching a movie using a CID (Chip ID), information prompts are displayed on the CID. Additionally, when the driver is operating a smartphone, eating, or doing other activities, information prompts are delivered via the speaker 22.

[0107] Furthermore, when providing information related to the identification result, the information prompt content control unit X8 can determine the detailed content of the prompt based on the monitoring results of the driver's behavior obtained from the passenger monitoring unit X5. For example, if the driver engages in an unauthorized behavior, the information prompt content control unit X8 can instruct the information prompt device 20 to provide at least one of the reasons why the unauthorized behavior should not be carried out and a suggestion to move the unauthorized behavior to a permitted behavior. Such information prompts are continuously provided, for example, until the driver stops the unauthorized behavior.

[0108] As an example, let's consider a scenario where an action classified as driver requirement level 1-2 is a permitted action, the reclining chair angle is less than 45 degrees, and the driver reclines the reclining chair to 70 degrees. In this case, an example of a reason why this non-permitted action should not be performed could be given: "Since the road is curved, considering the driver's workload immediately after taking over driving, it should be possible to return to a state where driving can be performed through some action." Furthermore, as an example of a suggestion to change a non-permitted action to a permitted action, the reason could be given: "Please return the reclining chair angle to less than 45 degrees."

[0109] Even though the driver has been informed of the permitted behavior via the information display device 20, if the driver continues to engage in an unpermitted behavior, the passenger monitoring unit X5 monitors the continuation of such unpermitted behavior. If the prescribed alarm setting time for the unpermitted behavior continues, the alarm judgment unit X6 decides to issue an alarm as described above. Then, the information display content control unit X8 causes the information display device 20 to issue an alarm to the driver. Preferably, the information display content control unit X8 selects the alarm device from among the multiple information display devices 20. Furthermore, the information display content control unit X8 raises the alarm level compared to the information display situation described above. Alternatively, in addition to such an alarm, the information display content control unit X8 may also lower the autopilot level, in other words, cause the information display device 20 to issue a warning of a driving transition that would terminate the autopilot without the obligation to monitor the surrounding area.

[0110] On the other hand, when the driver performs a permitted action, the information prompt content control unit X8 may prompt information related to the discrimination result without exceeding what is necessary. However, the information prompt content control unit X8 prompts information related to the discrimination result from time to time (e.g., periodically) based on a limited time interval set as a time interval to limit the frequency of information prompts.

[0111] In such prompts, it is preferable to present the information in a way that is less forceful than when the driver is performing an unauthorized action, so as not to hinder the driver's permitted actions. Here, "less forceful" is the opposite of "more forceful." Examples of "less forceful" actions, if based on an image display, include reducing saturation, reducing brightness, decreasing display size, and thinning lines used in the display. Similarly, if based on an audio notification, examples of "less forceful" actions include lowering volume, avoiding the use of high-pitched sounds, and reducing emphasis.

[0112] Next, based on Figure 3 The flowchart describes the method of providing information based on the information prompting program stored in the storage unit 13 and executed by the processing unit 11.

[0113] First, in S10, the HCU10's permission behavior recognition unit X4 determines whether the autonomous driving level managed by the autonomous driving ECU40's state management unit Y1 is at level 3, or whether it is predetermined to become level 3. If an affirmative determination is made in S10, the process moves to S11. If a negative determination is made in S10, the processing in S10 is repeated, for example, after a predetermined re-determination time has elapsed.

[0114] In S11, estimations related to each discrimination point are performed. Specifically, the driving environment estimation unit X1 estimates the driving environment of the road on which vehicle 1 travels. The alertness estimation unit X2 estimates the driver's alertness. The pressure estimation unit X3 estimates the driver's pressure. After processing in S11, the process moves to S12.

[0115] In S12, the permitted behavior identification unit X4 identifies the permitted behavior of the driver. After processing in S12, the process moves to S13.

[0116] In S13, the information prompt content control unit X8 causes the information prompt device 20 to display information related to the determination result of the permitted behavior. The driver can then receive information related to the determination result of the permitted behavior. After processing in S13, the process proceeds to S14.

[0117] In S14, the passenger monitoring unit X5 and the alarm judgment unit X6 determine whether the driver continues to engage in unauthorized behavior beyond the alarm set time. If an affirmative determination is made in S14, the alarm is triggered, and the process proceeds to S15. If a negative determination is made in S14, the process returns to S11.

[0118] In step S15, the information prompt content control unit X8 causes the information prompt device 20 to issue an alarm to the driver. The driver is able to recognize the alarm. After processing in step S15, proceed to step S16.

[0119] In S16, the retreat request unit X7 determines whether the driver continued to engage in unpermitted behavior beyond the set retreat time after the alarm was initiated. If an affirmative determination is made in S16, the process proceeds to S17. If a negative determination is made in S16, the process returns to S11.

[0120] In S17, the retreat request unit X7 requests the automatic driving ECU 40 to force vehicle 1 to retreat. The automatic driving ECU 40 implements MRM control. The series of processes ends at S17.

[0121] Next, based on Figure 4 as well as Figure 5 The timing diagram illustrates an example of the operation of the autonomous driving ECU40 and HCU10. Figure 4 as well as Figure 5 In the diagram showing non-driving behaviors, solid lines indicate the driver's actual non-driving behavior and the corresponding driver qualification level. Dashed lines represent the results identified by the behavior discrimination unit X4 as the driver qualification level.

[0122] Figure 4 The example illustrates the scenario where the autonomous driving level moves from Level 0 to Level 3, and further to MRM control.

[0123] First, in driving control, the level of automated driving gradually increases from level 0 to level 1 and 2. At this point, actions other than driving are not actually permitted, so actions performed by the driver are considered equivalent to level 0 actions, which are considered driver qualification levels.

[0124] If the autonomous driving level becomes Level 3, the range of permissible behaviors determined by the behavior recognition unit X4 is set to behaviors up to Driver Qualification Level 3. At this time, event P1 is executed as an event that prompts information via the information prompting device 20 based on the request from HCU10 (hereinafter referred to as the information prompt event). Event P1 is an event that prompts the driver with information indicating that the autonomous driving level has moved to Level 3 and that the range of permissible behaviors is up to Driver Qualification Level 3. Assuming that in response to such information prompts, the driver gradually performs behaviors at Driver Qualification Level 1, Level 2, and Level 3, as behaviors other than driving.

[0125] Based on predictable future behavior, the permissible behavior identification unit X4 narrows down the range of permissible behaviors to behaviors up to driver qualification level 2. In this way, event P2 is implemented as an information notification event. Event P2 is an event that provides the driver with advance notice that the predetermined permissible behaviors have been narrowed down from driver qualification level 3 to behaviors up to level 2.

[0126] Furthermore, if the scope of currently permitted behaviors is narrowed down to behaviors up to driver qualification level 2, then event P3 is implemented as an information notification event. Event P3 is an event that notifies the driver that permitted behaviors have been narrowed down from driver qualification level 3 to behaviors up to level 2.

[0127] If, regardless of events P2 and P3, the driver continues to engage in driver competence level 3 behavior beyond the alarm set time, then event W1 is implemented as an information alert event. Event W1 is an event that alerts the driver to stop driver competence level 3 behavior. If, in response to event W1, the driver stops driver competence level 3 behavior, for example, by moving to driver competence level 1 behavior, then the countdown of the backoff set time stops.

[0128] Subsequently, it is assumed that the scope of permitted behavior is expanded to include behaviors up to Driver Qualification Level 3. In such a situation, if it is determined that the driver is performing a Driver Qualification Level 4 behavior, i.e., a behavior that prohibits the implementation of Level 3 automated driving, event W2 is immediately implemented. Event W2 is an event that alerts the driver to suspend Driver Qualification Level 4 behavior. This warning can be described as a warning that the driver's posture does not meet the minimum safe posture requirements.

[0129] If, regardless of event W2, the driver continues the driver requirement level 4 behavior avoidance setting time for more than the specified time, then event M1 is implemented as an information prompt event. Event M1 is an event that prompts the driver that driving control has been transferred to MRM control. Simultaneously, the driving control of the automatic driving ECU 40 is transferred to MRM control. This concludes the process. Figure 4 Explanation of working examples.

[0130] Figure 5 The example illustrates a so-called driving handover from vehicle 1 to the driver, where the level of automation changes from level 3 to level 2 as vehicle 1 moves from within the defined area to outside the defined area.

[0131] Vehicle 1 operates within a limited area at Level 3 of automated driving. As the distance from Vehicle 1 to the boundary of the limited area approaches, the range of permissible behaviors of the permissible behavior recognition unit X4 begins to narrow gradually from behaviors up to Level 3 of driver requirements.

[0132] If the scope of permitted behaviors narrows from the level 3 driver requirement, event P4 is implemented as an informational event. Event P4 is an event that provides the driver with information recommending the cessation of behaviors other than driving in preparation for a driving handover.

[0133] If, corresponding to event P4, the driver ceases any behavior other than driving, and vehicle 1 approaches the boundary of the restricted area, then event C1 is executed. Event C1 is an event that prompts the driver with a request for a handover of driving from vehicle 1 to the driver. If the driver takes over driving based on event C1, the level of automatic driving control in the driving system becomes level 2 or lower. End of explanation. Figure 5 Explanation of working examples.

[0134] (Effects)

[0135] The effects of the first embodiment described above will be explained again below.

[0136] According to the first embodiment, the information prompting device 20 displays information related to the determination result of the permitted behavior identified by the permitted behavior determination unit X4. When the autonomous driving function is in operation, which reduces the need for the driver to understand the driving environment and other conditions of the vehicle 1, the driver can obtain information related to the determination result of the permitted behavior via the information prompting device 20. Therefore, by referring to such information, the driver can easily take appropriate actions when the autonomous driving function is in operation.

[0137] Furthermore, according to the first embodiment, the information prompting device 20 displays information indicating permitted actions other than driving. By directly displaying the permitted actions, the driver's sense of security when performing such actions can be improved.

[0138] Furthermore, according to the first embodiment, the information prompting device 20 displays information indicating prohibited behaviors other than driving. By directly displaying prohibited behaviors, it is easier to avoid such behaviors when the driver is engaging in activities other than driving.

[0139] Furthermore, according to the first embodiment, permissible actions are determined based on the driving environment. When an autonomous driving function is in operation where the necessity for the driver to understand the driving environment of vehicle 1 is reduced, the driving environment is considered in the determination of permissible actions, replacing the driver. Therefore, the appropriateness of actions other than driving by the driver can be improved.

[0140] Furthermore, according to the first embodiment, permitted actions are identified based on the shape of the road on which vehicle 1 travels. That is, the identification result of permitted actions corresponds to the ease or difficulty of driving immediately after the driver takes over driving in the assumption of driver takeover. Therefore, it is possible to improve the safety of the driver's driving behavior after a driver takeover while enabling the driver to perform actions other than driving.

[0141] Furthermore, according to the first embodiment, when the road in which vehicle 1 travels is curved and the length of the curved section exceeds a predetermined distance, the range of permissible behavior is narrowed. That is, even if the road is curved, the range of permissible behavior remains at driver qualification level 3 when the curved section is short. Therefore, it is possible to avoid situations where, for example, the driver reclines to relax, a warning is issued whenever driving in a shorter curved section. On the other hand, when the curved section continues for a predetermined distance or longer, the driver qualification level is changed less, so an appropriate warning to the driver can be implemented.

[0142] Furthermore, according to the first embodiment, permitted actions are identified based on the driving scenario of vehicle 1. That is, the identification result of permitted actions corresponds to the probability of a sudden driving alternation with the driver. Therefore, it is possible to improve safety during driving alternations while enabling the driver to perform actions other than driving.

[0143] Furthermore, according to the first embodiment, permissible actions are identified based on the driver's alertness. That is, the identification result of the permissible action corresponds to the driver's response capability in dealing with sudden driving transitions. Therefore, it is possible to improve safety during driving transitions while enabling the driver to perform actions other than driving.

[0144] Furthermore, according to the first embodiment, permissible actions are identified based on the driver's stress level. That is, the identification result of the permissible action corresponds to the probability of operational error in driving immediately after driver takeover, as assumed in the driver takeover scenario. Therefore, it is possible to improve the safety of the driver's driving behavior after a driver takeover while simultaneously enabling the driver to perform actions other than driving.

[0145] Furthermore, according to the first embodiment, individual permissible actions are tentatively identified from multiple identification perspectives. Subsequently, actions that are commonly identified as permissible actions across all identification perspectives are identified as definitive permissible actions. By performing multi-angle identification based on multiple identification perspectives and definitively identifying the minimum common actions as permissible actions, the appropriateness of permissible action identification can be significantly improved.

[0146] Furthermore, according to the first embodiment, the permissible behavior identification unit X4 identifies permissible behaviors based on the driver's posture during non-driving activities, using driver criteria that establish a correlation between non-driving activities and the degree of permissibility of permissible behaviors as identification indicators. The identification theory, which is based on the driver maintaining a "minimum safe posture and being able to perform emergency driving alternation," can be embodied through relatively simple processing calculations. Therefore, the identification of permissible behaviors with high appropriateness can be easily achieved.

[0147] Furthermore, according to the first embodiment, the permissible behavior identification unit X4 identifies permissible behaviors based on the driver's seat position in non-driving activities, using driver criteria that establish a correlation between non-driving activities and the degree of permissibility of permissible behaviors as identification indicators. In this case, the driver's seat position is, for example, the tilt angle. A identification theory based on the driver's ability to maintain a "minimum safe posture and be able to perform emergency driving alternation" can be implemented through relatively simple processing calculations. Therefore, the identification of permissible behaviors with high appropriateness can be easily achieved.

[0148] Furthermore, according to the first embodiment, when a functional restriction is imposed on the information display device 20 to enable permitted actions for using the information display device 20, the restriction is required to be lifted. Therefore, the driver can smoothly perform the permitted actions for using the information display device 20.

[0149] Furthermore, according to the first embodiment, the device used for information prompting by the information prompting device 20 is selected from devices that provide visual information and devices that provide auditory information, based on an estimate of the driver's current behavior other than driving. This selection makes the prompted information easier for the driver to recognize. With easily recognizable information prompts, the driver can be easily guided to appropriate behavior.

[0150] Furthermore, according to the first embodiment, when it is estimated that the driver is committing an unauthorized act, the information prompting device 20 provides the reason why the unauthorized act is prohibited. By making the driver understand the reason, the driver can be easily guided to perform an appropriate act.

[0151] Furthermore, according to the first embodiment, when it is estimated that the driver has committed an unpermitted act, a suggestion is provided via an information prompting device to change the driver's currently unpermitted act to a permitted act. This suggestion allows the driver's action to be smoothly changed to a permitted act.

[0152] Furthermore, according to the first embodiment, if it is estimated that the driver will continue to engage in unauthorized behavior, an alarm is issued to the driver via the information prompting device 20. By enabling the driver to recognize the alarm, the unauthorized behavior can be urged to cease.

[0153] Furthermore, according to the first embodiment, along with the implementation of the alarm, a warning of a driving alternation that would terminate the automatic driving function is provided via the information prompting device 20. By making the driver aware that maintaining this state will terminate the automatic driving function, the effect of facilitating the termination of unauthorized behavior can be improved.

[0154] Furthermore, according to the first embodiment, after the alarm is triggered, if it is estimated that the driver will continue to engage in unauthorized behavior, the vehicle 1 is required to forcibly reverse. By causing the vehicle 1 to reverse, the continuation of inappropriate unauthorized behavior during the operation of the automatic driving function can be forcibly prevented.

[0155] Furthermore, according to the first embodiment, when it is estimated that the driver has performed a permitted action, information related to the determination result of the permitted action is periodically displayed via the information display device 20, while maintaining a time interval between restrictions. By separating the time intervals between restrictions, the frequency of information display is limited, thus reducing the inconvenience of information display for the driver.

[0156] Furthermore, according to the first embodiment, when the range of permissible behavior changes, the greater the change in the range of permissible behavior, the higher the information prompt from the information prompting device 20 is set. By appropriately setting the strength of the information prompt based on its impact on the driver's behavior other than driving, the inconvenience caused by the information prompts can be reduced.

[0157] (Second Implementation)

[0158] like Figure 6 As shown, the second embodiment is a variation of the first embodiment. The second embodiment will be described focusing on the differences from the first embodiment.

[0159] In the second embodiment, the behavior recognition unit X4 is allowed to correspond to the autonomous driving level of the state management unit Y1 when it reaches level 3, and a predetermined continuous operating interval for the autonomous driving function to operate continuously is estimated. The continuous operating interval refers to a section of the vehicle 1 that can travel along a predetermined driving line generated by the autonomous driving ECU 40 and maintain the autonomous driving level at level 3.

[0160] Furthermore, the permitted behavior recognition unit X4 estimates the point with the lowest permissibility of the permitted behavior recognition result within a continuous working range based on the recognition perspective that can be predicted in the future. In the second embodiment, similar to the first embodiment, the permissible behavior recognition result is determined according to driver requirements. Therefore, the driver requirement level corresponds to the permissibility of the recognition result. That is, the point with the lowest permissibility of the permitted behavior recognition result is the point with the lowest upper limit of the driver requirement level considered for the permitted behavior. The permitted behavior recognition unit X4 provides such continuous working range information, information on the point with the lowest permissibility, and information on the permissible behavior recognition result at that point to the information prompt content control unit X8.

[0161] Then, during the period when the information prompt content control unit X8 is working in accordance with the automatic driving function and the continuous working area, the information prompt device 20 always prompts information related to the identification result of the permitted behavior for the point with the lowest permissibility.

[0162] For example, assuming that actions up to driver qualification level 2 are permitted at this point, the system temporarily becomes open up to driver qualification level 3 during the continuous work interval. In this temporary state, the information prompt content control unit X8 also causes the information prompt device 20 to display information indicating that actions up to driver qualification level 2 are permitted.

[0163] Next, based on Figure 6 The flowchart describes the method of providing information based on the information prompting program stored in the storage unit 13 and executed by the processing unit 11.

[0164] S20 is the same as S10 in the first embodiment. If a positive determination is made in S20, the process moves to S21. If a negative determination is made in S20, the process of S20 is repeated, for example, after a predetermined re-determination time has elapsed.

[0165] In S21, the behavior discrimination unit X4 is allowed to estimate the continuous working interval. After processing in S21, the process moves to S22.

[0166] In S22, the permissible behavior discrimination unit X4 estimates the point at which the driver's requirement level for a permissible behavior is considered to be at its lowest within the continuous operating range. After processing in S22, the process moves to S23.

[0167] In S23, the information prompt content control unit X8 causes the information prompt device 20 to display information related to the determination result of the permitted behavior at the point estimated in S22. The driver can obtain information related to the determination result of the permitted behavior. The series of processes ends at S23. Alternatively, after the processing in S23, the processing equivalent to S14 to S17 of the first embodiment may be performed.

[0168] According to the second embodiment described above, it is possible to suppress situations where the range of permitted behaviors frequently changes within a continuous work zone, necessitating changes to behaviors other than driving performed by the driver each time. The driver can then continuously and consistently perform behaviors other than driving that are optimized within the continuous work zone.

[0169] (Third Implementation)

[0170] like Figure 7 as well as Figure 8 As shown, the third embodiment is a variation of the first embodiment. The third embodiment will be described focusing on the differences from the first embodiment.

[0171] In the third embodiment of this disclosure, such as Figure 7 As shown, the function control device controls the function of the controlled object device CE. In the third embodiment, the function control device is the automatic driving ECU 340 used by the vehicle 1. The automatic driving ECU 340 of the third embodiment can not only realize automatic driving, but also control the function of the controlled object device CE to support the appropriate implementation of actions other than driving by the driver in the state of automatic driving level 3.

[0172] The control device CE here includes one or more in-vehicle devices configured to support actions other than driving by the driver. As an example, multiple in-vehicle devices such as the driver's seat 38, display 21, and speaker 22 are included in the control device CE.

[0173] In addition to the functional units such as the status management unit Y1 and the driving control unit Y2 provided in the first embodiment of the autonomous driving ECU 40, the autonomous driving ECU 340 also has functional units such as the driving environment estimation unit Y3, the alertness estimation unit Y4, the stress estimation unit Y5, the permissible behavior identification unit Y6, and the function restriction unit Y7.

[0174] The driving environment estimation unit Y3, alertness estimation unit Y4, pressure estimation unit Y5, and permissible behavior discrimination unit Y6 are the same as those provided in the HCU10 of the first embodiment, namely the driving environment estimation unit X1, alertness estimation unit X2, pressure estimation unit X3, and permissible behavior discrimination unit X4. However, the permissible behavior discrimination result of the permissible behavior discrimination unit Y6 is provided to the function restriction unit Y7.

[0175] Based on the results of permissible behavior identification, the function restriction unit Y7 restricts the function of the control device CE that enables the driver to perform non-permissible behaviors. The function restriction unit Y7 also restricts the angle at which the driver's seat 38 can tilt based on the permissible behavior identification results. For example, if the permissible behavior identification unit Y6 allows behaviors up to driver requirement level 2, the function restriction unit Y7, through the tilt drive device provided on the driver's seat 38, limits the tilt angle of the driver's seat 38 to more than 45 degrees. By limiting the tilt angle, the driver can easily maintain the safe posture required to cope with sudden driving transitions.

[0176] The function restriction unit Y7 restricts the display content of the display 21 based on the result of the permission behavior identification. For example, if the permission behavior identification unit Y6 allows behaviors up to driver requirement level 1, the function restriction unit Y7 restricts the functions of games that require two-handed touch operation on the display 21 (e.g., CID). By restricting the display content of the display 21, the driver can easily maintain a state that allows for emergency driving alternation in response to different situations.

[0177] The function limiting unit Y7 limits the volume of speaker 22 based on the result of permissible behavior identification. For example, if the permissible behavior identification unit Y6 allows behaviors up to driver requirement level 1, the function limiting unit Y7 sets an upper limit on the volume that speaker 22 can output in relation to the implementation of behaviors other than driving, so that the volume of speaker 22 does not become too loud. By setting the upper limit on the volume, the driver can easily maintain a state that allows for emergency driving alternation in response to the situation.

[0178] Next, based on Figure 8 The flowchart describes the method for limiting the function of the controlled device CE based on the function control program stored in the storage unit 43 and executed by the processing unit 41.

[0179] The processes performed in S30 to S32 are essentially the same as those performed in S10 to S12 of the first embodiment. After the process in S32, the steps are moved to S33.

[0180] In S33, the function restriction unit Y7, based on the identification results of the permitted behavior in S32, restricts the functions of the driver's seat 38, display 21, and speaker 22 as needed. The series of processes ends in S33.

[0181] Furthermore, in the third embodiment, the HCU10 may or may not have the function of prompting the information device to display information related to the identification result of the permitted behavior.

[0182] According to the third embodiment described above, based on the permitted behaviors identified by the permitted behavior identification unit Y6, the functions of the control object device CE, which can perform behaviors other than permitted behaviors other than driving, are restricted. When the autonomous driving function is operating, the necessity for the driver to understand the driving environment and other conditions of the vehicle 1 is reduced, and the possibility of the driver accidentally performing behaviors other than permitted behaviors is suppressed by the functional restrictions. Therefore, it is easier for the driver to take appropriate actions when the autonomous driving function is operating.

[0183] (Fourth Implementation)

[0184] Figure 9 as well as Figure 10 The example shown is an example of the operation of the autonomous driving ECU 40 and HCU 10 in the fourth embodiment. Figure 4 as well as Figure 5 A variation of the first embodiment shown. Figure 9 as well as Figure 10 In the diagram, the solid line represents the driver qualification level that the driver's actions meet, and the dotted line represents the recognition result of the permissible behavior identification unit X4, i.e., the driver qualification level.

[0185] exist Figure 9 In the illustrated example, the information prompt content control unit X8 sequentially executes event P1, which notifies the driver of a change to Level 3 autonomous driving, and events P2 and P3, which warn of a change to Level 2 driver qualification. Furthermore, if the driver continues to perform actions exceeding the driver qualification level identified by the permitted behavior identification unit X4, the information prompt content control unit X8 executes event W1, which requests the cessation of the ongoing action.

[0186] When the behavior detection unit X4 issues a warning for an unauthorized act via the information prompt content control unit X8, it fixes the driver's competence level at a predetermined level for a pre-defined period. Specifically, the behavior detection unit X4 sets the period until the main power supply or ignition switch of the vehicle 1 is turned off (hereinafter referred to as the restriction period LT), and during the restriction period LT, it sets the driver's competence level to a level lower than the normal level under which no warning is issued (see reference LT). Figure 4 Low. The permitted behavior identification unit X4 keeps the driver's qualification level fixed at the lowest level, 1, until the end of the restriction period LT. During the restriction period LT, the information prompt content control unit X8 prompts the driver with information indicating that the permitted behavior identification unit X4 has intentionally kept the driver's qualification level low.

[0187] exist Figure 10In the scenario shown in the work example, based on the predetermined end of the permitted area (restricted area) or the departure from the usage conditions, the State Management Unit Y1 decides to downgrade the autonomous driving level from Level 3 to below Level 2. Furthermore, if the State Management Unit Y1 estimates that the driver will continue to engage in behavior other than permitted, it also decides to terminate Level 3 autonomous driving. The State Management Unit Y1 sets a transition preparation interval STR before the driving transition, based on the time (refer to event C1) or location of the driving handover from the autonomous driving ECU 40 to the driver. The State Management Unit Y1 sets the transition preparation interval STR from the time or location of the end of event P4, which notifies the reduction of the driver's requirement level, to the time or location of the start of the driving handover.

[0188] During the transition preparation interval STR prior to a driving change, the permitted behavior identification unit X4 fixes the driver's requirements at a predetermined level or below. For example, after receiving an information prompt based on event P4 from the information prompt content control unit X8, the permitted behavior identification unit X4 narrows the range of permitted behaviors from driver requirement level 3, etc., to driver requirement levels 1 or 0, which are preset as the transition preparation range. After continuing to fix the driver requirement level to the lowest level 1 or below, the permitted behavior identification unit X4 moves to driver requirement level 0, which prohibits driving behaviors.

[0189] When the Information Prompt Content Control Unit X8 determines that Level 3 automated driving is scheduled to end based on factors such as the end of the permitted area, disengagement from usage conditions, or continuation of inappropriate behavior, it will at least provide a warning of a driving handover that would terminate the automated driving function in the information prompt at event P4. During the handover preparation period STR prior to the warned end of automated driving, the Information Prompt Content Control Unit X8 will provide the driver with information indicating the driver's requirement level that limits the handover preparation range. Furthermore, the Information Prompt Content Control Unit X8 will use event C1 to provide the driver with information requesting a driving handover.

[0190] In the fourth embodiment described so far, the same effect as in the first embodiment is achieved: the driver can obtain information related to the identification result of the permitted behavior, so it is easy to take appropriate action.

[0191] Furthermore, according to the fourth embodiment, after a warning about the unauthorized behavior is given, the driver's requirement level is fixed at level 1 during the subsequent restriction period LT. This sanction control makes the driver aware of not performing the unauthorized secondary task. As a result, the driver is prevented from over-relying on the automatic driving function, making it easier for the driver to take appropriate action during automatic driving.

[0192] Furthermore, according to the fourth embodiment, when the information prompt content control unit X8 has determined the predetermined end of the permitted area, the information prompt device 20 prompts a warning of a driving alternation that will terminate the automatic driving function. Therefore, in conjunction with the permitted behavior recognition unit X4 narrowing down the scope of permitted behaviors, the driver can be prompted to stop behaviors other than driving, allowing the driving operation to begin smoothly.

[0193] Furthermore, according to the fourth embodiment, the permissible behavior recognition unit X4 narrows the range of permissible behaviors to a predetermined alternation preparation range during the alternation preparation interval STR before the handover from the automatic driving function to the driver's driving. Therefore, it is possible to end non-driving behaviors as early as possible, urging the driver to begin driving operations smoothly.

[0194] (Fifth Implementation)

[0195] Figures 11-14 The fifth embodiment shown is another variation of the first embodiment. Figure 11 The driver support ECU 37 shown in the fifth embodiment is equipped with a safety alert function unit Z1. The safety alert function unit Z1 is similar to the alertness estimation unit X2, stress estimation unit X3, and passenger monitoring unit X5. The safety alert function unit Z1 functions as an abnormality detection device that automatically detects situations where the driver becomes difficult to drive due to unpredictable reasons such as sudden illness, based on driver status information acquired from the DSM 31. To diagnose a difficult driving state, the safety alert function unit Z1 detects the driver's upper body tilt and head tilt relative to the upper body, thus assessing postural instability. If postural instability persists, the safety alert function unit Z1 determines it as a difficult driving state and outputs the detection results indicating the abnormal state to the communication bus 99. Furthermore, in the following description, "abnormal driver state" refers to a state where the driver's posture has deteriorated to an unsuitable driving posture.

[0196] In addition to the various functional units that confirm the driver's requirements in Level 3 automated driving, HCU10 also includes a driver information acquisition unit X9. The driver information acquisition unit X9 acquires the abnormal state detection results from the safety alert function unit Z1, and monitors the driver's state in parallel with the alertness estimation unit X2, stress estimation unit X3, and passenger monitoring unit X5. If the driver information acquisition unit X9 detects that the driver is experiencing difficulty driving, it requests the information prompt content control unit X8 to report the anomaly. The information prompt content control unit X8 reports the main points of the anomaly detection to the driver and passengers in the passenger compartment via sound and display.

[0197] Here, during Level 3 autonomous driving without the obligation to monitor surroundings, as described above, the driver in the passenger compartment is allowed to engage in behaviors other than driving, such as sleeping, drinking alcohol, or removing the seatbelt. Therefore, during autonomous driving, the driver may, for example, recline the driver's seat back in a relaxed posture. While this posture is not a problem as a driver requirement in autonomous driving, it is considered a poor driving posture. Therefore, the safety alert function Z1 may misidentify the relaxed driver posture during autonomous driving as a state of driver instability leading to posture breakdown and driving difficulty, and output a detection result indicating an abnormal state.

[0198] To address this issue, when the driver information acquisition unit X9 obtains detection results indicating an abnormal state (posture breakdown) during Level 3 autonomous driving, the HCU10 implements a different reporting procedure than during manual driving or Level 2 and below autonomous driving. The following is based on... Figure 12 and refer to Figure 11 An example of the operation of HCU10 and autonomous driving ECU40 in the case of obtaining detection results indicating an abnormal state in Level 3 autonomous driving is explained.

[0199] When the driver's anomaly detection is effective in cooperation with the DSM31-based and safety alert function unit Z1, and a transition to Level 3 automated driving is initiated, the information prompting unit X8 implements event N1 as an information prompting event for the information prompting device 20. Event N1 may begin before or simultaneously with event P1, which reports the transition to Level 3 automated driving. Event N1 can also be implemented in parallel with the information prompting for event P1. During event N1, an information prompt is provided requesting the driver to adopt a posture that allows for DSM31-based driving posture detection.

[0200] If the driver's posture is poor during Level 3 automated driving, the safety alert function unit Z1 outputs a detection result indicating an abnormal state (posture breakdown) of the driver to the driver information acquisition unit X9. Based on the detection result acquired by the driver information acquisition unit X9, the information prompt content control unit X8 implements an information prompt event N2 based on the information prompt device 20. In event N2, even if the driver's posture breakdown is within the permitted range for Level 3 automated driving, posture abnormality information related to the driver's posture breakdown is displayed to passengers, including those who are also passengers. In event N2, at least a report is implemented regarding whether the driver responded to the abnormality and a report stating the reasons for judging the possibility of a driver abnormality.

[0201] The reversal restriction request unit X7a, in addition to having the same functions as the reversal request unit X7 in the first embodiment, also has the function of requesting the autonomous driving ECU 40 to impose functional restrictions on driving control. Based on the detection results indicating an abnormal state, the reversal restriction request unit X7a actually requests the autonomous driving ECU 40 to implement the functional restrictions on driving control simultaneously with event N2. Specifically, the reversal restriction request unit X7a outputs a speed suppression request to the autonomous driving ECU 40. Based on the speed suppression request, the autonomous driving ECU 40 switches to a set speed that is slower (lower) than the normal set speed for Level 3 autonomous driving and attempts to decelerate. Furthermore, even if deceleration control is implemented due to poor driving posture by the driver, Level 3 autonomous driving continues.

[0202] After prompting for a response regarding the presence or absence of an anomaly based on event N2, the information prompt content control unit X8 determines whether there is any user input to the operating device 24. The operating device 24 is an input unit that accepts user operations from the driver and passengers, such as turn signals, levers, and voice input devices. The driver and passengers can input user operations into the operating device 24 to indicate either that the driver has no anomalies or that the driver has an anomaly.

[0203] When the information prompt content control unit X8 receives user input information indicating that the driver's response is not abnormal, it implements information prompt event N3 of the information prompt device 20. In event N3, the information prompt content control unit X8 implements a report indicating the continuation of Level 3 automated driving and a report urging improvement in driving posture. In the information prompt of event N3, the driver is urged to improve their driving posture to a level that avoids misrecognition by the safety alert function unit Z1.

[0204] After the information prompt content control unit X8 prompts for improvement of driving posture in event N3, it refers to the detection results obtained by the driver information acquisition unit X9 to determine whether the driver's driving posture has been improved. If the information prompt content control unit X8 determines that the driver's driving posture has been improved, it outputs a request to cancel speed suppression from the retreat restriction request unit X7a to the automatic driving ECU 40. As a result, the automatic driving ECU 40 ends deceleration control and begins driving at the normal set speed.

[0205] On the other hand, if the driver's driving posture does not improve, the information prompt content control unit X8 repeatedly prompts the driver to improve their driving posture. In this case, the speed suppression control of the automatic driving ECU 40 also continues. Moreover, if the driver does not respond to the improvement request from the information prompt content control unit X8, the retreat restriction request unit X7a determines that the improvement in driving posture cannot be anticipated and requests the automatic driving ECU 40 to switch to MRM control. Based on the determination of the retreat restriction request unit X7a to start MRM control, the information prompt content control unit X8 implements information prompt event M1, prompting the driver with information indicating that driving control has switched to MRM control.

[0206] After prompting for a response based on the presence or absence of an anomaly in event N2, the information prompt content control unit X8, upon receiving user input indicating an abnormal driver operation, or in the absence of a response from the passenger, determines that the driver is experiencing a driving difficulty. Based on the determination made by the information prompt content control unit X8, the retreat restriction request unit X7a requests the automatic driving ECU 40 to switch to MRM control. In cases where the driver experiences a sudden illness, MRM control of the automatic driving ECU 40 is initiated. In this situation, the information prompt content control unit X8 implements information prompt event M1, primarily reporting the switch to MRM control to the passenger.

[0207] and, Figure 11 The passenger monitoring unit X5 in the fifth embodiment shown not only monitors the presence of the driver, but also monitors the presence or absence of passengers, i.e., co-passengers, in the vehicle 1. The passenger monitoring unit X5 acquires detection information from seat sensors 32 installed on the passenger seat or similar surfaces, and determines whether a co-passenger is seated in the passenger seat.

[0208] When the passenger monitoring unit X5 detects the presence of a passenger, the information prompt content control unit X8 retains the prompt regarding abnormal posture information based on the passenger's observation of the driver's state. That is, even if the driver information acquisition unit X9 obtains detection results indicating an abnormal driver state, the information prompt content control unit X8 does not immediately implement the information prompt for event N2. Therefore, the prompt requesting an answer regarding the presence or absence of a driver abnormality becomes standby.

[0209] If, after retaining the abnormal posture information prompt, the Information Prompt Content Control Unit X8 fails to improve the driver's posture, which is a significant factor in the abnormality detection of the Safety Alert Function Unit Z1, it implements an information prompt for event N2, requiring the driver to respond to the abnormality. That is, if the driver information acquisition unit X9 fails to detect any improvement in the driver's posture, the Information Prompt Content Control Unit X8 removes the retention of the abnormal posture information prompt. In this case, the information prompt is implemented not only for the driver but also for the passenger in the passenger seat. Similar to the information prompt in the absence of a passenger, the passenger in the passenger seat is informed of the reasons for the judgment that the driver is likely acting abnormally, along with the response request.

[0210] During Level 3 autonomous driving, when the driver's condition level is set to any one of Levels 1 to 3, the Permissible Behavior Recognition Unit X4, based on the recognition result of such permitted behavior, reduces the threshold for the Safety Alert Function Unit Z1 to determine that the driver is in an abnormal state. Based on the reduction requirement from the Permissible Behavior Recognition Unit X4, the Safety Alert Function Unit Z1 adjusts the threshold that will become the judgment benchmark during Level 3 autonomous driving to make it difficult to determine an abnormal state. As an example, if the Safety Alert Function Unit Z1 determines an abnormal state based on the duration of a posture breakdown for a specified time (e.g., 5 seconds), it implements a threshold adjustment that changes the specified time to a longer time (e.g., 10 seconds). Based on the above, the judgment benchmark of the Safety Alert Function Unit Z1 is optimized for Level 3 autonomous driving. Furthermore, it is preferable to implement the reduction of the judgment benchmark (threshold) within the scope of regulations and guidelines related to safety alert devices in the country or region where the vehicle 1 is used.

[0211] Next, the following is based on Figure 13 as well as Figure 14 The flowchart shown is referenced. Figure 11 as well as Figure 12 This section provides a detailed explanation of the information prompts and processing for various events related to safety alerts, as previously described. It begins with Level 3 autonomous driving, starting via HCU10. Figure 13 as well as Figure 14 The information prompts shown are processed.

[0212] In step S51, it is determined whether the driver information acquisition unit X9 has acquired a detection result indicating an abnormal state of the driver. If no detection result indicating an abnormal state has been acquired, the determination in S51 is repeated, and the system remains on standby for the occurrence of an abnormal state based on posture collapse detection. On the other hand, if it is determined in S51 that a detection result indicating an abnormal state has been acquired, the process proceeds to step S52.

[0213] In S52, the presence or absence of a passenger is determined. If a passenger is determined to be present in S52, proceed to S55. Conversely, if a passenger is determined not to be present in S52, the information from S55 is retained, and proceed to S53. In S53, it is determined whether the safety alert function unit Z1 and the driver information acquisition unit X9 have detected any improvement in the driver's posture. If the driver's posture has improved in S53, return to S51.

[0214] In contrast, in S53, if it is determined that no posture improvement has been achieved, proceed to S54. In S54, it is determined whether the elapsed time since the detection of the driver's posture breakdown (abnormal state) has exceeded a predetermined time. In S54, if it is determined that the prescribed time has elapsed since the detection of the posture breakdown, return to S53. On the other hand, if it is determined in S54 that the prescribed time has elapsed since the posture breakdown was detected, in other words, the state of not achieving driver posture improvement has continued for the prescribed time, proceed to S55. Based on the above, the retention of the posture abnormality information prompt is cancelled.

[0215] In S55, the aforementioned information prompt for event N2 is implemented, namely, prompting for a response regarding the presence or absence of driver abnormality, and providing reasons for judging the possibility of an abnormality. In addition, in S55, an implementation request for deceleration control based on a set speed suppression is output to the automatic driving ECU 40. Here, the information prompt for event N2 and the output of the deceleration control implementation request can also be implemented at different time intervals. As an example, the deceleration control implementation request is output after the information prompt for event N2. In such a variation, the passenger experiences the deceleration control after clearly detecting the driver's abnormal state. Therefore, the passenger is less likely to panic at the change in action accompanying the deceleration control. Alternatively, the information prompt for event N2 can also be implemented after the speed suppression control implementation request.

[0216] In S55, it is determined whether the driver's response is abnormal. In S56, if the driver's response is normal and the driver's condition is not difficult to drive, proceed to S58. On the other hand, if the driver's response is abnormal and the driver's condition is difficult to drive, proceed to S61. And if no response is received from the passenger, proceed to S57.

[0217] In S57, it is determined whether the elapsed time since the start of the response request has exceeded the specified time. If in S57 it is determined that the response request started but the specified time has elapsed, the process returns to S56. On the other hand, if in S57 it is determined that the specified time has elapsed, the process proceeds to S61.

[0218] In S58, the prompting based on the posture abnormality information of event N2, which began in S55, ends, and the information prompting for event N3 described above is implemented. Specifically, in S58, a report indicating the continuation of autonomous driving at level 3 and a report urging improvement of driving posture are implemented sequentially, and the process proceeds to S59. The order of implementing the autonomous driving continuation report and the request for improvement of driving posture can also be appropriately interchanged. In S59, it is determined whether the driver's posture breakdown has been improved. If it is determined in S59 that the driver's posture breakdown has been improved, the process proceeds to S62. In S62, the autonomous driving ECU 40 is requested to release the driving control function restriction requested in S55, i.e., release the deceleration control, and the series of processes ends.

[0219] On the other hand, if in S59 it is determined that the driver's posture deterioration has not been improved, proceed to S60. In S60, it is determined whether the elapsed time since the start of the request to improve the driving posture has exceeded a predetermined time. If in S60 it is determined that the improvement request started but before the predetermined time has elapsed, return to S59. On the other hand, if in S60 it is determined that the predetermined time has elapsed, proceed to S61. In S61, the prompts for posture abnormality information, etc., end, and the aforementioned event W1 information prompt is implemented. Specifically, in S61, a transition from driving control to MRM control is reported. Furthermore, in S61, a transition request to MRM control is output to the automatic driving ECU 40. Through the above, the series of processes ends.

[0220] In the fifth embodiment described so far, the same effect as in the first embodiment is achieved: the driver can obtain information related to the identification result of the permitted behavior, so it is easy to take appropriate action.

[0221] Furthermore, according to the fifth embodiment, the driver information acquisition unit X9 acquires detection results indicating the driver's abnormal state. As a result, even in Level 3 autonomous driving where the driver becomes difficult to drive due to a sudden illness or other reasons, a reliable transition from Level 3 autonomous driving to MRM control can be performed.

[0222] Furthermore, according to the fifth embodiment, when a detection result indicating a breakdown in the driver's posture is obtained during the operation of the Level 3 autonomous driving function, even if the broken driver's posture is within the range permitted for permissible behavior, a posture abnormality message is still sent to the information prompting device. Therefore, even if a posture breakdown caused by a sudden illness meets the driver requirements in autonomous driving, it is possible to inquire about the physical condition of the driver and other passengers. As a result, even in Level 3 autonomous driving, the safety alert system, mainly composed of the safety alert function unit Z1, can operate normally.

[0223] Furthermore, according to the fifth embodiment, if the passenger does not respond to the input requesting an answer regarding whether the driver is experiencing difficulty driving, the prompting of the abnormal posture information ends. Therefore, after determining that the driver is experiencing difficulty driving, it is possible to provide anxious passengers with highly useful information such as the status of the MRM control in an easily understandable manner.

[0224] Furthermore, according to the fifth embodiment, if a detection result indicating a breakdown in the driver's posture is obtained during the operation of the Level 3 autonomous driving function, functional restrictions are imposed on driving control even if the broken driver's posture is within the permitted range of permissible behavior. Therefore, even if a posture breakdown caused by an acute illness meets the driver requirements in autonomous driving, the autonomous driving ECU 40 can preemptively shift to a driving state that is easy to transition to MRM control while continuing autonomous driving. As a result, in cases where the driver does indeed experience difficulty driving, the transition from Level 3 autonomous driving to MRM control can be smoothly implemented.

[0225] Furthermore, according to the fifth embodiment, when the driver information acquisition unit X9 detects an improvement in the driver's posture, the functional restrictions on driving control required by the autonomous driving ECU 40 are lifted. Therefore, when the driver's posture deteriorates, the driving state of the vehicle 1 returns to the normal Level 3 autonomous driving state. Thus, even with an effective vehicle system with a safety alert system, the convenience of the autonomous driving function is not easily compromised.

[0226] Furthermore, according to the fifth embodiment, when the presence of a passenger is known, a prompt containing information about the abnormal posture, including whether the response request indicates an abnormal state, is retained. In this way, the passenger can observe the driver when a passenger is present. Therefore, even without strictly reporting any abnormal posture of the driver, a rapid response to unexpected situations arising from the driver is possible. Based on the above, the situation where passengers become annoyed due to frequent responses can be avoided.

[0227] Furthermore, according to the fifth embodiment, even if the presence of a passenger is known, the retention of the posture abnormality information prompt is deactivated if the driver's posture continues to deteriorate. Therefore, in cases where the driver is seriously ill and the passenger has difficulty responding, the safety alert system can operate normally, just as in cases where there is no passenger, allowing driving control to be transferred to MRM control.

[0228] Furthermore, according to the fifth embodiment, based on the identification results of permitted actions by the driver in Level 3 automated driving, the threshold for determining an abnormal state is lowered from the permitted action identification unit X4 to the safety alert function unit Z1. This adjustment of the threshold makes it less likely that a relaxed posture of the driver in automated driving will be mistaken for a postural breakdown caused by a sudden illness. As a result, it avoids situations where passengers become annoyed by frequent prompts to respond to requests.

[0229] Furthermore, in the fifth embodiment, the passenger monitoring unit X5 is equivalent to a "passenger control unit", the retreat restriction requirement unit X7a is equivalent to a "control requirement unit", and the safety alert function unit Z1 is equivalent to an "anomaly detection device".

[0230] (Other implementation methods)

[0231] The above describes several embodiments, but this disclosure is not limited to these embodiments and can be applied to various embodiments and combinations without departing from the spirit of this disclosure.

[0232] Specifically, as a variation 1, the discrimination viewpoints of the behavior discrimination units X4 and Y6 can also be only one.

[0233] As a variation 2, the permissible behavior discrimination units X4 and Y6 can also use a trained artificial intelligence model constructed primarily of a neural network to discriminate permissible behaviors. For example, the artificial intelligence model can be pre-trained using teacher data consisting of input data containing driving environment information, alertness information, and stress information, and output data containing discrimination results of the corresponding permissible behaviors. With such a trained artificial intelligence model, multiple discrimination points can be comprehensively evaluated without calculating separate provisional discrimination results for multiple discrimination points.

[0234] As a variation 3, the permissible behavior discrimination units X4 and Y6 may also predict the shift in the range of permissible behaviors along the driving path to the destination without relying on a discrimination perspective that can be predicted in the future.

[0235] As a variation 4, the permitted behavior discrimination units X4 and Y6 can also be configured to, for example, predict the shift of the range of permitted behaviors on the driving path up to the destination when the autonomous driving function of autonomous driving level 3 starts working, and then not discriminate the permitted behaviors currently permitted by the driver.

[0236] As a variation 5, at least one of the information prompting device 20 and the control object device CE may include a device (such as a smartphone) that can be brought into the vehicle 1.

[0237] As a variation 6 related to the first and second embodiments, the information prompting device 20 may also include a device that provides tactile information, such as a vibrator that vibrates the driver's seat.

[0238] In variation 7 of the fifth embodiment described above, the prompting of abnormal posture information when the passenger's posture is detected is not implemented. Furthermore, in variation 8 of the fifth embodiment described above, the process of requesting a baseline easing from the permissible behavior recognition unit X4 to the safety alert function unit Z1 is omitted. And, in variation 9 of the fifth embodiment described above, the prompting of abnormal posture information to the driver is omitted. In variation 9, as a cooperative processing mechanism between the safety alert function unit Z1 and HCU10, only the process of requesting a baseline easing from the permissible behavior recognition unit X4 to the safety alert function unit Z1 is implemented.

[0239] In the fifth embodiment described above, the safety alert function unit Z1 of Modification 10 can utilize vehicle movement and driving action information in addition to the driver status information of DSM31 to detect abnormalities in the driver. Furthermore, in Modification 11, the safety alert function unit Z1 is installed in a separate vehicle ECU, different from the driver support ECU 37, and outputs the detection results indicating abnormal driver status to the communication bus 99. In Modification 12, the safety alert function unit Z1 is installed in the control circuit of DSM31 and, together with the driver status information, outputs the detection results diagnosing abnormal status to the communication bus 99.

[0240] As a variation 13, the functions provided by the HCU10 and the autonomous driving ECU40 and 340 can be provided through software and the hardware executing the software, through software alone, through hardware alone, or through a combination thereof. Furthermore, in the case where such functions are provided through electronic circuits as hardware, the functions can also be provided through digital circuits or analog circuits containing a large number of logic circuits.

[0241] As a variation 14, the storage medium (non-transitory tangible computer-readable medium) used to store the program that enables the aforementioned information prompts and function controls can be appropriately modified. For example, the storage medium is not limited to a configuration mounted on the circuit board; it can also be provided in the form of a memory card, inserted into a slot, and electrically connected to the control circuitry of the HCU10 or the autopilot ECU40, 340. Furthermore, the storage medium can also be an optical disc or a hard disk, which serves as the basis for copying the program to the HCU10 or the autopilot ECU40, 340.

[0242] As a variation 15, the HCU10 of the first embodiment may also be replaced by the function restriction unit Y7 provided in the automatic driving ECU 340 of the third embodiment. The information prompt content control unit X8 provided in the HCU10 of the first embodiment may also be replaced by the automatic driving ECU 340 of the third embodiment. That is, information prompts and function restrictions utilizing the permitted behavior recognition units X4 and Y6 can be combined using a single electronic control device.

[0243] As a variation 16, the information display control device may not be mounted on vehicle 1. For example, if the information display control device is HCU10, HCU10 may not be mounted on vehicle 1, but may be fixedly configured outside vehicle 1, or mounted on another vehicle. In this variation 16, the information display device 20 is remotely controlled via communication such as the Internet, road-to-road communication, or vehicle-to-vehicle communication. Similarly, the function control device may not be mounted on vehicle 1.

[0244] As a variation 17, vehicle 1 can be optimized according to the road traffic laws of various countries and regions. Furthermore, the information displayed by the information prompt control device and the functions restricted by the function control device can be optimized according to the road traffic laws of various countries and regions.

[0245] The control unit and methods described herein may also be implemented by a dedicated computer configured with a processor programmed to perform one or more functions embodied in a computer program. Alternatively, the apparatus and methods described herein may be implemented by dedicated hardware logic circuits. Alternatively, the apparatus and methods described herein may be implemented by one or more dedicated computers configured with a combination of a processor executing a computer program and one or more hardware logic circuits. Furthermore, the computer program may also be stored as instructions executable by a computer on a non-transitional tangible recording medium readable by a computer.

Claims

1. An information presentation control device for a vehicle equipped with an automatic driving function capable of performing a driving operation in place of a driver, and controlling an information presentation device that presents information to the driver, comprising: an allowable behavior recognition section that recognizes an allowable behavior in which the driver is allowed to engage in a behavior other than the driving operation that is different from the driving operation, the behavior being capable of being performed by the driver while the automatic driving function is operating; and an information presentation content control section that causes the information presentation device to present information related to a recognition result of the allowable behavior, wherein the allowable behavior recognition section recognizes the allowable behavior based on stress of the driver, and wherein the allowable behavior recognition section recognizes the allowable behavior according to a tilt angle of a driver's seat in the behavior other than the driving operation, and based on a recognition index that establishes a correlation between the behavior other than the driving operation and a degree of allowance of the allowable behavior.

2. The information presentation control device according to claim 1, wherein the allowable behavior recognition section recognizes the allowable behavior according to a posture of the driver in the behavior other than the driving operation, and based on a recognition index that establishes a correlation between the behavior other than the driving operation and a degree of allowance of the allowable behavior.

3. An information presentation control device for a vehicle equipped with an automatic driving function capable of performing a driving operation in place of a driver, and controlling an information presentation device that presents information to the driver, comprising: an allowable behavior recognition section that recognizes an allowable behavior in which the driver is allowed to engage in a behavior other than the driving operation that is different from the driving operation, the behavior being capable of being performed by the driver while the automatic driving function is operating; and an information presentation content control section that causes the information presentation device to present information related to a recognition result of the allowable behavior, wherein the allowable behavior recognition section recognizes the allowable behavior according to a posture of the driver in the behavior other than the driving operation, and based on a recognition index that establishes a correlation between the behavior other than the driving operation and a degree of allowance of the allowable behavior.

4. The information presentation control device according to claim 3, wherein the allowable behavior recognition section recognizes the allowable behavior according to a tilt angle of a driver's seat in the behavior other than the driving operation, and based on a recognition index that establishes a correlation between the behavior other than the driving operation and a degree of allowance of the allowable behavior.

5. An information presentation control device for a vehicle equipped with an automatic driving function capable of performing a driving operation in place of a driver, and controlling an information presentation device that presents information to the driver, comprising: an allowable behavior recognition section that recognizes an allowable behavior in which the driver is allowed to engage in a behavior other than the driving operation that is different from the driving operation, the behavior being capable of being performed by the driver while the automatic driving function is operating; and an information presentation content control section that causes the information presentation device to present information related to a recognition result of the allowable behavior, wherein the allowable behavior recognition section recognizes the allowable behavior according to a posture of the driver in the behavior other than the driving operation, and based on a recognition index that establishes a correlation between the behavior other than the driving operation and a degree of allowance of the allowable behavior. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The above-mentioned allowable behavior discrimination section discriminates the above-mentioned allowable behavior based on the inclination angle of the driver's seat in the above-mentioned behavior other than driving and on the basis of a discrimination index that establishes a correlation between the above-mentioned behavior other than driving and the degree of allowance of the above-mentioned allowable behavior.

6. The information presentation control device according to claim 5, wherein The above-mentioned information presentation content control section causes the above-mentioned information presentation device to present a suggestion for moving the behavior other than the above-mentioned allowable behavior to the above-mentioned allowable behavior in the case where it is estimated that the above-mentioned driver is implementing a behavior other than the above-mentioned allowable behavior in the above-mentioned behavior other than driving.

7. An information presentation control device that is for a vehicle that is provided with an automatic driving function that enables a driver to implement a driving behavior in place of a driver and that controls an information presentation device that presents information to the driver, comprising: an allowable behavior discrimination section that discriminates an allowable behavior that is allowed to be implemented by the driver in a behavior other than driving that is different from the driving behavior while the automatic driving function is operating; and an information presentation content control section that causes the information presentation device to present information related to the discrimination result of the allowable behavior, the above-mentioned information presentation content control section causes the above-mentioned information presentation device to present a suggestion for moving the behavior other than the above-mentioned allowable behavior to the above-mentioned allowable behavior in the case where it is estimated that the above-mentioned driver is implementing a behavior other than the above-mentioned allowable behavior in the above-mentioned behavior other than driving, the above-mentioned information presentation content control section causes the above-mentioned information presentation device to present information related to the above-mentioned discrimination result of the allowable behavior on the basis of a restriction time interval that separates the frequency of restriction information presentation in the case where it is estimated that the above-mentioned driver is implementing the above-mentioned allowable behavior.

8. The information presentation control device according to claim 7, wherein it further comprises a driver information acquisition section that acquires a detection result that indicates an abnormal state of the driver from an abnormality detection device that detects a state in which the driver becomes difficult to drive.

9. The information presentation control device according to claim 8, wherein the above-mentioned information presentation content control section causes the above-mentioned information presentation device to present posture abnormality information related to the collapse of the driver's posture even if the collapsed posture of the driver is within a range of postures that are allowed as the above-mentioned allowable behavior in the case where the above-mentioned detection result that indicates the collapse of the driver's posture is acquired while the automatic driving function is operating.

10. The information presentation control device according to claim 9, wherein it further comprises a passenger grasping section that grasps the presence or absence of a passenger of the vehicle other than the driver, i.e., a co-rider, the above-mentioned information presentation content control section retains the presentation of the posture abnormality information in the case where the presence of the co-rider is grasped by the passenger grasping section.

11. An information presentation control device that is for a vehicle that is provided with an automatic driving function that enables a driver to implement a driving behavior in place of a driver and that controls an information presentation device that presents information to the driver, comprising: an allowable behavior discrimination unit that discriminates, in a behavior other than a driving behavior that can be performed by the driver at the time of operation of the automatic driving function, an allowable behavior that is performed by the driver; a passenger grasping unit that grasps the presence or absence of a passenger of the vehicle other than the driver, i.e., a co-rider; an information presentation content control unit that causes the information presentation device to present information related to a discrimination result of the allowable behavior; an abnormality state grasping unit that grasps an abnormality state of the driver from an abnormality detection device that detects a state in which the driver becomes difficult to drive, the information presentation content control unit causes the information presentation device to present posture abnormality information related to a collapse of the posture of the driver, even if the collapsed posture of the driver is a posture within a range that is allowed as the allowable behavior, in a case where the detection result indicating the collapse of the posture of the driver is obtained at the time of operation of the automatic driving function, the information presentation content control unit retains presentation of the posture abnormality information in a case where the passenger grasping unit grasps the presence of the co-rider.

12. The information presentation control device according to claim 11, wherein the information presentation content control unit cancels the retention of presentation of the posture abnormality information in a case where the state in which the posture of the driver is improved continues without being grasped by the driver information grasping unit.

13. The information presentation control device according to claim 11 or 12, wherein the information presentation content control unit requests input of an answer as to whether the state in which the driver is difficult to drive using presentation of the posture abnormality information, and ends presentation of the posture abnormality information in a case where there is an answer to the request for the input.

14. The information presentation control device according to claim 11 or 12, wherein the information presentation control device further includes a restriction request unit that requests imposition of a function restriction on travel control based on the automatic driving function, even if the collapsed posture of the driver is a posture within a range that is allowed as the allowable behavior, in a case where the detection result indicating the collapse of the posture of the driver is obtained at the time of operation of the automatic driving function.

15. The information presentation control device according to claim 14, wherein the restriction request unit requests cancellation of the function restriction on the travel control of the automatic driving function in a case where the posture of the driver is improved is grasped by the driver information grasping unit.

16. The information presentation control device according to claim 11 or 12, wherein the allowable behavior discrimination unit requests relaxation of a criterion for determining the collapse of the posture of the driver based on the discrimination result of the allowable behavior.

17. An information presentation control device that is for a vehicle that is provided with an automatic driving function that can perform a driving behavior in place of a driver, and that controls an information presentation device that presents information to the driver, the information presentation control device comprising: ​ an allowable behavior recognition unit that recognizes an allowable behavior that is allowed to be performed by the driver among behaviors other than the driving behavior that can be performed by the driver during operation of the automatic driving function, an information presentation content control unit that causes the information presentation device to present information related to the recognition result of the allowable behavior, and an abnormality state information acquisition unit that acquires, from an abnormality detection device that detects an abnormality state of the driver, information indicating the abnormality state of the driver, the allowable behavior recognition unit relaxes a criterion for determining collapse of the posture of the driver based on the recognition result of the allowable behavior.

18. The information presentation control device according to claim 17, wherein the information presentation content control unit causes the information presentation device to present the information related to the recognition result of the allowable behavior on the basis of a restriction time interval that restricts a frequency of information presentation, in a case where it is estimated that the driver performs the allowable behavior among the behaviors other than the driving behavior.

19. An information presentation control device that is used for a vehicle that is provided with an automatic driving function that enables a driving behavior to be performed in place of a driver, and that controls an information presentation device that presents information to the driver, the information presentation control device comprising: an allowable behavior recognition unit that recognizes an allowable behavior that is allowed to be performed by the driver among behaviors other than the driving behavior that can be performed by the driver during operation of the automatic driving function; and an information presentation content control unit that causes the information presentation device to present information related to a recognition result of the allowable behavior, the information presentation content control unit causes the information presentation device to present the information related to the recognition result of the allowable behavior on the basis of a restriction time interval that restricts a frequency of information presentation, in a case where it is estimated that the driver performs the allowable behavior among the behaviors other than the driving behavior, the information presentation content control unit causes the information presentation device to present information indicating a change in the range of the allowable behavior, in a case where the range of the allowable behavior recognized by the allowable behavior recognition unit changes.

20. The information presentation control device according to claim 19, wherein the allowable behavior recognition unit is able to update the recognition result of the allowable behavior based on the latest information during operation of the automatic driving function, the information presentation content control unit increases the information presentation by the information presentation device more as the amount of change in the range of the allowable behavior increases, in a case where the range of the allowable behavior recognized by the allowable behavior recognition unit changes.

21. The information presentation control device according to claim 19, wherein the allowable behavior recognition unit estimates a predetermined continuous operation interval during which the automatic driving function continuously operates, and a point within the continuous operation interval at which the degree of allowance of the recognition result of the allowable behavior is lowest, the information presentation content control unit causes the information presentation device to present information indicating the continuous operation interval and the point within the continuous operation interval, in a case where the degree of allowance of the recognition result of the allowable behavior is lowest at the point within the continuous operation interval. The information presentation content control section causes the information presentation device to present information indicating the permitted behavior among the behaviors other than the driving, in a case where the information presentation device is estimated to be used for the permitted behavior.

22. The information presentation control device according to any one of claims 19 to 21, wherein The information presentation content control section causes the information presentation device to present information indicating the permitted behavior among the behaviors other than the driving.

23. The information presentation control device according to any one of claims 19 to 21, wherein The information presentation content control section causes the information presentation device to present information indicating a behavior other than the permitted behavior among the behaviors other than the driving.

24. The information presentation control device according to any one of claims 19 to 21, wherein The permitted behavior recognition section recognizes the permitted behavior on the basis of a travel environment.

25. The information presentation control device according to claim 24, wherein The permitted behavior recognition section recognizes the permitted behavior on the basis of a shape of a road on which the vehicle travels, as the travel environment.

26. The information presentation control device according to claim 25, wherein The permitted behavior recognition section narrows down a range of the permitted behavior, in a case where the road on which the vehicle travels is a curved shape and a length of a curved section exceeds a prescribed distance.

27. The information presentation control device according to any one of claims 19 to 21, wherein The permitted behavior recognition section recognizes the permitted behavior on the basis of a travel scenario of the vehicle, as the travel environment.

28. The information presentation control device according to any one of claims 19 to 21, wherein The permitted behavior recognition section recognizes the permitted behavior on the basis of a degree of wakefulness of the driver.

29. The information presentation control device according to any one of claims 19 to 21, wherein The permitted behavior recognition section sets a plurality of recognition viewpoints, tentatively recognizes separate permitted behaviors for each of the recognition viewpoints, and definitely recognizes as the permitted behavior a behavior that is commonly recognized as the permitted behavior among the separate permitted behaviors for each of the recognition viewpoints.

30. The information presentation control device according to any one of claims 19 to 21, wherein The information presentation content control section requests release of the restriction, in a case where the function restriction for realizing the permitted behavior using the information presentation device is imposed on the information presentation device.

31. The information presentation control device according to any one of claims 19 to 21, wherein The information presentation device includes a device that presents visual information and a device that presents audible information, The information presentation content control section selects a device for information presentation from the device that presents the visual information and the device that presents the audible information, on the basis of an estimated behavior other than the driving that is currently implemented by the driver.

32. The information presentation control device according to any one of claims 19 to 21, wherein The information presentation content control section causes the information presentation device to present a reason why the behavior other than the permitted behavior among the behaviors other than the driving should not be implemented, in a case where the behavior other than the permitted behavior is estimated to be implemented by the driver.

33. The information presentation control device according to any one of claims 19 to 21, wherein the information presentation content control section causes the information presentation device to perform an alarm to the driver in a case where it is estimated that the driver continues to perform an action other than the allowed action among the actions other than the driving.

34. The information presentation control device according to claim 33, wherein a retreat request section is further provided, which requests the vehicle to forcibly retreat in a case where it is estimated that the driver further continues to perform an action other than the allowed action among the actions other than the driving after the start of the implementation of the alarm.

35. The information presentation control device according to claim 33 or 34, wherein the information presentation content control section causes the information presentation device to present a warning of a driving alternation to end the automatic driving function in conjunction with the implementation of the alarm.

36. The information presentation control device according to any one of claims 19 to 21, wherein the information presentation content control section causes the information presentation device to present a warning of a driving alternation to end the automatic driving function in a case where it is grasped that a predetermined end of a permission area that permits the actions other than the driving is reached.

37. The information presentation control device according to claim 35, wherein the allowed action discrimination section narrows down the range of the allowed action to a prescribed alternation preparation range in an alternation preparation interval before the driving alternation from the automatic driving function to the driving by the driver is performed.

Images