Control device and control program

Abstract

To provide a control device and the like that do not cause anxiety to a driver even when an emergency vehicle approaches during automatic driving.SOLUTION: A HCU is a control device used in a vehicle capable of performing automatic driving at a level 3 where a driver has no obligation to monitor surroundings. The HCU determines whether or not a state where the driver is allowed to perform a second task is continued when it is detected that an emergency vehicle approaches the own vehicle during an automatic driving period where the automatic driving at the level 3 is performed. The HCU limits a display of contents provided to the driver if it is determined that the allowed status of the second task is not continued.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The disclosure of this specification relates to a control device and a control program used in an autonomous vehicle. [Background technology]

[0002] Patent Document 1 describes that while the vehicle is being driven autonomously by an autonomous driving ECU, the driver may enter a second task state in which he or she operates a smartphone or watches a television broadcast. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-107502 A Summary of the Invention [Problem to be solved by the invention]

[0004] As in Patent Document 1, there is a possibility that an emergency vehicle may approach the vehicle during automatic driving. In such a situation, it is difficult for a driver in a second task state to notice the approach of the emergency vehicle. As a result, there is a risk that the driver may feel uneasy about how to respond to the emergency vehicle.

[0005] The present disclosure aims to provide a control device and a control program that are unlikely to cause anxiety to the driver even if an emergency vehicle approaches during automated driving. [Means for solving the problem]

[0006] In order to achieve the above object, one disclosed embodiment comprises: A control device used in a vehicle (Am) capable of performing autonomous driving without the driver being obligated to monitor the surroundings, the control device includes a continuation determination unit (72) that determines whether or not a state in which the driver is permitted to perform a specific action other than driving will continue when an approach of an emergency vehicle (Qs) to the vehicle, which is the vehicle, is detected during an autonomous driving period in which autonomous driving is performed, and a provision restriction unit (74) that restricts the display of content (CTV) provided to the driver when the continuation determination unit determines that the permitted state of the specific action will not continue, wherein the continuation determination unit further determines whether or not autonomous driving will continue when an approach of an emergency vehicle to the vehicle is detected, and the provision restriction unit restricts the display of the content when the continuation determination unit determines that autonomous driving will not continue more than when it is determined that autonomous driving will continue, and at least one of whether or not to continue the specific action and whether or not to continue autonomous driving is determined based on the relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling, and when the vehicle travels in an adjacent lane (Lna) adjacent to the emergency vehicle lane, the continuation of the specific action is restricted and continuation of autonomous driving is permitted. Also, one disclosed aspect is a control device used in a vehicle (Am) capable of performing autonomous driving without the driver being obligated to monitor the surroundings, the control device including a continuation determination unit (72) that determines whether or not a state in which the driver is permitted to perform a specific action other than driving continues when an approach of an emergency vehicle (Qs) to the vehicle, which is the vehicle, is detected during an autonomous driving period in which autonomous driving is performed, and a provision restriction unit (74) that restricts the display of content (CTV) provided to the driver when the continuation determination unit determines that the permitted state of the specific action does not continue. When the approach of an emergency vehicle to the vehicle is detected, the continuation determination unit When the continuation determination unit determines that autonomous driving will not continue, the provision restriction unit further determines whether or not autonomous driving will continue, and when the continuation determination unit determines that autonomous driving will not continue, the provision restriction unit restricts the display of the content more than when it is determined that autonomous driving will continue, and at least one of whether or not to continue the specific action and whether or not to continue autonomous driving is determined based on the condition of the road on which the vehicle is traveling, and when the vehicle lane in which the vehicle is traveling among the lanes included in the road is congested and a non-congested lane (Lnp) in which emergency vehicles can travel is present on the road, the continuation of the specific action is restricted and the continuation of autonomous driving is permitted.

[0007] Also disclosed is one aspect of the present invention, A control program used in a vehicle (Am) capable of performing autonomous driving without the driver being obligated to monitor the surroundings, the control program including at least one processing unit (11) that, when an approach of an emergency vehicle (Qs) to the vehicle is detected during an autonomous driving period in which autonomous driving is performed, determines whether or not a state in which the driver is permitted to perform a specific action other than driving continues (S155), and when it is determined that the state in which the specific action is permitted does not continue, restricts the display of content (CTV) provided to the driver (S158, S159), and when the approach of an emergency vehicle to the vehicle is detected, If the emergency vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, the control program executes processing including: further determining whether or not to continue autonomous driving if the emergency vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane; and restricting the continuation of the specific action and permitting the continuation of autonomous driving if the emergency vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane. One disclosed aspect is a control program used in a vehicle (Am) capable of performing autonomous driving without the driver being required to monitor the surroundings, in which at least one processing unit (11) determines whether or not a state in which the driver is permitted to perform a specific action other than driving continues (S155) when an approach of an emergency vehicle (Qs) to the vehicle, which is a host vehicle, is detected during an autonomous driving period in which autonomous driving is performed, and when it is determined that the state in which the specific action is permitted does not continue, restricts the display of content (CTV) provided to the driver (S158, S159), and when the approach of an emergency vehicle to the host vehicle is detected, and then further determining whether or not autonomous driving will continue (S157), and if it is determined that autonomous driving will not continue, restricting the display of content more than when it is determined that autonomous driving will continue, and determining whether or not at least one of whether or not the specific action will continue and whether or not autonomous driving will continue based on the condition of the road on which the vehicle is traveling, and if the lane in which the vehicle is traveling among the lanes included in the road is congested and there is a non-congested lane (Lnp) in which emergency vehicles can travel on the road, restricting the continuation of the specific action and allowing the continuation of autonomous driving.

[0008] In these aspects, when the driver is performing a specific action other than driving during an automatic driving period in which there is no obligation to monitor the surroundings, if it is determined that the permission state for the specific action will not continue due to the approach of an emergency vehicle to the vehicle, the display of the provided content is restricted. Therefore, even if the driver is performing a specific action, the restriction on the display of the content makes it easier for the driver to notice the approach of an emergency vehicle. As a result, it is possible to make it difficult for the driver to feel uneasy even if an emergency vehicle approaches during automatic driving.

[0009] In addition, one disclosed aspect is a control device used in a vehicle (Am) capable of performing autonomous driving without the driver being required to monitor the surroundings, the control device including an approach determination unit (61) that determines whether or not an emergency vehicle (Qs) is approaching a host vehicle during an autonomous driving period in which autonomous driving is performed, and a continuation determination unit (62) that determines whether or not autonomous driving should be continued and whether or not a state in which the driver is permitted to perform a specific action other than driving should be continued based on the relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling, when the approach determination unit determines that an emergency vehicle is approaching. When the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, the continuation of the autonomous driving is restricted, and the presence of the emergency vehicle is notified to the driver. It is said to be a control device. Also, one disclosed aspect is a control device used in a vehicle (Am) capable of autonomous driving without the driver being obligated to monitor the surroundings, which includes an approach determination unit (61) that determines whether or not an emergency vehicle (Qs) is approaching the vehicle, which is the vehicle, during an autonomous driving period in which autonomous driving is performed, and a continuation determination unit (62) that determines whether or not autonomous driving should be continued and whether or not a state in which the driver is permitted to perform specific actions other than driving should be continued when the approach determination unit determines that an emergency vehicle is approaching, and at least one of whether or not the specific action should be continued and whether or not autonomous driving should be continued is determined based on the relative position of the emergency vehicle lane (Lnq) in which the emergency vehicle is traveling, and when the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, the continuation of the specific action is restricted and continuation of autonomous driving is permitted. Also, one disclosed aspect is a control device used in a vehicle (Am) capable of autonomous driving without the driver being obligated to monitor the surroundings, which includes an approach determination unit (61) that determines whether or not an emergency vehicle (Qs) is approaching the vehicle during an autonomous driving period in which autonomous driving is performed, and a continuation determination unit (62) that determines whether or not autonomous driving should be continued and whether or not a state in which the driver is permitted to perform specific actions other than driving should be continued when the approach determination unit determines that an emergency vehicle is approaching, and at least one of whether or not the specific action should be continued and whether or not autonomous driving should be continued is determined based on the conditions of the road on which the vehicle is traveling, and when the lane on which the vehicle is traveling among the lanes included in the road is congested and there is a non-congested lane (Lnp) on the road in which emergency vehicles can travel, the continuation of the specific action is restricted and the continuation of autonomous driving is permitted.

[0010] One disclosed aspect is a control program used in a vehicle (Am) capable of performing autonomous driving without the driver being required to monitor the surroundings, which includes at least one processing unit (51) that determines whether or not an emergency vehicle (Qs) is approaching the vehicle during an autonomous driving period in which autonomous driving is performed (S101, S131), and when it is determined that an emergency vehicle is approaching, determines whether or not autonomous driving should be continued and whether or not a state in which the driver is permitted to perform specific actions other than driving should be continued based on the relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling. death (S106~S109, S136~S139), When the vehicle is traveling in an adjacent lane (Lna) adjacent to an emergency vehicle lane, the continuation of autonomous driving is restricted and the presence of an emergency vehicle is notified to the driver. The control program is a program for executing a process including the above. Another disclosed aspect is a control program used in a vehicle (Am) capable of autonomous driving without the driver being obligated to monitor the surroundings, which causes at least one processing unit (51) to execute processing including: determining whether or not an emergency vehicle (Qs) is approaching the vehicle, which is the vehicle, during an autonomous driving period in which autonomous driving is performed (S101, S131); if it is determined that an emergency vehicle is approaching, determining whether or not autonomous driving should be continued and whether or not a state in which the driver is permitted to perform specific actions other than driving should be continued (S106 to S109, S136 to S139); determining whether or not to continue the specific action and whether or not to continue autonomous driving based on the relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling; and, if the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, restricting the continuation of the specific action and permitting the continuation of autonomous driving. Another disclosed aspect is a control program used in a vehicle (Am) capable of performing autonomous driving without the driver being required to monitor the surroundings, which causes at least one processing unit (51) to execute processing including: determining whether or not an emergency vehicle (Qs) is approaching the vehicle, which is the vehicle, during an autonomous driving period in which autonomous driving is performed (S101, S131); if it is determined that an emergency vehicle is approaching, determining whether or not autonomous driving should be continued and whether or not a state in which the driver is permitted to perform specific actions other than driving should be continued (S106 to S109, S136 to S139); determining whether or not to continue the specific action and whether or not to continue autonomous driving based on the conditions of the road on which the vehicle is traveling; and, if the lane on which the vehicle is traveling, among the lanes included in the road, is congested and a non-congested lane (Lnp) in which emergency vehicles can travel, restricting the continuation of the specific action and allowing the continuation of autonomous driving.

[0011] In these aspects, when the driver is performing a specific action other than driving during an automatic driving period in which there is no obligation to monitor the surroundings, if it is determined that an emergency vehicle is approaching the vehicle, it is determined whether to continue automatic driving and whether to continue the state in which the specific action is permitted. Therefore, even if the driver is performing a specific action, the driver is more likely to notice the approach of an emergency vehicle because the automatic driving or the permission state for the specific action is not continued. As a result of the above, it is possible to make the driver less likely to feel uneasy even if an emergency vehicle approaches during automatic driving.

[0012] The reference numbers in parentheses above and in the claims merely indicate an example of a corresponding relationship with a specific configuration in an embodiment described later, and do not in any way limit the technical scope. [Brief description of the drawings]

[0013] [Figure 1] 1 is a diagram showing an overall view of an in-vehicle network according to a first embodiment of the present disclosure. [Diagram 2] FIG. 13 is a diagram showing an example of an interface layout around the driver's seat. [Diagram 3] FIG. 2 is a diagram showing an example of a functional unit constructed in an autonomous driving ECU and an HCU together with related configurations. [Figure 4] FIG. 13 is a diagram showing an example of how a CID is displayed in a second task mode. [Diagram 5] FIG. 13 is a diagram showing an example of a CID display when continuation of a second task is permitted in a scene where an emergency vehicle is approaching. [Figure 6] FIG. 13 is a diagram showing an example of a CID display when continuation of the second task is not permitted in a scene where an emergency vehicle is approaching. [Figure 7] FIG. 13 is a diagram showing an example of a CID display when level 3 autonomous driving is restricted in a scene where an emergency vehicle is approaching. [Figure 8] FIG. 1 is a diagram showing an example of a scene in which an emergency vehicle is approaching. [Figure 9] FIG. 1 is a diagram showing an example of a scene in which an approaching emergency vehicle moves away from the vehicle; [Figure 10] FIG. 1 is a diagram showing an example of a scene in which an approaching emergency vehicle moves away from the vehicle; [Figure 11] FIG. 1 is a diagram showing an example of a scene in which an emergency vehicle is approaching. [Figure 12] FIG. 1 is a diagram showing an example of a scene in which an emergency vehicle is approaching. [Figure 13] FIG. 1 is a diagram showing an example of a scene in which an emergency vehicle is approaching. [Figure 14] FIG. 1 is a diagram showing an example of a scene in which an emergency vehicle approaches during a traffic jam. [Figure 15] FIG. 1 is a diagram showing an example of a scene in which an emergency vehicle approaches during a traffic jam. [Figure 16] FIG. 1 is a diagram showing an example of a scene in which the host vehicle can evacuate from the planned driving route of an approaching emergency vehicle. [Figure 17]17 is a diagram showing an example of a CID display in the scene shown in FIG. 16, which is a modified example of the display shown in FIG. [Figure 18] FIG. 1 is a diagram showing an example of a scene in which an emergency vehicle traveling in an oncoming lane approaches from the front. [Figure 19] FIG. 20 is a diagram showing an example of how a CID is displayed in the scene shown in FIG. 18. [Figure 20] 13 is a flowchart showing details of a continuation decision process for determining whether or not to continue level 3 autonomous driving in a scene where an emergency vehicle is approaching. [Figure 21] 13 is a flowchart showing details of a continuation determination process for determining whether or not to continue the second task in a scene where an emergency vehicle is approaching. [Figure 22] 13 is a flowchart showing details of a display control process for switching the display mode of a CID based on a determination of whether or not to continue the autonomous driving and the second task. [Figure 23] 13 is a flowchart showing details of a display control process for returning the display mode of the CID to the state before switching. [Figure 24] 13 is a flowchart showing details of a sleep monitoring process performed by an HCU according to a second embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, multiple embodiments of the present disclosure will be described based on the drawings. In addition, by assigning the same reference numerals to corresponding components in each embodiment, duplicated descriptions may be omitted. When only a part of the configuration is described in each embodiment, the configuration of the other embodiment described above can be applied to the other parts of the configuration. In addition to the combination of configurations explicitly stated in the description of each embodiment, configurations of multiple embodiments can be partially combined even if not explicitly stated, as long as there is no particular problem with the combination. Furthermore, combinations of configurations described in multiple embodiments and modified examples that are not explicitly stated are also considered to be disclosed by the following description.

[0015] First embodiment The functions of the control device according to the first embodiment of the present disclosure are realized by an automatic driving ECU (Electronic Control Unit) 50b and an HCU (Human Machine Interface Control Unit) 100. The automatic driving ECU 50b shown in Fig. 1 constitutes an automatic driving system 50 used in a vehicle Am together with a driving assistance ECU 50a and the like. By installing the automatic driving system 50, the vehicle Am becomes an automatic driving vehicle equipped with an automatic driving function.

[0016] 1 to 3, the HCU 100 constitutes an HMI (Human Machine Interface) system used in the vehicle Am together with a plurality of display devices, an audio device 24, an operation device 26, etc. The HMI system has an input interface function that accepts operations by an occupant (e.g., the driver) of the vehicle Am, and an output interface function that presents information to the driver.

[0017] The autonomous driving ECU 50b and the HCU 100 are communicatively connected to a communication bus 99 of an in-vehicle network 1 mounted on the vehicle Am. The autonomous driving ECU 50b and the HCU 100 are each one of a plurality of nodes provided in the in-vehicle network 1. The driver monitor 29, the surroundings monitoring sensor 30, the locator 35, the vehicle-to-vehicle communication device 37, the road-to-vehicle communication device 38, the cruise control ECU 40, the driving assistance ECU 50a, and the like are further connected to the communication bus 99. These nodes connected to the communication bus 99 of the in-vehicle network 1 can communicate with each other. Specific nodes among these devices and ECUs may be directly electrically connected to each other and can communicate without going through the communication bus 99.

[0018] The driver monitor 29 includes a near-infrared light source, a near-infrared camera, and a control unit for controlling them. The driver monitor 29 is installed, for example, on the upper surface of the steering column or the upper surface of the instrument panel 9 with the near-infrared camera facing the headrest of the driver's seat. The near-infrared camera is configured integrally with the meter display 21 or the center information display (hereinafter, CID) 22 described later, and may be embedded in either screen.

[0019] The driver monitor 29 uses a near-infrared camera to capture an image of the driver's head illuminated with near-infrared light from a near-infrared light source. The image captured by the near-infrared camera is analyzed by a control unit. The control unit extracts information such as the position and line of sight of the driver from the captured image, and provides the extracted driver's state information to the HCU 100 and the like via a communication bus 99.

[0020] The periphery monitoring sensor 30 is an autonomous sensor that monitors the environment surrounding the vehicle Am. The periphery monitoring sensor 30 can detect predefined moving objects and stationary objects within a detection range around the vehicle. The periphery monitoring sensor 30 can at least detect a vehicle ahead, a vehicle behind, and vehicles on the front and rear sides of the vehicle traveling around the vehicle. The periphery monitoring sensor 30 provides detection information of objects around the vehicle to the driving assistance ECU 50a, the autonomous driving ECU 50b, and the like via a communication bus 99.

[0021] The perimeter monitoring sensor 30 includes, for example, a camera unit 31, a millimeter wave radar 32, and a microphone 33. The perimeter monitoring sensor 30 may further include a detection configuration such as a lidar and a sonar.

[0022] The camera unit 31 includes a plurality of camera modules mounted on the vehicle Am so as to be able to capture images of the front range, side range, rear range, etc. of the vehicle Am. Each camera module may include a monocular camera or a compound eye camera. The camera unit 31 outputs at least one of the image data captured by each camera module of the surroundings of the vehicle and the analysis result of the image data as detection information.

[0023] The millimeter wave radar 32 emits millimeter waves or quasi-millimeter waves toward the surroundings of the vehicle. The millimeter wave radar 32 outputs detection information generated by a process of receiving waves reflected by moving objects, stationary objects, etc.

[0024] The microphone 33 is a sound collector that collects environmental sounds around the vehicle Am. When an emergency vehicle Qs (see FIG. 8, etc.) is approaching the vehicle Am, the microphone 33 collects environmental sounds including a specific siren sound sounded by the emergency vehicle Qs and a voice from a loudspeaker. The microphone 33 outputs at least one of the environmental sounds or information that a siren sound has been extracted from the environmental sounds as detection information.

[0025] Locator 35 includes a GNSS (Global Navigation Satellite System) receiver, an inertial sensor, etc. Locator 35 sequentially determines the position and traveling direction of vehicle Am by combining the positioning signal received by the GNSS receiver, the measurement results of the inertial sensor, and the vehicle speed information output to communication bus 99. Locator 35 sequentially outputs position information and direction information of vehicle Am based on the positioning results to communication bus 99 as locator information.

[0026] Locator 35 further includes map database 36. Map database 36 is mainly composed of a large-capacity storage medium storing a large amount of three-dimensional map data and two-dimensional map data. The three-dimensional map data is so-called high-precision map data, and includes information necessary for advanced driving assistance and automated driving, such as three-dimensional shape information of roads and detailed information on each lane. Locator 35 reads map data of the area around the current position from map database 36, and provides it to driving assistance ECU 50a, automated driving ECU 50b, etc., together with locator information.

[0027] The vehicle-to-vehicle communication device 37 is mainly composed of a communication module for wireless communication with on-board devices mounted on other vehicles. The vehicle-to-vehicle communication device 37 can directly or indirectly transmit and receive information with on-board devices of other vehicles, and acquire information such as the position and speed of the other vehicles. When an emergency vehicle Qs (see FIG. 8, etc.) is approaching the vehicle Am, the vehicle-to-vehicle communication device 37 can receive information transmitted from the on-board device of the emergency vehicle Qs, i.e., the position and moving speed of the emergency vehicle Qs.

[0028] The road-to-vehicle communication device 38 is mainly composed of a communication module for receiving information distributed from a roadside unit installed at the side of the road. The road-to-vehicle communication device 38 receives traffic signal information, traffic congestion information, regulation information, and the like for the roads near the roadside unit by means of FM multiplex broadcasting, radio wave beacons, optical beacons, and the like. In addition, the road-to-vehicle communication device 38 can receive information about the approach of an emergency vehicle Qs (see FIG. 8, etc.) from the roadside unit.

[0029] The driving control ECU 40 is an electronic control device mainly including a microcontroller. The driving control ECU 40 has at least the functions of a brake control ECU, a drive control ECU, and a steering control ECU. The driving control ECU 40 continuously performs brake force control for each wheel, output control of the in-vehicle power source, and steering angle control based on one of an operation command based on the driver's driving operation, a control command from the driving assistance ECU 50a, and a control command from the automatic driving ECU 50b. In addition, the driving control ECU 40 generates vehicle speed information indicating the current driving speed of the vehicle Am based on detection signals from wheel speed sensors provided at the hubs of each wheel, and outputs the information to the communication bus 99 sequentially.

[0030] The driving assistance ECU 50a is an in-vehicle ECU that realizes a driving assistance function that assists the driver in driving operations. The driving assistance ECU 50a enables advanced driving assistance or partial autonomous driving control at about level 2, which requires the driver to monitor the surroundings, among the autonomous driving levels specified by the Society of Automotive Engineers.

[0031] The driving assistance ECU 50a is a computer mainly including a control circuit equipped with a processing unit, a RAM, a storage unit, an input / output interface, and a bus connecting these. The driving assistance ECU 50a has a plurality of functional units that realize advanced driving assistance by executing a program by the processing unit. Specifically, the driving assistance ECU 50a has an ACC (Adaptive Cruise Control) functional unit, an LTA (Lane Tracing Assist) functional unit, and an LCA (Lane Change Assist) functional unit.

[0032] In contrast to the driving assistance ECU 50a, the autonomous driving ECU 50b is an in-vehicle ECU that realizes an autonomous driving function that can take over driving operations from the driver. The autonomous driving ECU 50b enables autonomous driving of level 3 or higher, where the system is the main controller, only within a pre-set limited area (Operational Design Domain, ODD). The autonomous driving ECU 50b enables eyes-off autonomous driving, which does not require the driver to visually monitor the surroundings of the vehicle. The autonomous driving ECU 50b may also be capable of realizing an autonomous driving function of level 4 or higher.

[0033] The autonomous driving ECU 50b is a computer that mainly includes a control circuit including a processing unit 51, a RAM 52, a storage unit 53, an input / output interface 54, and a bus connecting these, etc. The autonomous driving ECU 50b has a higher computing capacity than the driving assistance ECU 50a, and can at least perform driving controls equivalent to ACC, LTA, and LCA.

[0034] The processing unit 51 is hardware for arithmetic processing coupled to the RAM 52. The processing unit 51 includes at least one arithmetic core such as a central processing unit (CPU) and a graphics processing unit (GPU). The processing unit 51 may further include an IP core having a field-programmable gate array (FPGA), a neural network processing unit (NPU), and other dedicated functions. The processing unit 51 accesses the RAM 52 to execute various processes for implementing the control method of the present disclosure. The storage unit 53 includes a non-volatile storage medium. The storage unit 53 stores various programs (such as an automatic driving control program) executed by the processing unit 51.

[0035] The autonomous driving ECU 50b has an environment recognition unit 61, an action judgment unit 62, and an operation execution unit 63 as multiple functional units that realize autonomous driving without the driver being required to monitor the surroundings by executing a program (autonomous driving control program) by the processing unit 51.

[0036] The environment recognition unit 61 recognizes the driving environment of the vehicle Am based on the locator information and map data acquired from the locator 35 and the detection information acquired from the perimeter monitoring sensor 30. Specifically, the environment recognition unit 61 grasps the position of the host vehicle lane Lns (see FIG. 8) in which the host vehicle is driving among multiple lanes, the lane shape of the host vehicle lane Lns, and the relative positions and relative speeds of other vehicles around the host vehicle. The position of the host vehicle lane Lns may be specified by the locator 35. Furthermore, the recognition result of the driving environment by the environment recognition unit 61 may be provided to the HCU 100 as recognition information.

[0037] The behavior determination unit 62 generates a planned driving line for the vehicle Am to travel on the basis of the recognition result of the driving environment by the environment recognition unit 61. When a destination is set in the navigation system installed in the vehicle Am, the behavior determination unit 62 refers to route information generated by the navigation device and sequentially generates a planned driving line for heading to the destination.

[0038] The operation execution unit 63, in cooperation with the cruise control ECU 40, executes acceleration / deceleration control, steering control, and the like of the vehicle Am in accordance with the planned driving line generated by the action determination unit 62. Specifically, the operation execution unit 63 generates control commands based on the planned driving line and outputs them to the cruise control ECU 40 one by one.

[0039] Next, the multiple display devices, the audio device 24, the operation device 26, and the HCU 100 included in the HMI system will be described in detail in turn.

[0040] The multiple display devices include a meter display 21, a CID 22, a head-up display (hereinafter, HUD) 23, and the like. The multiple display devices may further include displays EMB, EML, and EMR of the electronic mirror system (see FIG. 2). The meter display 21, the CID 22, and the HUD 23 present information to the driver through his / her vision.

[0041] The meter display 21 and the CID 22 are mainly configured to include, for example, a liquid crystal display or an OLED (Organic Light Emitting Diode) display. The meter display 21 and the CID 22 display various images on the display screen based on the control signal and video data acquired from the HCU 100. The meter display 21 is installed, for example, in front of the driver's seat. The CID 22 is installed in the center of the instrument panel 9. As an example, the CID 22 is a vertically long display of 10 inches or more. The display screen of the CID 22 may be curved. Furthermore, the display screen of the CID 22 may be continuous with the display screen of the meter display 21. The CID 22 has a function of a touch panel and detects, for example, a touch operation on the display screen by the driver or the like, specifically, an operation such as tapping, flicking, pinching, and swiping.

[0042] Based on the control signal and video data acquired from the HCU 100, the HUD 23 projects the light of the image formed in front of the driver onto a projection area PA defined on the windshield WS or the like. The light of the image reflected by the windshield WS into the interior of the vehicle is perceived by the driver sitting in the driver's seat. In this way, the HUD 23 displays a virtual image in the space in front of the projection area PA. The driver visually recognizes the virtual image within the angle of view VA displayed by the HUD 23 superimposed on the foreground of the vehicle Am.

[0043] The audio device 24 has a plurality of speakers arranged in the vehicle cabin so as to surround the driver's seat. The audio device 24 reproduces, in the vehicle cabin, notification sounds, voice messages, etc., through the speakers based on control signals and voice data acquired from the HCU 100. The audio device 24 presents information to the driver through his / her sense of hearing.

[0044] The operation device 26 is an input unit that accepts user operations by the driver, etc. User operations related to, for example, activation and deactivation of an autonomous driving function are input to the operation device 26. The operation device 26 includes a steering switch provided on the spokes of the steering wheel, an operation lever provided on the steering column, and a voice input device that recognizes the contents of the driver's speech.

[0045] A user terminal such as a smartphone may be connected to the in-vehicle network 1 or the HCU 100. The user terminal functions as an interface device and can replace at least one of the display device, the audio device 24, and the operation device 26. In addition, the user terminal may provide position information, direction information, map data, and the like to the driving assistance ECU 50a and the autonomous driving ECU 50b in place of the locator 35.

[0046] The HCU 100 is an in-vehicle computer that controls the meter display 21, the CID 22, the HUD 23, and the audio device 24 in the HMI system. The HCU 100 functions as a control device that comprehensively manages the presentation of information to the driver. The HCU 100 mainly includes a control circuit that includes a processing unit 11, a RAM 12, a storage unit 13, an input / output interface 14, and a bus connecting these.

[0047] The processing unit 11 is hardware for arithmetic processing coupled to the RAM 12. The processing unit 11 includes at least one arithmetic core such as a CPU and a GPU. The processing unit 11 may further include an IP core having an FPGA, an NPU, and other dedicated functions. The RAM 12 may include a video RAM for generating video data. The processing unit 11 accesses the RAM 12 to execute various processes for implementing the control method of the present disclosure. The storage unit 13 includes a non-volatile storage medium. The storage unit 13 stores various programs (such as a presentation control program) executed by the processing unit 11.

[0048] The HCU 100 has a plurality of functional units that perform integrated control of the presentation of information to the driver using each display device and the audio device 24 by executing a presentation control program stored in the storage unit 13 using the processing unit 11. Specifically, the HCU 100 has functional units such as an information acquisition unit 71, an integrated state estimation unit 73, and a provision control unit 74.

[0049] The information acquisition unit 71 acquires vehicle information indicating the state of the vehicle Am from the communication bus 99. The vehicle information includes, for example, vehicle speed information and status information indicating the state of the automatic driving function. The information acquisition unit 71 acquires operation information indicating the content of a user operation from the CID 22, the operation device 26, etc. The information acquisition unit 71 acquires driver's state information from the driver monitor 29.

[0050] In addition, the information acquisition unit 71 acquires information necessary for generating video data to be displayed on the display device. Specifically, the information acquisition unit 71 acquires video data to be displayed as a back monitor video BM (see FIG. 5, etc.) from the camera unit 31 that captures the rear or rear side. The information acquisition unit 71 acquires locator information, map data, route information, etc. necessary for generating a navigation map MP (see FIG. 4, etc.). Note that the video data of the navigation map MP may be generated at a predetermined resolution by a navigation device, a smartphone, or the like connected to the communication bus 99 and provided to the information acquisition unit 71.

[0051] The information acquisition unit 71 also acquires content data (video files, etc.) necessary for entertainment-related display, such as video content CTV (see FIG. 4, etc.). The content data is provided to the information acquisition unit 71 by a TV tuner mounted on the vehicle Am, an external medium electrically connected to the HCU 100, and a user terminal, such as a smartphone, paired with the HCU 100.

[0052] The integrated state estimation unit 73 acquires the driver's state information generated by the driver monitor 29 from the information acquisition unit 71. The integrated state estimation unit 73 determines the type of the second task being performed by the driver at least during the autonomous driving period. Specifically, the integrated state estimation unit 73 identifies the second task currently being performed by the driver from among a plurality of types of second tasks assumed in advance.

[0053] Here, the second task will be further explained. The autonomous driving ECU 50b and the HCU 100 temporarily permit the driver to perform actions other than driving. In detail, the autonomous driving function of level 3 by the autonomous driving ECU 50b allows the driver who is not required to monitor the surroundings to perform a specific action (second task) other than driving during the autonomous driving period when the vehicle Am is autonomously driving. In this case, the driver is a person (passenger) who takes over the control of driving from the autonomous driving system 50 when leaving a limited area or in an emergency. The driver may be legally permitted to perform the second task until a request to perform a driving operation by the autonomous driving system 50, i.e., a request to take over driving (Take Over Request) occurs.

[0054] The second task may be called a secondary activity or other activity. The second task must not prevent the driver from responding to a request to take over driving operations from the automated driving system 50. For example, viewing entertainment content such as video content CTV (see FIG. 4, etc.) and audiobooks, operating a smartphone, etc., and other actions (eating, etc.) are presumed as second tasks.

[0055] The integrated state estimation unit 73 refers to content playback information by the provision control unit 74 and operation information transmitted from a smartphone or the like, and identifies the second task being performed by combining it with state information of the driver monitor 29. The integrated state estimation unit 73 may be capable of estimating the second task being performed by the driver from an image captured by the near-infrared camera, using a determiner (determination logic) generated by machine learning, for example.

[0056] The process of identifying the second task performed by the driver may be performed by the driver monitor 29. In this embodiment, the integrated state estimation unit 73 acquires information indicating the type of the second task generated by the driver monitor 29 via the communication bus 99 and the information acquisition unit 71.

[0057] The provision control unit 74 performs integrated control of the provision of information to the driver using each display device and the audio device 24. The provision control unit 74 can change the display size and display layout of each content to be displayed on each display device according to the priority set for each content. As an example, the provision control unit 74 increases the display size or displays the content at the front of the screen as the content has a higher priority.

[0058] Specifically, the provision control unit 74 has an arbitration function for arbitrating information to be presented to the driver, and a data generation function for generating video data and audio data based on the arbitration result. As the arbitration function, the provision control unit 74 sets the above-mentioned priority for each content that is a candidate for provision based on various acquired information acquired by the information acquisition unit 71. The provision control unit 74 selects the content that is determined to have a high priority as the content to be provided to the driver. As the data generation function, the provision control unit 74 generates control signals and video data to be provided to each display device and control signals and audio data to be provided to the audio device 24 based on the content selection result. The provision control unit 74 sequentially outputs the generated control signals, video data, audio data, etc. to each presentation device.

[0059] During an autonomous driving period in which execution of the second task is permitted, the provision control unit 74 causes an image of the second task mode DMn, including entertainment content such as video content CTV, to be displayed on the screen of the CID22, as shown in FIG. 4, for example. As an example, the display of the second task mode DMn includes the video content CTV and the navigation map MP. The provision control unit 74 divides the vertically long screen of the CID22 into two screen areas, an upper one and an lower one, for use. The video content CTV is displayed in the upper screen area (hereinafter, the main display area DA1) so that it is easily visible to the driver. Meanwhile, the navigation map MP is displayed in the lower screen area (hereinafter, the sub-display area DA2).

[0060] In addition, in the second task mode DMn, the content displayed together with the video content CTV and the like may be changed as appropriate. Also, in the second task mode DMn, the video content CTV may be displayed on the entire screen of the CID 22. Furthermore, the content displayed in the second task mode DMn may be preset by a driver or the like.

[0061] When the driver is performing such a second task, the driver's attention is away from driving operations. Therefore, while the second task is being performed, the driver is likely to feel uneasy about behavior changes caused by the automatic driving control of the automatic driving ECU 50b. As an example, in a situation in which an emergency vehicle Qs (see FIG. 8, etc.) approaches the vehicle Am (hereinafter also referred to as the vehicle itself) during the automatic driving period, the driver in the second task state is unlikely to notice the approach of the emergency vehicle Qs. As a result, the behavior changes corresponding to the approach of the emergency vehicle Qs may cause uneasiness to the driver.

[0062] In order to avoid such a situation, the HCU 100 and the autonomous driving ECU 50b perform processing to respond to the emergency vehicle Qs approaching the vehicle. Hereinafter, the details of the operation of the HCU 100 and the autonomous driving ECU 50b related to the response to the emergency vehicle Qs will be described in order with reference to FIGS.

[0063] The emergency vehicle Qs includes at least one of a fire engine, an ambulance, a police vehicle, etc. In addition, unless otherwise specified, the "emergency vehicle Qs" in this disclosure refers to an emergency vehicle Qs traveling for emergency business.

[0064] The provision control unit 74 restricts the provision of entertainment content in the second task mode DMn by using an arbitration function, as shown in Fig. 5 to Fig. 7. The provision control unit 74 restricts the provision of entertainment content in stages. As an example, the provision control unit 74 switches the strength of the restriction on the provision of content among four stages, including a state where there is no restriction (see Fig. 4).

[0065] The restricted display mode DMr1 shown in Fig. 5 is a display mode in which restrictions on providing entertainment content are the weakest. The restricted display mode DMr1 is selected in a scene in which level 3 autonomous driving will continue in the future and the behavior of the vehicle is unlikely to change due to the approach of an emergency vehicle Qs.

[0066] In the limited display mode DMr1, an emergency vehicle notification MWq is displayed in the main display area DA1. The emergency vehicle notification MWq is a display object including a message such as "Pay attention to approaching emergency vehicles." The emergency vehicle notification MWq is displayed in a horizontally long strip near the upper edge of the main display area DA1. The emergency vehicle notification MWq is superimposed on a part of the video content CTV in the main display area DA1. Most of the video content CTV is displayed without being hidden by the emergency vehicle notification MWq. In addition, even if the emergency vehicle notification MWq is displayed, the video and audio of the video content CTV continue to be played. Therefore, the driver's viewing of the video content CTV can continue without being substantially hindered by the additional display of the emergency vehicle notification MWq.

[0067] In addition, in the limited display mode DMr1, the rear monitor image BM is displayed in the sub display area DA2 instead of the navigation map MP (see FIG. 4). The rear monitor image BM is an image of the rear or rear side captured by the camera unit 31. The provision control unit 74 can appropriately switch the camera unit 31 that provides the rear monitor image BM so that the emergency vehicle Qs approaching the vehicle is displayed on the screen. The provision control unit 74 may generate the rear monitor image BM by connecting image data from a plurality of camera units 31. The rear monitor image BM notifies the driver of the approach of the emergency vehicle Qs together with the emergency vehicle notification MWq. In addition, if the rear monitor image BM is displayed alongside the video content CTV, the driver can grasp the approaching situation of the emergency vehicle Qs without actually directing his / her gaze around the vehicle, in other words, without interrupting the viewing of the video content CTV.

[0068] The restricted display mode DMr2 shown in Fig. 6 is a display mode in which restrictions on providing entertainment content are set to a moderate level. The restricted display mode DMr2 is selected in a scene in which the behavior of the vehicle is likely to change due to the approach of an emergency vehicle Qs while level 3 autonomous driving will continue in the future. The restricted display mode DMr2 is a display mode that guides the attention of the driver watching the video content CTV to the surroundings of the vehicle.

[0069] In the limited display mode DMr2, as in the limited display mode DMr1 (see FIG. 5), the emergency vehicle notification MWq is displayed in the main display area DA1. At this time, a voice message such as "Watch out for approaching emergency vehicles" is played by the audio device 24. However, unlike the limited display mode DMr1, in the limited display mode DMr2, playback of the video of the video content CTV is temporarily stopped.

[0070] In addition, in the limited display mode DMr2, the display is not switched from the navigation map MP to the rear monitor image BM (see FIG. 5). If the rear monitor image BM is displayed on the CID 22, the driver's attention is not easily directed to the surroundings of the vehicle. Therefore, in the limited display mode DMr2, the display of the navigation map MP is continued.

[0071] In addition, when television images or audio books are provided as entertainment content, the audio playback of these contents may be resumed after the audio message alerting the driver to the emergency vehicle Qs is completed, for the convenience of the user. In this case, it is preferable to make the playback volume smaller than that before switching to the limited display mode DMr2 so that the driver can easily hear the siren sound of the emergency vehicle Qs.

[0072] The restricted display mode DMr3 shown in Fig. 7 is a display mode in which restrictions on providing entertainment content are most severe. The restricted display mode DMr3 is selected when level 3 autonomous driving ends, in other words, in a scene in which a request for a driver change occurs or has occurred.

[0073] In the limited display mode DMr3, both the video and audio playback of the video content CTV are stopped. As a result, the driver's viewing of the video content CTV is forcibly terminated at the discretion of the system. In addition, the provision control unit 74 swaps the display content of the main display area DA1 with the display content of the sub-display area DA2. As a result, the paused video content CTV moves to the sub-display area DA2. Meanwhile, the navigation map MP moves to the main display area DA1.

[0074] Furthermore, the provision control unit 74 causes at least one of the meter display 21 and the HUD 23 (see FIG. 1) to display an emergency vehicle warning MWa. The emergency vehicle warning MWa is a display object including a warning message such as "emergency vehicle approaching". The warning by the emergency vehicle warning MWa is stronger than the warning by the emergency vehicle notification MWq (see FIG. 6). The emergency vehicle warning MWa is displayed in the center of the screen of the meter display 21 or in the center of the field of view VA (see FIG. 1) of the HUD 23 so that it is easily visible to the driver who is looking around the vehicle. In addition, the provision control unit 74 causes the audio device 24 to play a voice message such as "emergency vehicle approaching".

[0075] The provision control unit 74 can change the method of notifying or warning the approach of the emergency vehicle Qs depending on the type of the second task being performed determined by the integrated state estimation unit 73. As an example, when the integrated state estimation unit 73 determines that a smartphone or the like is being operated, the provision control unit 74 displays a display corresponding to an emergency vehicle notification MWq or an emergency vehicle warning MWA on the screen of the smartphone. In addition, when the driver is performing an action such as eating, the provision control unit 74 notifies the approach of the emergency vehicle Qs by a method such as playing a voice message or lowering the volume of the video content CTV or the like. Furthermore, the provision control unit 74 may change whether to end the second task itself or to limit a part of it depending on the type of the second task being performed.

[0076] The display mode switching control by the HCU 100 described above is realized in cooperation with the autonomous driving ECU 50b shown in Figures 1 and 3. Below, the functions of the environment recognition unit 61 and the behavior determination unit 62 related to the response to the emergency vehicle Qs will be described in detail.

[0077] The environment recognition unit 61 determines whether or not an emergency vehicle Qs is approaching the vehicle during an autonomous driving period in which at least Level 3 autonomous driving is performed. The environment recognition unit 61 has an emergency vehicle detection unit 65, a lane recognition unit 66, and a road condition recognition unit 67 as auxiliary functional units based on the autonomous driving control program in order to grasp the approaching situation of the emergency vehicle Qs to the vehicle.

[0078] The emergency vehicle detection unit 65 detects the approach of the emergency vehicle Qs to the vehicle itself using acquired information acquired from a plurality of information sources. In addition, when the emergency vehicle detection unit 65 detects the approach of the emergency vehicle Qs, it further detects the type, relative position, moving direction, etc. of the emergency vehicle Qs as detailed information of the emergency vehicle Qs. The emergency vehicle detection unit 65 provides the detection result and detailed information of the emergency vehicle Qs to the behavior determination unit 62 as emergency vehicle information.

[0079] The emergency vehicle detection unit 65 can acquire emergency vehicle information through vehicle-to-vehicle communication and road-to-vehicle communication. During an emergency, the emergency vehicle Qs can transmit information such as its current location, traveling speed, and planned traveling route to a traffic management center and other surrounding vehicles using at least one of vehicle-to-vehicle communication, road-to-vehicle communication, and mobile communication. Therefore, the vehicle-to-vehicle communication device 37 receives emergency vehicle information through vehicle-to-vehicle communication with the emergency vehicle Qs. In addition, the road-to-vehicle communication device 38 receives emergency vehicle information about the approaching emergency vehicle Qs, which is distributed by, for example, a traffic management center, through road-to-vehicle communication with a roadside device. When the vehicle-to-vehicle communication device 37 and the road-to-vehicle communication device 38 receive emergency vehicle information, the vehicle-to-vehicle communication device 37 and the road-to-vehicle communication device 38 receive the emergency vehicle information to the emergency vehicle detection unit 65.

[0080] In addition, the emergency vehicle detection unit 65 can acquire emergency vehicle information based on detection information from the perimeter monitoring sensor 30. The emergency vehicle detection unit 65 detects the approach of the emergency vehicle Qs by extracting the emergency vehicle Qs captured in the image data captured by the camera unit 31 using a method such as pattern matching. The emergency vehicle detection unit 65 tracks changes in the position and size of the emergency vehicle Qs captured in the image data over time, and generates detailed information about the emergency vehicle Qs.

[0081] Furthermore, the emergency vehicle detection unit 65 extracts the sound of a siren emitted by the emergency vehicle Qs from the environmental sound collected by the microphone 33. When the emergency vehicle detection unit 65 extracts a siren sound of a predetermined pattern from the environmental sound, it detects the approach of the emergency vehicle Qs. The emergency vehicle detection unit 65 generates detailed information about the emergency vehicle Qs from changes in the volume and frequency of the siren sound.

[0082] The emergency vehicle detection unit 65 also recognizes voices such as "An ambulance is passing" and "Entering an intersection" emitted from the loudspeaker of the emergency vehicle Qs. The emergency vehicle detection unit 65 can generate text data by converting the recognized voices into text and provide the text data to the HCU 100.

[0083] The lane grasping unit 66 identifies the emergency vehicle lane Lnq (see FIG. 8) in which the emergency vehicle Qs runs, based on the position information of the emergency vehicle Qs grasped by the emergency vehicle detection unit 65. The lane grasping unit 66 grasps the positional relationship between the host vehicle lane Lns (see FIG. 8) in which the host vehicle runs and the emergency vehicle lane Lnq, based on the locator information of the host vehicle. Specifically, the lane grasping unit 66 grasps whether the host vehicle lane Lns is the emergency vehicle lane Lnq (see FIG. 13), the adjacent lane Lna (see FIG. 8 and FIG. 11), the separated lane Lnz (see FIG. 12), or the opposite lane Lno (see FIG. 18). The adjacent lane Lna is a lane adjacent to the emergency vehicle lane Lnq. The separated lane Lnz is a lane separated from the emergency vehicle lane Lnq. At least one lane exists between the separated lane Lnz and the emergency vehicle lane Lnq. In addition, when there is only one lane going in the same direction on the road on which the vehicle is traveling, the process of determining the positional relationship between the emergency vehicle lane Lnq and the vehicle itself does not have to be performed. In this case, it is simply determined that the vehicle is traveling in the emergency vehicle lane Lnq.

[0084] The road condition grasping unit 67 grasps the road condition around the vehicle by combining the congestion information acquired through road-to-vehicle communication, the detection information of the periphery monitoring sensor 30, the vehicle speed information, and the like. Specifically, the road condition grasping unit 67 judges whether the vehicle is traveling in a congestion. As an example, the road condition grasping unit 67 judges that the vehicle is traveling in a congestion when there is a vehicle ahead traveling in the same lane and the traveling speed is a predetermined speed (e.g., 10 km / h or less). After judging the congestion, the road condition grasping unit 67 judges that the congestion has been resolved when the traveling speed exceeds the predetermined speed (e.g., 50 km / h).

[0085] In addition, the road condition grasping unit 67 judges whether all lanes heading in the same direction are congested or whether only the host vehicle lane Lns is congested. In other words, the road condition grasping unit 67 judges whether there is a non-congested lane Lnp (see FIG. 15) in which the emergency vehicle Qs can travel.

[0086] Furthermore, the road condition grasping unit 67 may be capable of grasping the state of the roadside strip RST (see FIG. 14). As an example, the road condition grasping unit 67 grasps the width of the roadside strip RST and the presence or absence of other vehicles on the roadside strip RST, and judges whether or not the emergency vehicle Qs can travel on the roadside strip RST. In this case, the road condition grasping unit 67 judges whether or not the roadside strip RST can become a non-congested lane.

[0087] When the environment recognition unit 61 detects the approach of an emergency vehicle Qs, the behavior determination unit 62 determines whether or not it is necessary to restrict level 3 autonomous driving, that is, whether or not it is necessary to request the driver to take over driving. The behavior determination unit 62 has an automation level control unit 68 and an allowed range control unit 69 as auxiliary functional units based on an autonomous driving control program in order to determine the future behavior of the vehicle in response to the approach of the emergency vehicle Qs.

[0088] When an approach of an emergency vehicle Qs to the vehicle is detected during an automated driving period in which level 3 automated driving is performed, the automation level control unit 68 determines whether or not to continue the automated driving currently being performed. The automation level control unit 68 determines whether or not to continue level 3 automated driving based on the relative position of the emergency vehicle lane Lnq grasped by the lane grasping unit 66 and the road conditions around the vehicle grasped by the road condition grasping unit 67.

[0089] When the automation level control unit 68 determines to restrict level 3 autonomous driving, it further determines the control state of autonomous driving after canceling level 3 autonomous driving. Specifically, the automation level control unit 68 further determines whether to end all autonomous driving control and switch to manual driving, or to switch to level 1 or level 2 autonomous driving, etc. Among the processes for restricting level 3 autonomous driving, the process of switching to manual driving is referred to as "interruption."

[0090] When the automation level control unit 68 determines to continue the level 3 automated driving, it further determines whether or not to maintain the vehicle's driving speed set for automated driving. As an example, when the emergency vehicle Qs is driving in a lane adjacent to the vehicle Am, that is, when the lane grasping unit 66 grasps that the host vehicle lane Lns is the adjacent lane Lna, the automation level control unit 68 lowers the set speed for automated driving by a predetermined amount. As a result, the set speed after the approach of the emergency vehicle Qs is changed to be lower than the set speed before the approach of the emergency vehicle Qs. For example, when driving on a highway, the automation level control unit 68 can change the set speed to be equal to or lower than the legal minimum speed of the highway. However, when another vehicle Ax is present in front of the vehicle Am, the automation level control unit 68 can avoid deceleration control and perform control to match the speed of the other vehicle Ax in front. This prevents unnecessary risks caused by the sudden deceleration of only the vehicle Am.

[0091] The permission range control unit 69 controls the allowable range of the second task permitted to the driver. When an approach of an emergency vehicle Qs to the vehicle is detected during an autonomous driving period in which level 3 autonomous driving is performed, the permission range control unit 69 determines whether or not the second task should be continued in the permitted state permitted by the driver. Like the automation level control unit 68, the permission range control unit 69 determines whether or not the second task should be continued based on the relative position of the emergency vehicle lane Lnq grasped by the lane grasping unit 66 and the road conditions around the vehicle grasped by the road condition grasping unit 67.

[0092] Specifically, the permitted range control unit 69 determines to interrupt all types of second tasks when level 3 autonomous driving is interrupted. In addition, the permitted range control unit 69 also determines to interrupt the second tasks when it determines that the scene is one in which a sudden change in behavior of the vehicle is likely to occur. Furthermore, the permitted range control unit 69 may be able to specify second tasks that are permitted and second tasks that are not permitted among the multiple types of second tasks.

[0093] When the permission range control unit 69 determines that the second task is interrupted, the permission range control unit 69 further determines whether a release condition for the interruption is satisfied based on the positional relationship and traveling state of the vehicle Am, the emergency vehicle Qs, and the other vehicle Ax. The permission range control unit 69 can set one or more release conditions. The permission range control unit 69 can set, as a release condition, that a predetermined time has elapsed after the emergency vehicle Qs has overtaken (or been overtaken) the vehicle Am. The permission range control unit 69 can set, as a release condition, that the emergency vehicle Qs that has overtaken the vehicle Am has moved away from the vehicle Am by a predetermined distance (hereinafter, referred to as the separation distance DSq, see FIG. 9) or more. The permission range control unit 69 can set, as a release condition, that the behavior of the other vehicle Ax traveling around the vehicle Am has settled. The permission range control unit 69 can set, as a release condition, that the vehicle Am has overtaken the emergency vehicle Qs again after the emergency vehicle Qs has overtaken the vehicle Am (see FIG. 10).

[0094] The above-mentioned determination result by the automation level control unit 68 as to whether or not to continue the autonomous driving, the determination result by the permission range control unit 69 as to whether or not to continue the permission state of the second task, and the determination result as to whether or not the cancellation condition is satisfied are provided to the HCU 100 by the behavior determination unit 62. In addition, the determination result of the approach detection of the emergency vehicle Qs by the emergency vehicle detection unit 65 is also provided to the HCU 100 together with the determination result as to whether or not to continue. In the following description, the determination result as to whether or not to continue and the determination result of the approach detection provided by the autonomous driving ECU 50b are collectively referred to as the "continuation determination result." In addition, the determination result as to whether or not the cancellation condition is satisfied is referred to as the "cancellation determination result."

[0095] The HCU 100 performs display control for switching between the second task mode DMn and each of the restricted display modes DMr1 to DMr3 based on the continuation determination result and the release determination result provided by the autonomous driving ECU 50b. For such display control, the HCU 100 further includes a task continuation determination unit 72 as a functional unit for processing the continuation determination result based on the provided control program.

[0096] The task continuation judgment unit 72 acquires the continuation judgment result and the release judgment result output from the autonomous driving ECU 50b via the communication bus 99 and the information acquisition unit 71. The task continuation judgment unit 72 determines whether the approach of the emergency vehicle Qs has been detected based on the approach detection judgment result included in the continuation judgment result. In addition, the task continuation judgment unit 72 refers to the continuation judgment result and judges whether the permitted state in which the driver is permitted to perform the second task continues. Furthermore, the task continuation judgment unit 72 further judges whether level 3 autonomous driving will continue in the autonomous driving ECU 50b by referring to the continuation judgment result.

[0097] The provision control unit 74 controls the display of each display device according to the determination of the task continuation determination unit 72. When the task continuation determination unit 72 determines that the permitted state of level 3 autonomous driving or the second task will not continue, the provision control unit 74 restricts the display of entertainment content provided to the driver in relation to the second task. As described below, restrictions on the display of such content include interrupting the playback of at least one of the video and audio, hiding part of the video, lowering the volume, and terminating the display of the content itself.

[0098] Specifically, when the permitted state of the second task continues, the provision control unit 74 switches the display mode of CID22 from the second task mode DMn (see FIG. 4) to the restricted display mode DMr1 (see FIG. 5). Also, when the permitted state of the second task does not continue but level 3 autonomous driving continues, the provision control unit 74 switches the display mode of CID22 from the second task mode DMn to the restricted display mode DMr2 (see FIG. 6). Furthermore, when level 3 autonomous driving does not continue, the provision control unit 74 switches the display mode of CID22 from the second task mode DMn to the restricted display mode DMr3 (see FIG. 7).

[0099] Several specific examples of approach patterns of emergency vehicles Qs that result in the above-mentioned content display restrictions and approach patterns of emergency vehicles Qs that do not result in content display restrictions will be explained in order based on Figures 8 to 16 and 18, and with reference to Figure 3.

[0100] In the emergency vehicle approach pattern 1 shown in FIG. 8, the host vehicle (vehicle Am) is traveling in the adjacent lane Lna adjacent to the emergency vehicle lane Lnq. In this case, the continuation of the level 3 autonomous driving is permitted. On the other hand, the continuation of the second task is restricted. As an example, it is assumed that the other vehicle Ax traveling in the emergency vehicle lane Lnq will change lanes from the emergency vehicle lane Lnq to the host vehicle lane Lns toward the front of the host vehicle (vehicle Am) in order to secure a path for the emergency vehicle Qs. In response to such a movement of the other vehicle Ax, the autonomous driving ECU 50b changes the set speed of the level 3 autonomous driving to a lower speed and performs driving control to decelerate the host vehicle. The autonomous driving ECU 50b may cancel the change to lower the set speed in order to make the host vehicle follow the other vehicle Ax that has moved to the host vehicle lane Lns. In order to foresee the occurrence of such a behavior change and to direct the driver's attention outside the vehicle, the permitted range control unit 69 makes a decision to interrupt the second task.

[0101] When deceleration control is performed by the autonomous driving ECU 50b, the provision control unit 74 issues a notification to inform the driver that the driving speed will be reduced due to the approach of the emergency vehicle Qs before the deceleration control starts. In addition, the provision control unit 74 sets the display mode of the CID 22 to the restricted display mode DMr2 (see FIG. 6) in response to the continuation determination result of the behavior determination unit 62. As a result, the provision control unit 74 restricts the display of the video content CTV by stopping the playback of the video content CTV and displaying the emergency vehicle notification MWq.

[0102] 9 and 10, after the emergency vehicle Qs overtakes the vehicle Am, level 3 automated driving is permitted, while the use of the second task is restricted for at least a predetermined time. When the provision control unit 74 restricts the display of the video content CTV, the provision control unit 74 continues to restrict the display of the video content CTV until at least one cancellation condition is satisfied in the automation level control unit 68. The provision control unit 74 can appropriately combine multiple cancellation conditions.

[0103] The provision control unit 74 sets the release condition as being that the emergency vehicle Qs that has overtaken the vehicle is at least the separation distance DSq (see FIG. 9), and continues to restrict the display of the video content CTV until the emergency vehicle Qs is at least the separation distance DSq from the vehicle. Based on the establishment of the release condition resulting from the securing of the separation distance DSq, the provision control unit 74 notifies the driver that the second task can be used. Based on a user operation that instructs the resumption of the second task, the provision control unit 74 starts playing the video content CTV.

[0104] As another example, the provision control unit 74 sets the release condition as being that the emergency vehicle Qs that has overtaken the host vehicle and has stopped near a broken-down vehicle Ab or an accident vehicle has overtaken the host vehicle again (see FIG. 10). The provision control unit 74 continues to restrict the display of the video content CTV until the host vehicle overtakes the emergency vehicle Qs again, and issues a permission notification for the second task at the timing when the host vehicle overtakes the emergency vehicle Qs.

[0105] Furthermore, the provision control unit 74 sets the elapse of a predetermined time after the emergency vehicle Qs overtakes the host vehicle as a release condition, and continues to restrict the display of the video content CTV until the predetermined time has elapsed. In addition, the provision control unit 74 sets the elapse of the behavior of the other vehicle Ax traveling around the host vehicle as a release condition, and restricts the display of the content until the behavior of the other vehicle Ax has settled. For example, in a scene in which the other vehicle Ax changes lanes ahead of the host vehicle, the automation level control unit 68 determines that the behavior of the other vehicle Ax has settled when the speed of the other vehicle Ax has become substantially constant or when the inter-vehicle distance to the other vehicle Ax has become the target inter-vehicle distance for the following driving control.

[0106] In emergency vehicle approach pattern 2 shown in Fig. 11, the host vehicle (vehicle Am) is traveling in the adjacent lane Lna, similarly to the above-mentioned emergency vehicle approach pattern 1 (see Fig. 8). Even in the same scene, the automation level control unit 68 and the permission range control unit 69 can change the result of the determination as to whether or not to continue, for example, based on the traveling state of another vehicle Ax traveling ahead of the emergency vehicle Qs in the emergency vehicle lane Lnq, or on a pre-set setting by the user.

[0107] Specifically, in emergency vehicle approach pattern 2, not only the continuation of the second task but also the continuation of level 3 autonomous driving is restricted. As a result, the display mode of CID 22 is set to the restricted display mode DMr3 (see FIG. 7). As a result, the provision control unit 74 restricts the display of the video content CTV by stopping the playback of the video content CTV and moving the sub-display area DA2 of the video content CTV, etc. In addition, the provision control unit 74 warns the driver of the presence of the approaching emergency vehicle Qs by displaying an emergency vehicle warning MWe using the meter display 21 and the HUD 23. After the emergency vehicle Qs has overtaken the vehicle Am, for example, when the separation distance DSq is secured, the provision control unit 74 notifies the driver that level 3 autonomous driving has become available.

[0108] In emergency vehicle approach pattern 3 shown in FIG. 12, the host vehicle (vehicle Am) is traveling in a separation lane Lnz away from the emergency vehicle lane Lnq. In this case, level 3 autonomous driving is permitted to continue. In addition, the second task is also permitted to continue. As a result, the display mode of CID22 is set to the restricted display mode DMr1 (see FIG. 5). As a result, playback of the video content CTV continues. In addition, the presence of the approaching emergency vehicle Qs is notified to the driver by additionally displaying an emergency vehicle notification MWq in the main display area DA1 and displaying a rear monitor image BM in the sub-display area DA2.

[0109] In emergency vehicle approach pattern 4 shown in FIG. 13, the vehicle (vehicle Am) is traveling in the emergency vehicle lane Lnq. In this emergency vehicle approach pattern 4, it is considered that the desirable behavior of the vehicle is to move away from the emergency vehicle lane Lnq by changing lanes to the adjacent lane Lna. Therefore, the behavior determination unit 62 restricts not only the continuation of the second task but also the continuation of level 3 autonomous driving. In addition, the automation level control unit 68 restricts the activation of the LCA function by the autonomous driving ECU 50b or the driving assistance ECU 50a. This is because a lane change by the autonomous driving function requires a longer time than a lane change by manual driving, and the use of the LCA function prevents a quick move away from the emergency vehicle lane Lnq.

[0110] In the above emergency vehicle approaching pattern 4, the display mode of the CID 22 is set to the restricted display mode DMr3 (see FIG. 7) in response to the continuation judgment result that restricts both the autonomous driving and the second task. As a result, the display of the video content CTV is restricted. In addition, the meter display 21 and the HUD 23 display the emergency vehicle warning MWA to warn the driver of the presence of the approaching emergency vehicle Qs.

[0111] In emergency vehicle approach pattern 4, the driver may change lanes to retreat to the adjacent lane Lna by voluntarily acquiring control by override before a request to take over driving operations occurs. In this case, level 3 automated driving is temporarily stopped, and the behavior determination unit 62 transitions the control state to manual driving or level 1 driving control by the ACC function.

[0112] In emergency vehicle approach pattern 5 shown in FIG. 14, the vehicle (vehicle Am) is traveling in a traffic jam. In addition, in emergency vehicle approach pattern 5, all lanes included in the road on which the vehicle is traveling are congested. In this case, the behavior determination unit 62 permits continuation of the second task in addition to continuation of level 3 autonomous driving. As a result, the display mode of CID22 is set to the restricted display mode DMr1 (see FIG. 5). As a result, while the playback of the video content CTV continues, the presence of the approaching emergency vehicle Qs is notified to the driver by additionally displaying an emergency vehicle notification MWq in the main display area DA1 and displaying a back monitor image BM in the sub-display area DA2.

[0113] Furthermore, in emergency vehicle approach pattern 5, the behavior determination unit 62 changes the determination of whether or not to continue the automatic driving and the second task depending on the positional relationship between the host vehicle and the emergency vehicle Qs. Specifically, when the emergency vehicle Qs approaches the host vehicle, the behavior determination unit 62 restricts the continuation of the automatic driving and the second task. As an example, the emergency vehicle Qs can pass through a congested section by traveling on the side strip RST. Assuming such a scene, when the emergency vehicle Qs heading toward the side strip RST approaches the host vehicle, the behavior determination unit 62 restricts the continuation of both the second task and the automatic driving, and switches to a state that allows the driver to respond.

[0114] In response to the above-mentioned continuation judgment result, the display mode of the CID 22 is set to the restricted display mode DMr3 (see FIG. 7). As a result, the display of the video content CTV is restricted. In addition, the emergency vehicle warning MWA is displayed on the meter display 21 and the HUD 23. In this way, the driver is prompted to take measures such as getting out of the way of the emergency vehicle Qs heading toward the shoulder strip RST.

[0115] In emergency vehicle approach pattern 6 shown in Fig. 15, the host vehicle (vehicle Am) is traveling in a congested traffic jam. In addition, in emergency vehicle approach pattern 6, only the host vehicle lane Lns is congested among multiple lanes included in the road, and there is a non-congested lane Lnp in which the emergency vehicle Qs can travel. In this case, the continuation of level 3 automated driving is permitted. On the other hand, the continuation of the second task is restricted.

[0116] For example, when the emergency vehicle Qs moves to the non-congested lane Lnp, other vehicles traveling in the non-congested lane Lnp try to cut in the queue ahead of the vehicle to secure the path for the emergency vehicle Qs. Such movements of other vehicles tend to cause a sudden change in the behavior of the vehicle. Therefore, in order to direct the driver's attention outside the vehicle, the permitted range control unit 69 limits the continuation of the second task.

[0117] In response to the above-mentioned continuation determination result, the display mode of CID22 is set to the restricted display mode DMr2 (see FIG. 6). As a result, the provision control unit 74 stops the playback of the video content CTV and displays the emergency vehicle notification MWq to guide the driver's attention to the approaching emergency vehicle Qs.

[0118] In emergency vehicle approach pattern 7 shown in Fig. 16, emergency vehicle Qs is scheduled to leave the main roadway. Meanwhile, the host vehicle (vehicle Am) continues traveling on the main roadway. When the behavior determination unit 62 has acquired the planned traveling route of the emergency vehicle Qs, it determines whether the host vehicle can evacuate from the planned traveling route of the emergency vehicle Qs. When the behavior determination unit 62 determines that the host vehicle can evacuate from the planned traveling route of the emergency vehicle Qs, it continues both the level 3 autonomous driving and the permitted state of the second task.

[0119] In response to the above-mentioned continuation judgment result, the display mode of CID22 is set to the restricted display mode DMr1 as shown in Fig. 17. As a result, the emergency vehicle notification MWq additionally displayed in the main display area DA1 and the rear monitor image BM displayed in the sub-display area DA2 notify the driver of the presence of the approaching emergency vehicle Qs. In the emergency vehicle approach pattern 7, the imaging data of the camera unit 31 (e.g., the left side) including the emergency vehicle Qs is automatically selected and displayed as the rear monitor image BM.

[0120] In emergency vehicle approach pattern 8 shown in Fig. 18, emergency vehicle Qs is approaching from the front of the host vehicle (vehicle Am). That is, the host vehicle lane Lns is the opposite lane Lno to the emergency vehicle lane Lnq. When the behavior determination unit 62 recognizes that the emergency vehicle Qs is approaching from the front of the host vehicle and that the traveling direction of the emergency vehicle Qs is directly opposite to the traveling direction of the host vehicle, it continues both the level 3 automated driving and the second task permitted state.

[0121] In response to the above-mentioned continuation determination result, the display mode of CID22 is set to the unrestricted second task mode DMn as shown in FIG. 19. In this case, the emergency vehicle notification MWq is temporarily displayed so as to overlap a part of the video content CTV. The emergency vehicle notification MWq is displayed in a horizontally long strip shape near the upper edge of the main display area DA1, similar to the restricted display mode DMr1. The emergency vehicle notification MWq includes a message such as "An emergency vehicle is approaching from the front" to notify the presence of the emergency vehicle Qs traveling in the oncoming lane. The display of the emergency vehicle notification MWq when the emergency vehicle Qs is approaching from the front is terminated after a predetermined time (for example, about several seconds) has elapsed. Even with such a short display, the driver can grasp the state outside the vehicle through the emergency vehicle notification MWq while continuing to watch the video content CTV, in other words, without performing surrounding monitoring. Therefore, the emergency vehicle notification MWq can give the driver a sense of security.

[0122] In each of the scenes described above, when the level 3 autonomous driving is temporarily restricted due to the approach of the emergency vehicle Qs, the autonomous driving ECU 50b resumes the level 3 autonomous driving based on the driver's input of a return operation. The autonomous driving ECU 50b can also automatically resume the level 3 autonomous driving based on the recognition of the driving environment around the vehicle by the environment recognition unit 61. Specifically, when the emergency vehicle Qs moves forward from the vehicle, the autonomous driving ECU 50b raises the autonomous driving, which is restricted to level 2 (driving within the lane by the LTA function), to level 3. When the vehicle changes lanes from the emergency vehicle lane Lnq to the adjacent lane Lna, the autonomous driving ECU 50b resumes the level 3 autonomous driving after returning to the original lane. Furthermore, the autonomous driving ECU 50b monitors the status of one or more vehicles ahead (other vehicles Ax) traveling in the vehicle lane Lns, and when a lane change of the vehicle ahead is not detected for a certain period of time, it presumes that the behavior of the vehicle ahead has settled down and resumes the level 3 autonomous driving. As described above, the autonomous driving ECU 50b may automatically return from manual driving to level 3 autonomous driving, or may return from level 1 or 2 autonomous driving to level 3 autonomous driving.

[0123] Next, the details of each process that distinguishes the emergency vehicle approach pattern explained so far and determines whether to continue or not, and the details of each process that switches the display mode based on the continuation judgment result and the cancellation judgment result will be explained below based on Figures 20 to 23 and with reference to Figures 3 to 19.

[0124] The autonomous driving continuation determination process shown in Fig. 20 is started by the autonomous driving ECU 50b based on the start of the autonomous driving period of Level 3. The autonomous driving ECU 50b continues the autonomous driving continuation determination process until Level 3 autonomous driving is restricted.

[0125] In S101, it is determined whether or not an emergency vehicle Qs is approaching the vehicle. If it is determined in S101 that the emergency vehicle Qs is not approaching, the process proceeds to S110, where it is determined to continue level 3 automated driving, and the process returns to S101. On the other hand, if it is determined in S101 that the emergency vehicle Qs is approaching, the process proceeds to S102. In S102, the latest vehicle information related to the vehicle, such as locator information, vehicle speed information, and recognition information about the surroundings of the vehicle, is obtained, and the process proceeds to S103. In S103, emergency vehicle information is obtained, and the process proceeds to S104.

[0126] In S104, the positional relationship between the emergency vehicle Qs and the vehicle is grasped, and it is determined whether the vehicle is traveling in the opposite lane Lno to the emergency vehicle lane Lnq. If it is determined in S104 that the vehicle is traveling in the opposite lane Lno as in the above "emergency vehicle approach pattern 8 (see FIG. 18)" and that the emergency vehicle Qs traveling in the oncoming lane is approaching from in front of the vehicle, the process proceeds to S110, and it is determined to continue the automatic driving. On the other hand, if it is determined in S104 that the emergency vehicle Qs is approaching from behind the vehicle, the process proceeds to S105.

[0127] In S105, it is determined whether the vehicle can be evacuated outside the planned route of the emergency vehicle Qs. In S105, it is determined whether the planned driving line of the vehicle can be deviated from the planned driving route based on the planned driving route of the emergency vehicle Qs acquired in S103. If the planned driving line of the vehicle cannot be deviated from the planned driving route of the emergency vehicle Qs, it is determined in S105 that evacuation outside the planned driving route is impossible, and the process proceeds to S106.

[0128] On the other hand, when the host vehicle can escape from the planned driving route of the emergency vehicle Qs as in the above-mentioned "emergency vehicle approach pattern 7 (see FIG. 16)", the process proceeds to S110 and it is determined that the autonomous driving is to be continued. For example, at an intersection, the host vehicle can escape from the planned driving route of the emergency vehicle Qs by turning right or left, or the host vehicle escapes from the planned driving route of the emergency vehicle Qs by turning right or left, etc., and also proceeds from S105 to S110. According to the determination to continue at S110, the operation execution unit 63 performs escape driving outside the planned driving route according to the planned driving line generated by the action determination unit 62.

[0129] In S106, the road conditions around the vehicle are determined. If it is determined in S106 that the vehicle is traveling in a traffic jam and all lanes are congested, for example, as in "emergency vehicle approaching pattern 5 (see FIG. 14)", the process proceeds to S107.

[0130] In S107, it is determined whether or not the distance from the vehicle to the emergency vehicle Qs is secured. In S107, for example, if it is determined that the emergency vehicle Qs, which is traveling on the shoulder strip RST, is approaching directly behind or beside the vehicle, in other words, the vehicle and the emergency vehicle Qs are adjacent to each other on the road, it is determined that the distance is not secured, and the process proceeds to S111. In S111, it is determined that the level 3 autonomous driving is to be restricted, and a series of continuation determination processes is terminated. On the other hand, in S107, if it is determined that the vehicle and the emergency vehicle Qs are not adjacent to each other on the road, and the distance to the emergency vehicle Qs is secured, the process proceeds to S110, and it is determined that the autonomous driving is to be continued.

[0131] Also, in S106, if it is determined that the host vehicle lane Lns is congested while a non-congested lane Lnp through which the emergency vehicle Qs can pass exists, for example, as in "emergency vehicle approach pattern 6 (see FIG. 15)", the process proceeds to S110 and it is determined to continue the autonomous driving. On the other hand, if it is determined in S106 that the surroundings of the host vehicle are not congested (non-congestion determination), the process proceeds to S108.

[0132] In S108, the positional relationship between the emergency vehicle lane Lnq and the vehicle is determined. If it is determined in S108 that the vehicle and the emergency vehicle Qs are traveling in the same lane, for example, as in "emergency vehicle approach pattern 4 (see FIG. 13)", the process proceeds to S111, where it is determined that level 3 autonomous driving should be restricted, and the series of continuation determination processes is terminated.

[0133] On the other hand, if it is determined in S108 that the host vehicle is not traveling in the emergency vehicle lane Lnq, the process proceeds to S109. In S109, the positional relationship between the adjacent lane Lna and the host vehicle is determined. If it is determined in S109 that the host vehicle is traveling in the separated lane Lnz, for example, as in "emergency vehicle approach pattern 3 (see FIG. 12)", the process proceeds to S110, and it is determined to continue the automatic driving.

[0134] For example, in the case of "emergency vehicle approach pattern 1 (see FIG. 8)," when the autonomous driving is set to continue even if the host vehicle is traveling in the adjacent lane Lna, S109 may be omitted. In this case, unless the host vehicle and the emergency vehicle Qs are located in the same lane, it is determined in S110 that the autonomous driving is to continue.

[0135] On the other hand, if it is determined in S109 that the vehicle is traveling in the adjacent lane Lna, for example, as in "emergency vehicle approach pattern 2 (see Figure 11)", the process proceeds to S111, where it is determined that level 3 autonomous driving should be restricted, and the series of continuation decision processes is terminated.

[0136] 21 is started by the autonomous driving ECU 50b based on the start of the autonomous driving period, similar to the autonomous driving continuation decision process. The autonomous driving ECU 50b repeatedly starts the continuation decision process for the second task in parallel with the autonomous driving continuation decision process until level 3 autonomous driving is restricted.

[0137] In S131, it is determined whether or not an emergency vehicle Qs is approaching the host vehicle. In S131, it is further determined whether or not the emergency vehicle Qs is approaching the host vehicle from behind. If it is determined in S131 that the emergency vehicle Qs is approaching from behind, the process proceeds to S132. On the other hand, if it is determined in S131 that the emergency vehicle Qs is not approaching or that the emergency vehicle Qs is approaching from the front of the host vehicle, the process proceeds to S140.

[0138] In S140, it is determined that the second task is to be permitted to continue, and the continuation determination process is temporarily terminated. In this case, the behavior determination unit 62 generates a continuation determination result including a notification indicating that the emergency vehicle Qs is not approaching or a notification indicating that the emergency vehicle Qs is approaching from the front, together with a notification indicating that the autonomous driving and the second task are permitted to continue, and outputs the result to the HCU 100.

[0139] In S132, it is determined whether level 3 autonomous driving has been cancelled by an autonomous driving continuation determination process or the like. If it is determined in S132 that level 3 autonomous driving is continuing, the process proceeds to S133. On the other hand, if it is determined in S132 that level 3 autonomous driving has been interrupted (cancelled), the process proceeds to S142. In S142, it is determined that the second task should be restricted (interrupted), and the continuation determination process is terminated. In this case, the action determination unit 62 generates a continuation determination result that includes a notification instructing the interruption of autonomous driving and the second task together with a notification indicating the approach of an emergency vehicle Qs from behind, and outputs the result to the HCU 100.

[0140] In S133, the latest vehicle information related to the vehicle is obtained, and the process proceeds to S134. In S134, emergency vehicle information is obtained, and the process proceeds to S135. In S135, it is determined whether the vehicle can be evacuated outside the planned route of the emergency vehicle Qs. If it is determined in S135 that the vehicle can be evacuated from the planned driving route of the emergency vehicle Qs (see FIG. 16), the process proceeds to S141. In S141, it is determined that the second task is to be permitted to continue, and the continuation determination process is temporarily terminated. In this case, the behavior determination unit 62 generates a continuation determination result including a notification indicating permission to continue the automatic driving and the second task together with a notification indicating the approach of the emergency vehicle Qs, and outputs the result to the HCU 100.

[0141] On the other hand, if it is determined in S135 that the host vehicle cannot evacuate from the planned driving route of the emergency vehicle Qs, the process proceeds to S136. In S136, the road conditions around the host vehicle are determined. If it is determined in S136 that the host vehicle is traveling in a traffic jam and all lanes are in a traffic jam (see FIG. 14), the process proceeds to S137.

[0142] In S137, it is determined whether or not the distance from the vehicle to the emergency vehicle Qs is secured. If it is determined in S137 that the emergency vehicle Qs is approaching the vehicle in close proximity, the process proceeds to S142, where it is determined that the second task should be limited, and the continuation determination process is temporarily terminated. On the other hand, if it is determined in S137 that the distance to the emergency vehicle Qs is secured, the process proceeds to S141, where it is determined that the second task should be continued, and the continuation determination process is temporarily terminated.

[0143] Also, in S136, if it is determined that the lane Lns of the vehicle is congested, but there is a non-congested lane Lnp through which the emergency vehicle Qs can pass (see FIG. 15), the process proceeds to S142, where it is determined that the second task is to be restricted. On the other hand, if it is determined in S136 that there is no congestion around the vehicle, the process proceeds to S138.

[0144] In S138, the positional relationship between the emergency vehicle lane Lnq and the host vehicle is determined. If it is determined in S138 that the host vehicle and the emergency vehicle Qs are traveling in the same lane (see FIG. 13), the process proceeds to S142, where it is determined that the second task is to be interrupted, and the continuation determination process is temporarily terminated. On the other hand, if it is determined in S138 that the host vehicle is not traveling in the emergency vehicle lane Lnq, the process proceeds to S139.

[0145] In S139, the positional relationship between the adjacent lane Lna and the vehicle is determined. If it is determined in S139 that the vehicle is traveling in the separated lane Lnz (see FIG. 12), the process proceeds to S141, where it is determined that the second task is to be continued in the permitted state, and the continuation determination process is temporarily terminated. On the other hand, if it is determined in S139 that the vehicle is traveling in the adjacent lane Lna (see FIG. 8 and FIG. 11), the process proceeds to S142, where it is determined that the second task is to be interrupted, and the continuation determination process is temporarily terminated.

[0146] 22 is started by the HCU 100 based on the start of display of entertainment content such as video content CTV. The HCU 100 repeatedly starts the display control process until the second task is interrupted or an operation to end the video content CTV is input by the driver.

[0147] In S151, the continuation determination result provided by the autonomous driving ECU 50b is acquired, and the process proceeds to S152. In S152, the continuation determination result acquired in S151 is referenced, and it is determined whether or not there is a notification indicating the approach of the emergency vehicle Qs. If it is determined in S152 that there is no notification indicating the approach of the emergency vehicle Qs, the display control process is temporarily terminated. On the other hand, if it is determined in S152 that there is a notification indicating the approach of the emergency vehicle Qs, the process proceeds to S153.

[0148] In S153, the continuation determination result acquired in S151 is referenced to determine whether or not there is a notification indicating the approach of an emergency vehicle Qs from ahead of the vehicle. If it is determined in S153 that an emergency vehicle Qs is approaching from ahead of the vehicle, the process proceeds to S154. In S154, an emergency vehicle notification MWq is temporarily additionally displayed in the main display area DA1 of the normal second task mode DMn (see FIG. 19) to notify the driver of the approach of the emergency vehicle Qs from ahead. In S154, the emergency vehicle notification MWq is terminated based on the passage of a predetermined time, and the display control process is temporarily terminated.

[0149] On the other hand, if it is determined in S153 that the emergency vehicle Qs is approaching from behind the vehicle, the process proceeds to S155. In S155, the process refers to the continuation determination result acquired in S151 and determines whether the state in which the driver is permitted to perform the second task continues. If it is determined in S155 that the state in which the second task is permitted continues, the process proceeds to S156. In S156, the approach of the emergency vehicle Qs is notified to the driver by additionally displaying the emergency vehicle notification MWq by switching to the limited display mode DMr1 (see FIG. 5), and the display control process is temporarily terminated.

[0150] On the other hand, if it is determined in S155 that the second task permission state will not continue, the process proceeds to S157. In S157, the continuation determination result acquired in S151 is referenced to further determine whether or not level 3 autonomous driving will continue. If it is determined in S157 that autonomous driving will continue, the process proceeds to S158. In S158, the provision of the video content CTV is terminated by switching to the restricted display mode DMr2 (see FIG. 6). In addition, in S158, the driver is notified of the approach of the emergency vehicle Qs by additionally displaying an emergency vehicle notification MWq, and the display control process is temporarily terminated.

[0151] On the other hand, if it is determined in S157 that the autonomous driving will not continue, the process proceeds to S159. In S159, the display mode is switched to the limited display mode DMr3 (see FIG. 7), thereby terminating the provision of the video content CTV, and the display layout is changed to switch the contents of the main display area DA1 and the sub-display area DA2, and the process proceeds to S160. In S160, an emergency vehicle warning MWA is displayed on at least one of the meter display 21 and the HUD 23 to warn the driver of the approach of the emergency vehicle Qs, and the display control process is temporarily terminated.

[0152] The display control process shown in FIG. 23 is started by the HCU 100 based on switching of the display mode to the limited display modes DMr1 to DMr3.

[0153] In S171, the release determination result provided by the automatic driving ECU 50b is acquired, and the process proceeds to S172. In S172, the release determination result acquired in S171 is referenced, and it is determined whether or not a release condition for permitting the release of the display restriction of the video content CTV is satisfied. If it is determined in S172 that at least one of the preset release conditions is satisfied, the process proceeds to S173. In S173, the release of the restricted display modes DMr1 to DMr3 is permitted, and the display control process ends. In this case, the display mode is switched from each of the restricted display modes DMr1 to DMr3 to the normal second task mode DMn automatically or based on a user operation. On the other hand, if it is determined in S172 that the release condition is not satisfied, it is determined in S174 that the restricted display modes DMr1 to DMr3 are continued, and the process returns to S171. As described above, the restriction on the display of the video content CTV continues until a specific release condition is satisfied.

[0154] The HCU 100 of the first embodiment described above restricts the display of the video content CTV when the driver is performing the second task and determines that the second task permission state will not continue due to the approach of the emergency vehicle Qs to the vehicle. Therefore, even if the driver is performing the second task, the driver can easily notice the approach of the emergency vehicle Qs due to the restriction on the display of the video content CTV. As a result, it is possible to prevent the driver from feeling uneasy even if the emergency vehicle Qs approaches during level 3 automated driving.

[0155] In addition, when the provision control unit 74 of the first embodiment restricts the display of the video content CTV, the provision control unit 74 continues the restriction on the display of the video content CTV until at least one of the cancellation conditions is satisfied. Therefore, the driver can pay appropriate attention to the emergency vehicle Qs without being hindered by the continued provision of the video content CTV.

[0156] In addition, the provision control unit 74 of the first embodiment continues the restriction on the display of the content until the passage of a predetermined time after the emergency vehicle Qs has overtaken the vehicle, as a release condition. As a result, the display restriction on the video content CTV is not released during the period in which behavioral changes caused by the emergency vehicle Qs are likely to occur. Therefore, a situation in which the driver becomes anxious due to the early resumption of the second task can be avoided.

[0157] Furthermore, the provision control unit 74 of the first embodiment continues the display restriction of the video content CTV until the emergency vehicle Qs that has overtaken the vehicle is at least the separation distance DSq, using the condition for lifting the restriction. Even with this display restriction, the display restriction of the video content CTV is not lifted during the period in which behavior changes caused by the emergency vehicle Qs are likely to occur. Therefore, a situation in which the driver becomes uneasy due to the early resumption of the second task can be avoided.

[0158] In addition, the provision control unit 74 of the first embodiment continues the display restriction of the video content CTV until the behavior of the other vehicle Ax traveling around the vehicle settles down, with the condition for lifting the restriction. According to the above, the display restriction of the video content CTV is not lifted during a period in which behavior changes due to the movement of the other vehicle Ax are likely to occur. Therefore, a situation in which the driver becomes uneasy due to early resumption of the second task can be avoided.

[0159] In addition, the provision control unit 74 of the first embodiment sets the release condition as the vehicle overtaking the emergency vehicle Qs again after the emergency vehicle Qs has overtaken the vehicle, and continues the restriction on the display of the content until the vehicle overtakes the emergency vehicle Qs again. By setting such a release condition, in a scene where the emergency vehicle Qs stops at the scene of a breakdown or accident, the display restriction can be released after the driver confirms that the emergency vehicle Qs has stopped. As a result, the driver can resume the second task with peace of mind.

[0160] Furthermore, in the first embodiment, even if an emergency vehicle Qs traveling in the oncoming lane approaches the vehicle from the front, the approach of the emergency vehicle Qs is notified. This notification allows the driver to grasp the situation outside the vehicle, that is, the approach of the emergency vehicle Qs, while continuing the second task. As a result, it is possible to increase the driver's sense of security.

[0161] In addition, when the autonomous driving ECU 50b detects the approach of an emergency vehicle Qs to the vehicle, the task continuation determination unit 72 in the first embodiment further determines whether or not level 3 autonomous driving will continue based on the continuation determination result acquired from the autonomous driving ECU 50b. Then, when it is determined that autonomous driving will not continue (see FIG. 7), the display of the video content CTV is more restricted than when it is determined that autonomous driving will continue (see FIG. 6). As described above, when autonomous driving is restricted, the display control that strongly restricts the display of the video content CTV can strongly encourage the driver to voluntarily monitor the surroundings. As a result, it is possible to provide the video content CTV in a manner that does not hinder the driver's response to a request to take over driving operations from the autonomous driving system 50.

[0162] Furthermore, when the autonomous driving ECU 50b of the first embodiment determines that an emergency vehicle Qs is approaching the vehicle, it determines whether or not to continue level 3 autonomous driving and whether or not to continue the permitted state of the second task. Therefore, even if the second task is being performed, the driver is more likely to notice the approach of the emergency vehicle Qs because level 3 autonomous driving or the permitted state of the second task is not continued. As a result of the above, it is possible to make the driver less likely to feel uneasy even if an emergency vehicle Qs approaches during autonomous driving.

[0163] Furthermore, in the first embodiment, whether or not to continue the level 3 autonomous driving and the second task is determined based on the relative position of the emergency vehicle lane Lnq with respect to the vehicle itself. As a result, the continuation of the autonomous driving and the second task can be appropriately restricted depending on the approaching situation of the emergency vehicle Qs. Therefore, convenience for the driver can be ensured while preventing the driver from feeling uneasy.

[0164] Specifically, in the first embodiment, as shown in emergency vehicle approach pattern 1 (see FIG. 8), when the host vehicle travels in the adjacent lane Lna, continuation of level 3 autonomous driving is permitted. On the other hand, continuation of the second task is restricted. As a result, the driver's attention can be smoothly shifted from the second task to monitoring the surroundings of the host vehicle. As a result, the driver is less likely to feel uneasy even when, for example, a behavior change occurs due to another vehicle Ax cutting in ahead of the host vehicle.

[0165] In addition, in the first embodiment, even if the continuation of Level 3 autonomous driving is permitted, the traveling speed of the vehicle Am set for autonomous driving is changed to be lower than that before the approach of the emergency vehicle Qs. This speed suppression control allows the driver to understand that driving control corresponding to the approach of the emergency vehicle Qs has been started. As a result of the above, the anxiety of the driver who recognizes the approach of the emergency vehicle Qs can be further reduced.

[0166] In the first embodiment, as shown in emergency vehicle approach pattern 2 (see FIG. 11), when the vehicle is traveling in the adjacent lane Lna, continuation of level 3 autonomous driving is restricted. Then, the presence of the emergency vehicle Qs is notified to the driver. As described above, when the emergency vehicle Qs passes very close to the vehicle, the autonomous driving is terminated and the driver is prompted to monitor the surroundings, which can further reduce the driver's anxiety.

[0167] Furthermore, in the first embodiment, as shown in emergency vehicle approach pattern 3 (see FIG. 12), when the host vehicle travels in a separated lane Lnz away from the emergency vehicle lane Lnq, continuation of the second task is permitted. In this way, in a scene where the risk of a sudden behavior change is low, high convenience for the driver can be ensured by allowing continuation of the second task. In addition, even if continuation of the second task is permitted, the presence of the emergency vehicle Qs is notified to the driver. Therefore, even if a behavior change occurs, the driver is less likely to feel anxious because he or she knows the cause of the behavior change.

[0168] In addition, in the first embodiment, when the vehicle is traveling in the emergency vehicle lane Lnq as shown in emergency vehicle approach pattern 4 (see FIG. 13), not only the continuation of the second task but also the continuation of the automated driving is restricted. As described above, in a scene where there is a high risk of a sudden change in behavior, the driver's anxiety can be reduced by handing over the authority of driving operations to the driver in advance.

[0169] In the first embodiment, the continuation of the level 3 autonomous driving and the second task is determined based on the road conditions around the vehicle. Therefore, the continuation of the autonomous driving and the second task can be appropriately restricted according to the road conditions. As a result, the convenience of the driver can be ensured while preventing the driver from feeling uneasy.

[0170] Specifically, in the first embodiment, as shown in emergency vehicle approach pattern 5 (see FIG. 14), when all lanes included in the road are congested, continuation of the second task is permitted. In addition, the approach of the emergency vehicle Qs is notified to the driver by an additional display of the emergency vehicle notification MWq. As a result, the driver can continue watching the video content CTV while recognizing the approach of the emergency vehicle Qs. As a result, convenience for the driver is easily ensured.

[0171] On the other hand, when the emergency vehicle Qs comes very close to the vehicle, not only the second task but also the level 3 automated driving is restricted. As described above, the process of quickly transferring the driving operation authority to the driver allows the driver to take flexible measures, such as yielding driving space to the emergency vehicle Qs that is traveling on the shoulder strip RST.

[0172] In addition, in the first embodiment, as shown in emergency vehicle approach pattern 6 (see FIG. 15), even if the host vehicle lane Lns is congested, if there is a non-congested lane Lnp in which the emergency vehicle Qs can travel, continuation of level 3 automated driving is permitted. Therefore, the driving burden on the driver who is driving in a congested traffic jam can be reduced. On the other hand, continuation of the second task is limited. Therefore, the driver's attention is directed to the surroundings of the host vehicle. Therefore, the driver is less likely to feel uneasy even when a behavior change occurs due to a situation such as another vehicle Ax cutting in front of the host vehicle.

[0173] In the first embodiment, as shown in emergency vehicle approach pattern 7 (see FIG. 16), if the planned travel route of the emergency vehicle Qs is known and the vehicle can evacuate from this planned travel route, continuation of the second task is permitted. As a result, excessive restrictions on the second task are avoided, making it easier to ensure convenience for the driver.

[0174] In the above embodiment, the environment recognition unit 61 corresponds to the "approach determination unit", the behavior determination unit 62 and the task continuation determination unit 72 correspond to the "continuation determination unit", the integrated state estimation unit 73 corresponds to the "sleep estimation unit", and the provision control unit 74 corresponds to the "provision restriction unit". Furthermore, the autonomous driving ECU 50b and HCU 100 correspond to the "control device", the video content CTV corresponds to the "content", the second task corresponds to the "specific act other than driving", and the separation distance DSq corresponds to the "predetermined distance".

[0175] Second Embodiment In the second embodiment of the present disclosure, the autonomous driving ECU 50b shown in FIG. 3 is capable of implementing autonomous driving at level 4. During the autonomous driving period at level 4, the permission range control unit 69 permits the driver to take a posture with the seat belt removed and to sleep, in addition to watching the video content CTV and operating the smartphone. When an emergency vehicle Qs approaches the vehicle traveling at level 4 autonomous driving, the permission range control unit 69 determines whether or not to continue the state in which the driver is permitted to sleep. Note that the automation level control unit 68 in principle continues autonomous driving at level 4 even in a scene where an emergency vehicle Qs is approaching.

[0176] The permitted range control unit 69 does not permit the driver to continue sleeping in each approaching scene of the emergency vehicle Qs in which the level 3 autonomous driving or the second task is restricted in the first embodiment. The permitted range control unit 69 does not permit the driver to sleep in emergency vehicle approaching patterns 1, 2, and 4 to 6 in the first embodiment. On the other hand, in emergency vehicle approaching patterns 3, 7, and 8 in the first embodiment in which the implementation of the second task is not restricted, the permitted range control unit 69 permits the driver to continue sleeping. That is, the permitted range control unit 69 permits the driver to continue sleeping when there is no other vehicle Ax around the vehicle and the emergency vehicle Qs is in a different lane from the vehicle. In addition, the permitted range control unit 69 permits the driver to continue sleeping in any situation as long as the emergency vehicle Qs is traveling in the opposing lane of the vehicle. The result of the determination by the permitted range control unit 69 as to whether or not to continue sleeping is provided to the HCU 100, as are the continuation determination result and the release determination result.

[0177] The HCU 100 starts a sleep monitoring process (see FIG. 24) based on the activation of level 4 autonomous driving by the autonomous driving ECU 50b. The sleep monitoring process is performed by the task continuation determination unit 72, the integrated state estimation unit 73, and the provision control unit 74. The sleep monitoring process issues a notification (hereinafter, awakening notification) to encourage awakening from a sleeping state as necessary.

[0178] The task continuation determination unit 72 acquires the result of the sleep continuation determination output from the autonomous driving ECU 50b via the information acquisition unit 71 (S283). The task continuation determination unit 72 refers to the result of the sleep continuation determination and determines whether the permitted state in which the driver is permitted to sleep will continue in the future (S284).

[0179] The integrated state estimation unit 73 acquires driver state information indicating whether the driver is asleep (hereinafter, driver sleep information) from the driver monitor 29 through the information acquisition unit 71 (S281 and S286). The driver sleep information may be information indicating in a binary manner whether the driver is asleep or not, or may be information indicating the driver's sleep state in multiple stages or a score, etc. The integrated state estimation unit 73 refers to the driver sleep information and estimates whether the driver is asleep or awake (S282 and S287).

[0180] When the integrated state estimation unit 73 determines that the driver is asleep (S282: YES) and the task continuation determination unit 72 determines that the sleep permission state does not continue (S284: NO), the provision control unit 74 performs an awakening notification (S285). The awakening notification is, for example, an actuation mainly consisting of playing a voice message by the audio device 24. The provision control unit 74 may further perform an act of raising the backrest of the driver's seat as the awakening notification. After starting the awakening notification, when the integrated state estimation unit 73 determines that the driver is asleep (S287: YES) or when a timeout occurs, the provision control unit 74 ends the awakening notification (S288).

[0181] In a scene where an emergency vehicle Qs is approaching, the provision control unit 74 sets the start timing of the awakening notification when level 4 autonomous driving is performed earlier than the start timing of the display restriction when level 3 autonomous driving is performed. Specifically, the threshold value for determining the approach of the emergency vehicle Qs is relaxed when the driver is asleep when level 4 autonomous driving is performed compared to when level 3 autonomous driving is performed. As an example, when the approach of the emergency vehicle Qs is determined based on the distance from the vehicle to the emergency vehicle Qs, the distance is set longer when the driver is asleep than when level 3 autonomous driving is performed. Note that, when the driver is not asleep, the threshold value for determining the approach of the emergency vehicle Qs may be set to substantially the same timing when level 4 autonomous driving is performed and when level 3 autonomous driving is performed.

[0182] The second embodiment described so far also has the same effect as the first embodiment, and can reduce the driver's anxiety when level 4 autonomous driving is being performed.

[0183] In addition, in the second embodiment, when the driver is estimated to be asleep and it is determined that the sleep permission state will not continue due to the approach of the emergency vehicle Qs, an awakening notification is implemented to encourage the driver to wake up. As a result, the driver can watch the automatic driving ECU 50b's response to the emergency vehicle Qs in a somewhat awake state. Therefore, a situation in which the driver encounters a change in the behavior of the vehicle Am due to the approach of the emergency vehicle Qs while asleep is avoided.

[0184] In addition, the notification control unit 74 of the second embodiment performs the awakening notification to wake up the sleeping driver at an earlier timing than the approach notification (see FIG. 22 S156, etc.) that informs the awake driver of the approach of the emergency vehicle Qs when level 3 automated driving is being performed. By adjusting the notification timing in this way, the driver can be provided with enough time to wake up from the sleeping state and recognize the approach of the emergency vehicle Qs. As a result, it is possible to increase the driver's sense of security.

[0185] Furthermore, in the second embodiment, when the vehicle is traveling in a lane different from the emergency vehicle lane Lnq and there is no other vehicle Ax around the vehicle, the permitted range control unit 69 continues the sleep permission state. As described above, if the scene is very unlikely to request the driver to respond to the emergency vehicle Qs, the state in which the driver is permitted to sleep may be continued. Omission of such an awakening notification further improves the convenience of level 4 autonomous driving. In the second embodiment, the integrated state estimation unit 73 corresponds to the "sleep determination unit."

[0186] (Other embodiments) Although several embodiments of the present disclosure have been described above, the present disclosure should not be construed as being limited to the above-described embodiments, and can be applied to various embodiments and combinations within the scope not departing from the gist of the present disclosure.

[0187] In the first modification of the above embodiment, the autonomous driving ECU 50b outputs a determination result as to whether or not the second task can be continued to the HCU 100. When the task continuation determination unit 72 determines that the permitted state of the second task is not to be continued, the provision control unit 74 suspends the display of the video content CTV. In the first modification, regardless of whether the autonomous driving of level 3 is continued or not, the provision control unit 74 sets the restricted display mode DMr3 and ends the second task being performed by the driver. As in the first modification above, there may be only one display mode that suspends the playback of entertainment content. Alternatively, three or more display modes that suspend the playback of entertainment content may be prepared.

[0188] In the second modification of the above embodiment, the positional relationship between the emergency vehicle lane Lnq and the vehicle itself is used to determine whether or not to continue the level 3 autonomous driving and the second task, but the road conditions around the vehicle are not used. In contrast, in the third modification of the above embodiment, the road conditions around the vehicle itself are used to determine whether or not to continue the autonomous driving and the second task, but the positional relationship between the emergency vehicle lane Lnq and the vehicle itself is not used.

[0189] In the fourth modification of the above embodiment, the timing of notifying the approach of the emergency vehicle Qs is changed depending on whether the vehicle is traveling in a traffic jam. The timing of notifying the approach of the emergency vehicle Qs is set to a timing that allows the driver time (for example, about 13 seconds) to respond to a request for a driver handover from the automatic driving system 50. Therefore, when the vehicle is traveling in a traffic jam, the provision control unit 74 delays the timing of notifying the approach of the emergency vehicle Qs compared to normal.

[0190] Furthermore, the provision control unit 74 may change the timing of notifying the driver of the approach of the emergency vehicle Qs depending on the route of acquisition of the emergency vehicle information. As an example, when the approach of the emergency vehicle Qs is determined from the detection information of the periphery monitoring sensor 30, the provision control unit 74 immediately notifies the driver of the approach of the emergency vehicle Qs. On the other hand, when the approach of the emergency vehicle Qs is determined based on the information acquired through road-to-vehicle communication, the provision control unit 74 notifies the driver of the approach of the emergency vehicle Qs at a timing when the distance from the emergency vehicle Qs to the vehicle becomes less than a predetermined distance (for example, about 1 km).

[0191] In the fifth modification of the above embodiment, a text message other than the prescribed one is displayed on the emergency vehicle notification MWq and the emergency vehicle warning MWA. In more detail, the emergency vehicle detection unit 65 can recognize the voice emitted from the emergency vehicle Qs and provide the text data to the HCU 100. The provision control unit 74 causes the emergency vehicle notification MWq and the emergency vehicle warning MWA to display a message text based on the acquired text data.

[0192] In the above embodiment, the display of the video content CTV is restricted by pausing the playback of the video content CTV. However, the method of restricting the display of the content can be changed as appropriate. As an example, in Modification 6 of the above embodiment, in the restricted display mode DMr2, a display process of graying down the video content CTV is performed as a restriction on the display of the content. In Modification 7 of the above embodiment, in at least one of the restricted display mode DMr2 and the restricted display mode DMr3, a window including a warning message is superimposed so as to cover the center of the display of the video content CTV.

[0193] The combination of conditions for releasing the display restriction set in the first embodiment may be changed as appropriate. For example, when at least one of the above-mentioned four release conditions is met, the display restriction of the video content CTV may be released. Also, some of the release conditions may be omitted. As an example, in the eighth modification of the above embodiment, when a predetermined time has elapsed and the separation distance DSq has been secured, or when the emergency vehicle Qs has been overtaken, the display restriction is released. Furthermore, in the ninth modification of the above embodiment, when the behavior of the other vehicle Ax around the vehicle becomes stable, the display restriction is released regardless of the elapsed time after overtaking and the distance to the emergency vehicle Qs.

[0194] In the above-mentioned modified example 10 of the second embodiment, the driver is notified of the awakening when the driver is asleep at substantially the same timing as the notification of the approach of the emergency vehicle Qs. In this case, the notification of the approach of the emergency vehicle Qs may also serve as the awakening notification. In addition, in the above-mentioned modified example 11 of the second embodiment, when the emergency vehicle Qs is approaching from behind the vehicle, the driver is not permitted to continue sleeping, regardless of the position of the vehicle rail or the presence or absence of another vehicle Ax. Furthermore, in the above-mentioned modified example 12 of the embodiment, the notification of the emergency vehicle Qs approaching from the front of the vehicle is omitted.

[0195] The display device that displays the content related to the second task is not limited to the CID 22. For example, the meter display 21 and the HUD 23 may be used to provide entertainment content. Furthermore, each display EMB, EML, and EMR of the electronic mirror system may be used to provide content. Also, the display device that displays the content may be selectable by the driver.

[0196] In addition, the shape, luminous color, display position, etc. of each image displayed on each display device may be changed as appropriate. Also, the language type of messages displayed on each display device may be changed as appropriate based on user settings such as the driver's settings and settings such as the country and region in which the vehicle Am is used. Similarly, the language type of audio messages played back by the audio device 24 may also be changed as appropriate.

[0197] The second tasks permitted to the driver are not limited to the actions described above, and may be changed as appropriate depending on the road traffic laws and other regulations of the country and region in which the vehicle Am is used.

[0198] In a thirteenth modification of the above embodiment, any one of the meter display 21, the CID 22, and the HUD 23 is configured integrally with the HCU 100. In other words, the processing function of the HCU 100 is implemented in a control circuit of any one of the display devices. In this thirteenth modification, the display device including the processing function of the HCU 100 corresponds to the "control device."

[0199] The autonomous driving system 50 of the above embodiment is provided with two in-vehicle ECUs, the driving assistance ECU 50a and the autonomous driving ECU 50b. However, the autonomous driving system 50 may be configured with a single in-vehicle ECU having the functions of the driving assistance ECU 50a and the autonomous driving ECU 50b. Furthermore, the functions of the autonomous driving ECU 50b and the HCU 100 of the above embodiment may be implemented in a single in-vehicle ECU.

[0200] In the above embodiment, each function provided by the autonomous driving ECU 50b and the HCU 100 can be provided by software and hardware that executes the software, software only, hardware only, or a combination of these. Furthermore, when such a function is provided by an electronic circuit as hardware, each function can be provided by a digital circuit including a large number of logic circuits, or an analog circuit.

[0201] The form of the storage medium that stores the program that can realize the above control method may also be changed as appropriate. For example, the storage medium is not limited to a configuration provided on a circuit board, but may be provided in the form of a memory card or the like, inserted into a slot, and electrically connected to the control circuit of the autonomous driving ECU or HCU. Furthermore, the storage medium may be an optical disk or a hard disk drive that is a source from which the program is copied to the autonomous driving ECU or HCU.

[0202] The control unit and the method described herein may be implemented by a special-purpose computer having a processor programmed to execute one or more functions embodied in a computer program. Alternatively, the device and the method described herein may be implemented by a special-purpose hardware logic circuit. Alternatively, the device and the method described herein may be implemented by one or more special-purpose computers configured by a combination of a processor that executes a computer program and one or more hardware logic circuits. The computer program may also be stored in a computer-readable non-transitory tangible recording medium as instructions to be executed by the computer. The technical ideas grasped from the embodiments and modifications described thus far are described below as Appendices 1 to 23, 2-6, 2-14, 2-29, 2-31, 2-38, and 2-46 to 2-50. (Appendix 1) A control device used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, a continuation determination unit (72) that, when an approach of an emergency vehicle (Qs) to the vehicle, which is the vehicle, is detected during an automatic driving period in which the automatic driving is performed, determines whether or not a state in which the driver is permitted to perform a specific act other than driving continues; a provision restriction unit (74) that restricts display of content (CTV) provided to the driver when the continuation determination unit determines that the permitted state of the specific action will not continue; A control device comprising: (Appendix 2) 2. The control device according to claim 1, wherein the provision restriction unit, when restricting display of the content, continues the restriction on display of the content until at least one lifting condition is satisfied. (Appendix 3) The control device according to claim 2, wherein the provision restriction unit sets the elapse of a predetermined time after the emergency vehicle has overtaken the host vehicle as the release condition, and continues to restrict the display of the content until the predetermined time has elapsed. (Appendix 4) The control device according to claim 2 or 3, wherein the provision restriction unit sets the release condition as being that the emergency vehicle that has overtaken the host vehicle has moved a predetermined distance (DSq) or more away, and continues to restrict the display of the content until the emergency vehicle moves a predetermined distance or more away. (Appendix 5) The control device according to any one of Appendices 2 to 4, wherein the provision restriction unit sets the release condition to be that the behavior of other vehicles traveling around the vehicle has settled down, and continues to restrict the display of the content until the behavior of the other vehicles has settled down. (Appendix 6) The control device according to any one of Appendices 2 to 5, wherein the provision restriction unit sets the release condition to the fact that the emergency vehicle has overtaken the host vehicle and the host vehicle has overtaken the emergency vehicle again, and continues to restrict the display of the content until the emergency vehicle has overtaken again. (Appendix 7) a sleep estimation unit (73) for estimating whether the driver is asleep, The continuation determination unit determines whether the state in which the driver is permitted to sleep continues, The control device according to any one of appendices 1 to 6, wherein the provision restriction unit issues an alert to encourage the driver to wake up when it is determined that the sleep-permitted state will not continue due to the approach of the emergency vehicle under conditions in which the driver is estimated to be in a sleep state. (Appendix 8) The control device according to claim 7, wherein the provision restriction unit issues a notification to the driver who is asleep to wake him up earlier than a notification to the driver who is awake to notify him of the approach of the emergency vehicle. (Appendix 9) The control device according to any one of claims 1 to 8, wherein the provision restriction unit notifies a driver of the approach of the emergency vehicle when the emergency vehicle traveling in an oncoming lane approaches the vehicle from ahead of the vehicle. (Appendix 10) The continuation determination unit further determines whether or not to continue the autonomous driving when the approach of the emergency vehicle to the host vehicle is detected, The control device according to any one of appendices 1 to 9, wherein the provision restriction unit restricts the display of the content when the continuation judgment unit determines that the autonomous driving will not be continued more than when the autonomous driving is determined to be continued. (Appendix 11) A control device used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, an approach determination unit (61) that determines whether or not an emergency vehicle (Qs) is approaching the vehicle, which is the vehicle, during an automatic driving period in which the automatic driving is performed; a continuation determination unit (62) that, when the approach determination unit determines that the emergency vehicle is approaching, determines whether or not to continue the automated driving and whether or not to continue a state in which the driver is permitted to perform a specific action other than driving; A control device comprising: (Appendix 12) The control device according to claim 10 or 11, wherein at least one of whether or not to continue the specific action and whether or not to continue the autonomous driving is determined based on the relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling. (Appendix 13) The control device according to claim 12, wherein, when the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, the continuation of the specific action is restricted and the continuation of the autonomous driving is permitted. (Appendix 14) The control device according to claim 13, wherein, when continuation of the autonomous driving is permitted, the traveling speed of the vehicle set for the autonomous driving is changed to be lower than that before the approach of the emergency vehicle. (Appendix 15) The control device described in Appendix 12, wherein, when the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, continuation of the autonomous driving is restricted and the presence of the emergency vehicle is notified to the driver. (Appendix 16) A control device as described in any one of appendices 12 to 15, in which, when the vehicle is traveling in a separation lane (Lnz) away from the emergency vehicle lane, continuation of the specific action is permitted and the presence of the emergency vehicle is notified to the driver. (Appendix 17) 17. The control device according to any one of claims 12 to 16, wherein, when the host vehicle is traveling in the emergency vehicle lane, continuation of the specific action is restricted and continuation of the autonomous driving is restricted. (Appendix 18) The control device according to any one of appendices 12 to 17, wherein the sleep-permitted state continues when the host vehicle is traveling in a lane different from the emergency vehicle lane and there is no other vehicle around the host vehicle. (Appendix 19) The control device according to any one of appendices 10 to 18, wherein at least one of whether to continue the specific action and whether to continue the autonomous driving is determined based on the condition of a road on which the vehicle is traveling. (Appendix 20) 20. The control device according to claim 19, wherein, if all lanes of the road are congested, continuation of the specific action is permitted and the presence of the emergency vehicle is notified to the driver. (Appendix 21) A control device as described in Appendix 19 or 20, wherein, when the lane in which the vehicle is traveling among the lanes included in the road is congested and a non-congested lane (Lnp) in which the emergency vehicle can travel is present on the road, the continuation of the specific action is restricted and the continuation of the autonomous driving is permitted. (Appendix 22) A control program used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, At least one processing section (11) During the automatic driving period in which the automatic driving is performed, when an approach of an emergency vehicle (Qs) to the vehicle, which is the vehicle itself, is detected, it is determined whether or not the state in which the driver is permitted to perform a specific act other than driving continues (S155); When it is determined that the permission state of the specific act will not continue, the display of the content (CTV) provided to the driver is restricted (S158, S159). A control program that causes the device to execute a process including the above. (Appendix 23) A control program used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, At least one processing section (51) During the automatic driving period in which the automatic driving is performed, it is determined whether or not an emergency vehicle (Qs) is approaching the vehicle (S101, S131); When it is determined that the emergency vehicle is approaching, it is determined whether or not the automated driving should be continued and whether or not the state in which the driver is permitted to perform a specific act other than driving should be continued (S106 to S109, S136 to S139). A control program that causes the device to execute a process including the above. (Appendix 2-6) A control device used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, a continuation determination unit (72) that, when an approach of an emergency vehicle (Qs) to the vehicle, which is the vehicle, is detected during an automatic driving period in which the automatic driving is performed, determines whether or not a state in which the driver is permitted to perform a specific act other than driving continues; a provision restriction unit (74) that restricts display of content (CTV) provided to the driver when the continuation determination unit determines that the permitted state of the specific action will not continue, The continuation determination unit further determines whether or not to continue the autonomous driving when the approach of the emergency vehicle to the host vehicle is detected, the provision restriction unit, when it is determined by the continuation determination unit that the autonomous driving will not be continued, restricts display of the content more than when it is determined that the autonomous driving will be continued; At least one of whether or not to continue the specific action and whether or not to continue the automated driving is determined based on the relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling, A control device that restricts the continuation of the specific action and allows the continuation of the automated driving when the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane. (Appendix 2-14) A control device used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, a continuation determination unit (72) that, when an approach of an emergency vehicle (Qs) to the vehicle, which is the vehicle, is detected during an automatic driving period in which the automatic driving is performed, determines whether or not a state in which the driver is permitted to perform a specific act other than driving continues; a provision restriction unit (74) that restricts display of content (CTV) provided to the driver when the continuation determination unit determines that the permitted state of the specific action will not continue, The continuation determination unit further determines whether or not to continue the autonomous driving when the approach of the emergency vehicle to the host vehicle is detected, the provision restriction unit, when it is determined by the continuation determination unit that the autonomous driving will not be continued, restricts display of the content more than when it is determined that the autonomous driving will be continued; At least one of whether to continue the specific action and whether to continue the automated driving is determined based on the condition of the road on which the vehicle is traveling, A control device that restricts the continuation of the specific action and allows the continuation of the automated driving when the lane in which the vehicle is traveling among the lanes included in the road is congested and a non-congested lane (Lnp) in which the emergency vehicle can travel is present on the road. (Appendix 2-29) A control device used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, an approach determination unit (61) that determines whether or not an emergency vehicle (Qs) is approaching the vehicle, which is the vehicle, during an automatic driving period in which the automatic driving is performed; a continuation determination unit (62) that, when the approach determination unit determines that the emergency vehicle is approaching, determines whether or not to continue the automated driving and whether or not to continue a state in which the driver is permitted to perform a specific action other than driving, based on a relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling; A control device comprising: (Appendix 2-31) A control device used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, an approach determination unit (61) that determines whether or not an emergency vehicle (Qs) is approaching the vehicle, which is the vehicle, during an automatic driving period in which the automatic driving is performed; a continuation determination unit (62) that, when the approach determination unit determines that the emergency vehicle is approaching, determines whether or not to continue the automated driving and whether or not to continue a state in which the driver is permitted to perform a specific action other than driving, At least one of whether or not to continue the specific action and whether or not to continue the automated driving is determined based on the relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling, A control device that restricts the continuation of the specific action and allows the continuation of the automated driving when the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane. (Appendix 2-38) A control device used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, an approach determination unit (61) that determines whether or not an emergency vehicle (Qs) is approaching the vehicle, which is the vehicle, during an automatic driving period in which the automatic driving is performed; a continuation determination unit (62) that, when the approach determination unit determines that the emergency vehicle is approaching, determines whether or not to continue the automated driving and whether or not to continue a state in which the driver is permitted to perform a specific action other than driving, At least one of whether to continue the specific action and whether to continue the automated driving is determined based on the condition of the road on which the vehicle is traveling, A control device that, when the lane in which the vehicle is traveling among the lanes included in the road is congested and a non-congested lane (Lnp) in which the emergency vehicle can travel is present on the road, restricts the continuation of the specific action and allows the continuation of the automated driving. (Appendix 2-46) A control program used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, At least one processing section (11) During the automatic driving period in which the automatic driving is performed, when an approach of an emergency vehicle (Qs) to the vehicle, which is the vehicle itself, is detected, it is determined whether or not the state in which the driver is permitted to perform a specific act other than driving continues (S155); When it is determined that the permission state of the specific act will not continue, the display of the content (CTV) provided to the driver is restricted (S158, S159); When the approach of the emergency vehicle to the vehicle is detected, it is further determined whether or not the autonomous driving should be continued (S157); When it is determined that the autonomous driving will not continue, the display of the content is limited more than when it is determined that the autonomous driving will continue; Determine at least one of whether to continue the specific action and whether to continue the automated driving based on a relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling; When the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, the continuation of the specific action is restricted and the continuation of the automated driving is permitted. A control program that causes the device to execute a process including the above. (Appendix 2-47) A control program used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, At least one processing section (11) During the automatic driving period in which the automatic driving is performed, when an approach of an emergency vehicle (Qs) to the vehicle, which is the vehicle itself, is detected, it is determined whether or not the state in which the driver is permitted to perform a specific act other than driving continues (S155); When it is determined that the permission state of the specific act will not continue, the display of the content (CTV) provided to the driver is restricted (S158, S159); When the approach of the emergency vehicle to the vehicle is detected, it is further determined whether or not the autonomous driving should be continued (S157); When it is determined that the autonomous driving will not continue, the display of the content is limited more than when it is determined that the autonomous driving will continue; determining whether or not to continue the specific action and / or whether or not to continue the automated driving based on the condition of the road on which the vehicle is traveling; When a lane in which the vehicle is traveling among lanes included in the road is congested and a non-congested lane (Lnp) in which the emergency vehicle can travel is present on the road, the continuation of the specific action is restricted and the continuation of the automated driving is permitted. A control program that causes the device to execute a process including the above. (Appendix 2-48) A control program used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, At least one processing section (51) During the automatic driving period in which the automatic driving is performed, it is determined whether or not an emergency vehicle (Qs) is approaching the vehicle (S101, S131); When it is determined that the emergency vehicle is approaching, it is determined whether or not to continue the automated driving and whether or not to continue the state in which the driver is permitted to perform a specific act other than driving, based on the relative position of the emergency vehicle lane (Lnq) in which the emergency vehicle is traveling (S106 to S109, S136 to S139). A control program that causes the device to execute a process including the above. (Appendix 2-49) A control program used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, At least one processing section (51) During the automatic driving period in which the automatic driving is performed, it is determined whether or not an emergency vehicle (Qs) is approaching the vehicle (S101, S131); When it is determined that the emergency vehicle is approaching, it is determined whether or not the automated driving should be continued and whether or not the state in which the driver is permitted to perform a specific act other than driving should be continued (S106 to S109, S136 to S139); Determine at least one of whether to continue the specific action and whether to continue the automated driving based on a relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling; When the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, the continuation of the specific action is restricted and the continuation of the automated driving is permitted. A control program that causes the device to execute a process including the above. (Appendix 2-50) A control program used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, At least one processing section (51) During the automatic driving period in which the automatic driving is performed, it is determined whether or not an emergency vehicle (Qs) is approaching the vehicle (S101, S131); When it is determined that the emergency vehicle is approaching, it is determined whether or not the automated driving should be continued and whether or not the state in which the driver is permitted to perform a specific act other than driving should be continued (S106 to S109, S136 to S139); determining whether or not to continue the specific action and / or whether or not to continue the automated driving based on the condition of the road on which the vehicle is traveling; When a lane in which the vehicle is traveling among lanes included in the road is congested and a non-congested lane (Lnp) in which the emergency vehicle can travel is present on the road, the continuation of the specific action is restricted and the continuation of the automated driving is permitted. A control program that causes the device to execute a process including the above. [Explanation of symbols]

[0203] Am vehicle (own vehicle), Qs emergency vehicle, DSq separation distance (predetermined distance), Lna adjacent lane, Lnq emergency vehicle lane, Lnz separation lane, Lnp non-congested lane, CTV video content (content), 11, 51 processing unit, 50b autonomous driving ECU (control unit), 61 environment recognition unit (approach determination unit), 62 action determination unit (continuation determination unit), 72 task continuation determination unit (continuation determination unit), 73 integrated state estimation unit (sleep estimation unit), 74 provision control unit (provision restriction unit), 100 HCU (control unit)

Claims

1. A control device used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, a continuation determination unit (72) that, when an approach of an emergency vehicle (Qs) to the vehicle, which is the vehicle, is detected during an automatic driving period in which the automatic driving is performed, determines whether or not a state in which the driver is permitted to perform a specific act other than driving continues; a provision restriction unit (74) that restricts display of content (CTV) provided to the driver when the continuation determination unit determines that the permitted state of the specific action will not continue; The continuation determination unit further determines whether or not to continue the autonomous driving when the approach of the emergency vehicle to the host vehicle is detected, the provision restriction unit, when it is determined by the continuation determination unit that the autonomous driving will not be continued, restricts display of the content more than when it is determined that the autonomous driving will be continued; At least one of whether or not to continue the specific action and whether or not to continue the automated driving is determined based on the relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling, A control device that restricts the continuation of the specific action and allows the continuation of the autonomous driving when the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane.

2. The control device according to claim 1 , wherein the notification restriction unit notifies the approach of the emergency vehicle when the emergency vehicle traveling in an oncoming lane approaches the host vehicle from in front of the host vehicle.

3. The control device according to claim 1 or 2, wherein, when continuation of the autonomous driving is permitted, a traveling speed of the vehicle set for the autonomous driving is changed to a speed lower than that before the approach of the emergency vehicle.

4. A control device according to any one of claims 1 to 3, wherein when the vehicle is traveling in a separation lane (Lnz) away from the emergency vehicle lane, continuation of the specific action is permitted and the presence of the emergency vehicle is notified to the driver.

5. The control device according to any one of claims 1 to 4, wherein, when the vehicle is traveling in the emergency vehicle lane, the continuation of the specific action is restricted and the continuation of the autonomous driving is restricted.

6. The control device according to any one of claims 1 to 5, wherein the sleep permission state continues when the vehicle is traveling in a lane other than the emergency vehicle lane and there are no other vehicles around the vehicle.

7. The control device according to any one of claims 1 to 6, wherein at least one of whether or not to continue the specific action and whether or not to continue the autonomous driving is determined based on the condition of the road on which the vehicle is traveling.

8. The control device according to claim 7, wherein when the lane in which the vehicle is traveling among the lanes included in the road is congested and a non-congested lane (Lnp) in which the emergency vehicle can travel is present on the road, the continuation of the specific action is restricted and the continuation of the autonomous driving is permitted.

9. A control device used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, a continuation determination unit (72) that, when an approach of an emergency vehicle (Qs) to the vehicle, which is the vehicle, is detected during an automatic driving period in which the automatic driving is performed, determines whether or not a state in which the driver is permitted to perform a specific act other than driving continues; a provision restriction unit (74) that restricts display of content (CTV) provided to the driver when the continuation determination unit determines that the permitted state of the specific action will not continue; The continuation determination unit further determines whether or not to continue the autonomous driving when the approach of the emergency vehicle to the host vehicle is detected, the provision restriction unit, when it is determined by the continuation determination unit that the autonomous driving will not be continued, restricts display of the content more than when it is determined that the autonomous driving will be continued; At least one of whether to continue the specific action and whether to continue the automated driving is determined based on the condition of the road on which the vehicle is traveling, A control device that restricts the continuation of the specific action and allows the continuation of the automated driving when the lane in which the vehicle is traveling among the lanes included in the road is congested and a non-congested lane (Lnp) in which the emergency vehicle can travel is present on the road.

10. The control device according to claim 9 , wherein the notification restriction unit notifies the approach of the emergency vehicle when the emergency vehicle traveling in an oncoming lane approaches the host vehicle from in front of the host vehicle.

11. The control device according to claim 9 or 10, wherein at least one of whether or not to continue the specific action and whether or not to continue the autonomous driving is determined based on the relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling.

12. 12. The control device according to claim 11, wherein, when the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, continuation of the autonomous driving is restricted, and the presence of the emergency vehicle is notified to the driver.

13. 13. The control device according to claim 11 or 12, wherein, when the vehicle is traveling in a separation lane (Lnz) away from the emergency vehicle lane, continuation of the specific action is permitted and the presence of the emergency vehicle is notified to the driver.

14. The control device according to any one of claims 11 to 13, wherein, when the vehicle is traveling in the emergency vehicle lane, the continuation of the specific action is restricted and the continuation of the autonomous driving is restricted.

15. The control device according to any one of claims 11 to 14, wherein the sleep permission state continues when the vehicle is traveling in a lane different from the emergency vehicle lane and there are no other vehicles around the vehicle.

16. A control device as described in any one of claims 7 to 15, wherein, when all lanes of the road are congested, continuation of the specific action is permitted and the presence of the emergency vehicle is notified to the driver.

17. The control device according to any one of claims 1 to 16, wherein, when the provision restriction unit has restricted the display of the content, the provision restriction unit continues the restriction on the display of the content until at least one release condition is satisfied.

18. The control device according to claim 17 , wherein the provision restriction unit sets the release condition to the lapse of a predetermined time after the emergency vehicle has overtaken the host vehicle, and continues the restriction on the display of the content until the lapse of the predetermined time.

19. The control device according to claim 17 or 18, wherein the provision restriction unit sets the release condition as being that the emergency vehicle that has overtaken the host vehicle has moved a predetermined distance (DSq) or more away, and continues to restrict the display of the content until the emergency vehicle has moved a predetermined distance or more away.

20. The control device according to any one of claims 17 to 19, wherein the provision restriction unit sets the release condition to be that the behavior of other vehicles traveling around the vehicle has settled down, and continues to restrict the display of the content until the behavior of the other vehicles has settled down.

21. The control device according to any one of claims 17 to 20, wherein the provision restriction unit sets the release condition to the fact that the emergency vehicle has overtaken the vehicle and the vehicle has overtaken the emergency vehicle again, and continues to restrict the display of the content until the vehicle overtakes the emergency vehicle again.

22. A sleep estimation unit (73) for estimating whether the driver is asleep or not, The continuation determination unit determines whether the state in which the driver is permitted to sleep continues, The control device according to any one of claims 17 to 21, wherein the provision restriction unit issues an alert to encourage the driver to wake up when it is determined that the sleep permission state will not continue due to the approach of the emergency vehicle under conditions in which the driver is estimated to be in a sleep state.

23. The control device according to claim 22 , wherein the provision restriction unit issues a notification to the driver in a sleeping state to urge the driver to wake up earlier than a notification to notify the driver in an awake state of the approach of the emergency vehicle.

24. A control device used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, An approach determination unit (61) that determines whether or not an emergency vehicle (Qs) is approaching the vehicle, which is the vehicle, during an automatic driving period in which the automatic driving is performed; a continuation determination unit (62) that, when the approach determination unit determines that the emergency vehicle is approaching, determines whether or not to continue the automated driving and whether or not to continue a state in which the driver is permitted to perform a specific act other than driving, based on a relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling; A control device that, when the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, restricts continuation of the autonomous driving and notifies the driver of the presence of the emergency vehicle.

25. A control device used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, An approach determination unit (61) that determines whether or not an emergency vehicle (Qs) is approaching the vehicle, which is the vehicle, during an automatic driving period in which the automatic driving is performed; a continuation determination unit (62) that, when the approach determination unit determines that the emergency vehicle is approaching, determines whether or not to continue the automated driving and whether or not to continue a state in which the driver is permitted to perform a specific act other than driving, At least one of whether or not to continue the specific action and whether or not to continue the automated driving is determined based on the relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling, A control device that restricts the continuation of the specific action and allows the continuation of the autonomous driving when the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane.

26. The control device according to claim 25, wherein, when continuation of the autonomous driving is permitted, a traveling speed of the vehicle set for the autonomous driving is changed to a speed lower than that before the approach of the emergency vehicle.

27. When the vehicle is traveling in a separation lane (Lnz) away from the emergency vehicle lane, the specific action is permitted to continue, and the presence of the emergency vehicle is notified to the driver. The control device according to any one of claims 24 to 26.

28. The control device according to any one of claims 24 to 27, wherein, when the vehicle is traveling in the emergency vehicle lane, the continuation of the specific action is restricted and the continuation of the autonomous driving is restricted.

29. A control device according to any one of claims 24 to 28, wherein the sleep permission state continues when the vehicle is traveling in a lane different from the emergency vehicle lane and there are no other vehicles around the vehicle.

30. The control device according to any one of claims 24 to 29, wherein at least one of whether to continue the specific action and whether to continue the autonomous driving is determined based on the condition of the road on which the vehicle is traveling.

31. The control device according to claim 30, wherein when the lane in which the vehicle is traveling among the lanes included in the road is congested and a non-congested lane (Lnp) in which the emergency vehicle can travel is present on the road, the continuation of the specific action is restricted and the continuation of the automated driving is permitted.

32. A control device used in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, An approach determination unit (61) that determines whether or not an emergency vehicle (Qs) is approaching the vehicle, which is the vehicle, during an automatic driving period in which the automatic driving is performed; a continuation determination unit (62) that, when the approach determination unit determines that the emergency vehicle is approaching, determines whether or not to continue the automated driving and whether or not to continue a state in which the driver is permitted to perform a specific act other than driving, At least one of whether to continue the specific action and whether to continue the automated driving is determined based on the condition of the road on which the vehicle is traveling, A control device that restricts the continuation of the specific action and allows the continuation of the automated driving when the lane in which the vehicle is traveling among the lanes included in the road is congested and a non-congested lane (Lnp) in which the emergency vehicle can travel is present on the road.

33. The control device according to claim 32, wherein at least one of whether or not to continue the specific action and whether or not to continue the autonomous driving is determined based on the relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling.

34. 34. The control device according to claim 33, wherein, when the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, continuation of the automated driving is restricted, and the presence of the emergency vehicle is notified to the driver.

35. 35. The control device according to claim 33, wherein when the vehicle is traveling in a separation lane (Lnz) away from the emergency vehicle lane, continuation of the specific action is permitted and the presence of the emergency vehicle is notified to the driver.

36. The control device according to any one of claims 33 to 35, wherein, when the vehicle is traveling in the emergency vehicle lane, the continuation of the specific action is restricted and the continuation of the autonomous driving is restricted.

37. A control device according to any one of claims 33 to 36, wherein the sleep permission state continues when the vehicle is traveling in a lane different from the emergency vehicle lane and there are no other vehicles around the vehicle.

38. A control device according to any one of claims 30 to 37, wherein, when all lanes of the road are congested, continuation of the specific action is permitted and the presence of the emergency vehicle is notified to the driver.

39. A control program for use in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, At least one processing section (11) During the automatic driving period in which the automatic driving is performed, when an approach of an emergency vehicle (Qs) to the vehicle, which is the vehicle itself, is detected, it is determined whether or not the state in which the driver is permitted to perform a specific act other than driving continues (S155); When it is determined that the permission state of the specific act will not continue, the display of the content (CTV) provided to the driver is restricted (S158, S159); When the approach of the emergency vehicle to the vehicle is detected, it is further determined whether or not the automatic driving should be continued (S157). When it is determined that the autonomous driving will not continue, the display of the content is limited more than when it is determined that the autonomous driving will continue; Determine at least one of whether to continue the specific action and whether to continue the automated driving based on a relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling; When the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, the continuation of the specific action is restricted and the continuation of the automated driving is permitted. A control program that causes the device to execute a process including the above.

40. A control program for use in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, At least one processing section (11) During the automatic driving period in which the automatic driving is performed, when an approach of an emergency vehicle (Qs) to the vehicle, which is the vehicle itself, is detected, it is determined whether or not the state in which the driver is permitted to perform a specific act other than driving continues (S155); When it is determined that the permission state of the specific act will not continue, the display of the content (CTV) provided to the driver is restricted (S158, S159); When the approach of the emergency vehicle to the vehicle is detected, it is further determined whether or not the automatic driving should be continued (S157). When it is determined that the autonomous driving will not continue, the display of the content is limited more than when it is determined that the autonomous driving will continue; determining whether or not to continue the specific action and / or whether or not to continue the automated driving based on the condition of the road on which the vehicle is traveling; When a lane in which the vehicle is traveling among lanes included in the road is congested and a non-congested lane (Lnp) in which the emergency vehicle can travel is present on the road, the continuation of the specific action is restricted and the continuation of the automated driving is permitted. A control program that causes the device to execute a process including the above.

41. A control program for use in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, At least one processing section (51) During the automatic driving period in which the automatic driving is performed, it is determined whether or not an emergency vehicle (Qs) is approaching the vehicle (S101, S131), When it is determined that the emergency vehicle is approaching, whether or not to continue the automated driving and whether or not to continue the state in which the driver is permitted to perform a specific act other than driving are determined based on the relative position of the emergency vehicle lane (Lnq) in which the emergency vehicle is traveling (S106 to S109, S136 to S139); When the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, the continuation of the automated driving is restricted, and the presence of the emergency vehicle is notified to the driver. A control program that causes the device to execute a process including the above.

42. A control program for use in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, At least one processing section (51) During the automatic driving period in which the automatic driving is performed, it is determined whether or not an emergency vehicle (Qs) is approaching the vehicle (S101, S131), When it is determined that the emergency vehicle is approaching, it is determined whether or not the automated driving should be continued and whether or not the state in which the driver is permitted to perform a specific act other than driving should be continued (S106 to S109, S136 to S139); Determine at least one of whether to continue the specific action and whether to continue the automated driving based on a relative position of an emergency vehicle lane (Lnq) in which the emergency vehicle is traveling; When the vehicle is traveling in an adjacent lane (Lna) adjacent to the emergency vehicle lane, the continuation of the specific action is restricted and the continuation of the automated driving is permitted. A control program that causes the device to execute a process including the above.

43. A control program for use in a vehicle (Am) capable of performing automated driving without a driver's obligation to monitor the surroundings, At least one processing section (51) During the automatic driving period in which the automatic driving is performed, it is determined whether or not an emergency vehicle (Qs) is approaching the vehicle (S101, S131), When it is determined that the emergency vehicle is approaching, it is determined whether or not the automated driving should be continued and whether or not the state in which the driver is permitted to perform a specific act other than driving should be continued (S106 to S109, S136 to S139); determining whether or not to continue the specific action and / or whether or not to continue the automated driving based on the condition of the road on which the vehicle is traveling; When a lane in which the vehicle is traveling among lanes included in the road is congested and a non-congested lane (Lnp) in which the emergency vehicle can travel is present on the road, the continuation of the specific action is restricted and the continuation of the automated driving is permitted. A control program that causes the device to execute a process including the above.

Images