Vehicle control device, vehicle control method, and vehicle control program

Abstract

A vehicle control device according to the present invention is an HMI control device that is to be used in a vehicle that can travel by automatic drive that allows a driver that is sitting in a driver seat to sleep. The HMI control device perceives the approach of an emergency vehicle toward the vehicle. The HMI control device also perceives whether the driver is asleep. When an emergency vehicle has approached the vehicle while the driver is asleep during a period of automatic travel by automatic drive, the HMI control device executes restricted approach notification that uses a display to notify a passenger of the approach of the emergency vehicle while restricting awakening notification that encourages the driver to awaken.

Description

Vehicle control device and vehicle control method CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Patent Application No. 2024-34386 filed in Japan on March 6, 2024, and the contents of the original application are incorporated by reference in their entirety.

[0002] The disclosure of this specification relates to a vehicle control technology used in a vehicle capable of autonomous driving.

[0003] The automated driving assistance device disclosed in Patent Document 1 includes an emergency vehicle detection unit that detects the approach of an emergency vehicle. When the emergency vehicle detection unit detects the approach of an emergency vehicle, the automated driving assistance device notifies the driver of the approach of the emergency vehicle and prompts the driver to wake up using an awakening device.

[0004] Japanese Patent Application Laid-Open No. 2018-151208

[0005] In the automated driving system disclosed in Patent Document 1, the driver is not permitted to sleep. Meanwhile, development of automated driving systems that allow the driver to sleep has been progressing in recent years. In an emergency vehicle response during such automated driving that allows the driver to sleep, if the notification disclosed in Patent Document 1 is implemented, it becomes difficult for the driver to continue sleeping. As a result, the convenience of the automated driving system may be impaired.

[0006] The present disclosure aims to provide a vehicle control device and a vehicle control method that are capable of responding to an emergency vehicle in a manner that is convenient for passengers.

[0007] In order to achieve the above object, one disclosed aspect is a vehicle control device used in a vehicle that can be driven by autonomous driving and allows a passenger seated in the driver's seat to sleep, and includes an approach detection unit that detects the approach of an emergency vehicle to the vehicle, a state detection unit that detects whether the passenger is asleep or not, and a notification implementation unit that issues an emergency approach notification to the passenger of the approaching emergency vehicle, wherein the notification implementation unit is a vehicle control device that, when an emergency vehicle approaches the vehicle while the passenger is asleep during a period of autonomous driving by autonomous driving, limits the awakening notification to encourage the passenger to wake up, while implementing a limited approach notification to inform the passenger of the approaching emergency vehicle by display.

[0008] Another disclosed aspect is a vehicle control method used in a vehicle capable of traveling by autonomous driving in which a passenger seated in the driver's seat is allowed to sleep, the vehicle control method including the steps of detecting the approach of an emergency vehicle to the vehicle, detecting whether the passenger is asleep, and, if an emergency vehicle approaches the vehicle during an autonomous driving period in which the passenger is asleep, implementing a limited approach notification that notifies the passenger of the approach of the emergency vehicle by a display while limiting a wake-up notification to encourage the passenger to wake up.

[0009] In these embodiments, even if an approaching emergency vehicle is detected, the wake-up notification to wake the occupant is limited, so the occupant can continue sleeping. In addition, since the approaching emergency vehicle is notified to the occupant by a display, even if the occupant wakes up suddenly, the occupant can understand from the display that the emergency vehicle is approaching. As a result, it is possible to respond to an emergency vehicle in a convenient manner for the occupant.

[0010] It should be noted that the reference numbers in parentheses in the claims merely indicate an example of the correspondence with the specific configurations in the embodiments described below, and do not limit the technical scope in any way. Furthermore, claims not explicitly stated in the claims may be combined together if no particular problems arise in the combination.

[0011] FIG. 1 is a diagram showing an overall view of an in-vehicle network including an HMI control device and an autonomous driving ECU according to a first embodiment of the present disclosure. FIG. 2 is a block diagram showing details of the autonomous driving ECU together with related configuration. FIG. 3 is a block diagram showing details of the HMI control device together with related configuration. FIG. 4 is a diagram showing a list of notification devices used for notifying of the approach of an emergency vehicle. FIG. 5 is a flowchart showing details of an emergency vehicle notification process performed by the HMI control device. FIG. 6 is a flowchart showing details of an outside vehicle notification process performed by the HMI control device. FIG. 7 is a flowchart showing details of an awakening response process performed by the HMI control device. FIG. 8 is a flowchart showing details of an emergency vehicle notification process performed by the HMI control device according to a second embodiment of the present disclosure. FIG. 9 is a flowchart showing details of an emergency vehicle notification process performed by the HMI control device according to a third embodiment of the present disclosure.

[0012] Hereinafter, several embodiments will be described with reference to the drawings. Note that corresponding components in each embodiment are given the same reference numerals, and redundant description may be omitted. When only a portion of the configuration is described in each embodiment, the configuration of another embodiment described previously can be applied to the remaining portion of the configuration. Furthermore, in addition to the combinations of configurations explicitly stated in the description of each embodiment, configurations of several embodiments can also be partially combined together even if not explicitly stated, as long as there is no particular problem with the combination.

[0013] (First embodiment) The functions of a vehicle control device according to a first embodiment of the present disclosure are realized by an HMI (Human Machine Interface) control device 100 shown in Fig. 1. The HMI control device 100 is an interface control device used in a vehicle (hereinafter referred to as host vehicle Am). The HMI control device 100 configures an HMI (Human Machine Interface) system 10 of the host vehicle Am together with a plurality of input / output devices, etc. The HMI system 10 has an input interface function that accepts operations by an occupant such as a driver of the host vehicle Am, and an output interface function that presents information to the driver.

[0014] The HMI control device 100 is communicatively connected to a communication bus 99 of an in-vehicle network 1 mounted on the host vehicle Am. The HMI control device 100 is one of a plurality of nodes provided in the in-vehicle network 1. An autonomous driving ECU (Electronic Control Unit) 50 is connected to the communication bus 99 of the in-vehicle network 1.

[0015] The autonomous driving ECU 50 is installed in the host vehicle Am together with the HMI control device 100. By installing the autonomous driving ECU 50, the host vehicle Am becomes an autonomous driving vehicle or an autonomously driven vehicle equipped with an autonomous driving function. The autonomous driving levels in this disclosure are based on standards defined by the Society of Automotive Engineers. In this disclosure, autonomous driving control by the autonomous driving ECU 50 at autonomous driving level 3 or higher is referred to as "autonomous driving control."

[0016] Level 3 autonomous driving is eyes-off autonomous driving, which means that there is no need to monitor the area around the vehicle and there is no obligation to monitor the surroundings. The autonomous driving ECU 50 can perform level 4 fully autonomous driving, in which the system performs all driving tasks under certain conditions. Level 4 autonomous driving is brain-off autonomous driving, in which there is essentially no request for the driver to take over driving. The autonomous driving ECU 50 may also be capable of performing level 5 fully autonomous driving, in which the system performs all driving tasks under all conditions. Level 5 autonomous driving is driverless autonomous driving, which does not require a driver to be on board.

[0017] During an autonomous driving period in which the host vehicle Am is driven by autonomous driving control of autonomous driving level 3 or higher by the autonomous driving ECU 50, the driver may be permitted to perform a specific action other than driving as specified in advance (hereinafter referred to as a second task). The driver is legally permitted to perform the second task until a take-over request (TOR) is issued by the autonomous driving ECU 50 in cooperation with the HMI control device 100. For example, actions such as watching entertainment content such as videos, operating a device such as a smartphone, and eating are conceivable as second tasks. Furthermore, during an autonomous driving period in which the host vehicle Am is driven by autonomous driving control of autonomous driving level 4 or higher, the passenger (driver) seated in the driver's seat is permitted to sleep.

[0018] The communication bus 99 is further connected to the driver monitor 29, the surroundings monitoring sensor 30, the locator 35, the navigation ECU 38, the in-vehicle communication device 39, the cruise control ECU 40, the body ECU 43, and the exterior display 45. The multiple nodes connected to the communication bus 99 can communicate with each other. Specific nodes among these ECUs may be electrically connected to each other directly and can communicate without going through the communication bus 99.

[0019] The driver monitor 29 continuously monitors the state of a passenger seated in the driver's seat. The driver monitor 29 includes a near-infrared light source, a near-infrared camera, and a control unit for controlling them. The driver monitor 29 is installed, for example, on the top surface of the steering column or the top surface of the instrument panel, with the near-infrared camera facing the headrest of the driver's seat. The driver monitor 29 uses the near-infrared camera to capture an image of the driver's head illuminated with near-infrared light from the near-infrared light source. The image captured by the near-infrared camera is analyzed by the control unit. The control unit extracts information such as the driver's facial orientation, eye position, gaze direction, and eye opening degree from the captured image. The control unit generates alertness information indicating whether the driver is asleep based on the extracted information from the captured image. The driver monitor 29 provides the facial orientation information, eye position information, gaze direction information, eye opening degree information, and other information extracted by the control unit, as well as alertness information based on this information, to the HMI control device 100, etc., as driver status information.

[0020] The perimeter monitoring sensor 30 is an autonomous sensor mounted on the host vehicle Am and monitors the environment surrounding the host vehicle Am. The perimeter monitoring sensor 30 is capable of detecting moving and stationary objects within a detection range around the host vehicle. The perimeter monitoring sensor 30 provides detection information of objects around the host vehicle to the autonomous driving ECU 50, etc. The perimeter monitoring sensor 30 includes a camera unit 31, a millimeter-wave radar 32, a lidar 33, and an exterior acoustic sensor 34. The perimeter monitoring sensor 30 may further include a sonar sensor, etc.

[0021] The camera unit 31 includes a front camera module, a rear camera module, a left side camera module, and a right side camera module. The camera unit 31 includes multiple camera modules, so that it can capture images of the entire surroundings of the vehicle Am. The camera unit 31 provides the autonomous driving ECU 50 with image data captured by each camera module or analysis information of the image data as detection information.

[0022] The millimeter-wave radar 32 emits millimeter waves or quasi-millimeter waves toward the surroundings of the vehicle. The millimeter-wave radar 32 receives reflected waves from moving objects, stationary objects, and the like, and generates detection information, which is then provided to the autonomous driving ECU 50.

[0023] The lidar 33 emits laser light toward the surroundings of the vehicle. The lidar 33 receives the laser light reflected by moving objects and stationary objects within the irradiation range, and generates detection information (point cloud data) that is then provided to the autonomous driving ECU 50.

[0024] The exterior acoustic sensor 34 is primarily composed of a microphone element that converts sound into an electrical signal. The exterior acoustic sensor 34 is held on the exterior structure of the host vehicle Am with the sound collection surface of the microphone element facing the exterior structure. Multiple exterior acoustic sensors 34 are provided on the front, rear, left and right sides, and ceiling of the host vehicle Am. The exterior acoustic sensors 34 provided at each location can detect incoming sounds coming from all around the host vehicle Am and collect environmental sounds around the host vehicle Am. For example, when an emergency vehicle is approaching the host vehicle Am, the exterior acoustic sensor 34 collects environmental sounds including a specific siren sound sounded by the emergency vehicle and the sound from a loudspeaker. The exterior acoustic sensor 34 provides the autonomous driving ECU 50 with sound data of the environmental sounds generated by each microphone element or analysis information of the sound data (e.g., information indicating that a siren sound has been extracted) as detection information. In addition, emergency vehicles in this disclosure include police vehicles, fire engines, ambulances, etc., and unless otherwise specified, refer to emergency vehicles traveling for emergency purposes and sounding a siren.

[0025] The locator 35 includes a GNSS (Global Navigation Satellite System) receiver, an inertial sensor, etc. The locator 35 sequentially determines the position and traveling direction of the host vehicle Am by combining positioning signals received from multiple positioning satellites by the GNSS receiver, measurement results from the inertial sensor, and vehicle speed information output to the communication bus 99. The locator 35 sequentially outputs position information and direction information of the host vehicle Am based on the positioning results to the communication bus 99 as locator information.

[0026] The locator 35 also has a map database that stores map data. The map database is primarily composed of a large-capacity storage medium that stores a large amount of three-dimensional map data and two-dimensional map data. The three-dimensional map data is a so-called high-definition (HD) map and includes road information necessary for autonomous driving. Specifically, the three-dimensional map data includes three-dimensional road shape information and detailed information about each lane. The locator 35 can update the three-dimensional map data and two-dimensional map data to the latest information through external communication via the on-board communication device 39. The locator 35 reads map data about the area around the current location from the map database and provides it to the autonomous driving ECU 50, the HMI control device 100, etc., along with locator information.

[0027] The navigation ECU 38 acquires information about a destination specified by a passenger such as a driver based on operation information acquired from the HMI control device 100. The navigation ECU 38 acquires vehicle position information and direction information from the locator 35, and sets a route from the current position to the destination. The navigation ECU 38 provides route information indicating the set route to the destination to the autonomous driving ECU 50, the HMI control device 100, etc. The navigation ECU 38 works in cooperation with the HMI control device 100 to provide route guidance to the destination by combining screen displays and voice messages, etc., and notifying the driver of the direction of travel of the vehicle Am at intersections, branching points, etc.

[0028] Here, a user terminal such as a smartphone or tablet may be connected to the in-vehicle network 1 or the HMI control device 100. Such a user terminal may provide the autonomous driving ECU 50 with information such as vehicle position information, direction information, and map data in place of the locator 35. Furthermore, the user terminal may provide the autonomous driving ECU 50 and the HMI control device 100 with information such as route information to a destination in place of the navigation ECU 38. Furthermore, the user terminal may play video content in place of the HMI system 10 or together with the HMI system 10.

[0029] The in-vehicle communication device 39 is an external communication unit mounted on the host vehicle Am. The in-vehicle communication device 39 functions as a V2X (Vehicle to Everything) communication device. The in-vehicle communication device 39 transmits and receives information via wireless communication between roadside devices installed on the side of the road and in-vehicle communication devices 39 of other vehicles. For example, the in-vehicle communication device 39 acquires detection information of stopped vehicles, parked vehicles, pedestrians, cyclists, etc. from the roadside devices. The in-vehicle communication device 39 may be capable of acquiring emergency vehicle approach information indicating the approach of an emergency vehicle from the roadside devices or other vehicles. The in-vehicle communication device 39 provides the detection information and emergency vehicle approach information acquired by reception as received traffic information to the navigation ECU 38, the autonomous driving ECU 50, the HMI control device 100, etc.

[0030] The cruise control ECU 40 is an electronic control device that mainly includes a microcontroller. The cruise control ECU 40 has at least the functions of a brake control ECU, a drive control ECU, and a steering control ECU. The cruise control ECU 40 continuously controls the braking force of each wheel, the output of the powertrain, and the steering angle based on either an operation command based on the driver's driving operation or a control command from the automatic driving ECU 50.

[0031] The body ECU 43 is an electronic control device that mainly includes a microcontroller. The body ECU 43 has a function of controlling various on-board devices (e.g., lighting devices) installed in the host vehicle Am. The lighting devices include turn signals and emergency flashers (hereinafter, hazard lights 44). The body ECU 43 starts flashing the hazard lights 44 based on the detection of a user operation input to a hazard switch provided on the instrument panel or the like. During the autonomous driving period, the body ECU 43 causes the hazard lights 44 to flash based on a control command from the autonomous driving ECU 50.

[0032] The exterior display 45 is provided on the outer surface of the host vehicle Am. The exterior display 45 is an exterior alarm that displays information to the outside of the host vehicle Am. The exterior display 45 is configured to be able to display text. The host vehicle Am is equipped with at least one of a front display, a side display, a rear display, etc. as the exterior display 45. The exterior display 45 notifies pedestrians, cyclists, drivers of other vehicles, etc. around the host vehicle Am of the operating status of the autonomous driving of the host vehicle Am and various information grasped by the host vehicle Am. Note that the host vehicle Am may also be provided with an exterior speaker as an exterior alarm that plays an alarm sound or a voice message to the outside of the vehicle. The exterior display 45 may also be a light-emitting device that indicates information by changing the light-emitting pattern.

[0033] 1 and 2 is a computer that mainly includes a processing unit 51, a RAM 52, a storage unit 53, an input / output interface 54, and a control circuit that includes a bus connecting these components. The processing unit 51 accesses the RAM 52 to execute various processes (instructions) for implementing the autonomous driving control method and vehicle control method according to the present disclosure. The storage unit 53 stores various programs (autonomous driving control program, vehicle control program, etc.) executed by the processing unit 51. By executing the programs by the processing unit 51, the autonomous driving ECU 50 is configured with an information linkage unit 61, an environment recognition unit 62, an action determination unit 63, a control execution unit 64, etc. as functional units for implementing the autonomous driving function and vehicle control function.

[0034] The information linking unit 61 provides information to an information linking unit 82 (see FIG. 3 ) of the HMI control device 100 and acquires information from the information linking unit 82. Through the linking of these information linking units 61, 82, the autonomous driving ECU 50 and the HMI control device 100 share the information that they have acquired.

[0035] The information linking unit 61 generates control status information indicating the operation status of the autonomous driving function, and provides the generated control status information to the information linking unit 82. The information linking unit 61 provides the information linking unit 82 with the recognition results of the surroundings of the host vehicle by the environment recognition unit 62. Furthermore, the information linking unit 61 enables the HMI control device 100 to issue a notification synchronized with the operation status of the autonomous driving function by outputting a request to issue a notification to the information linking unit 82. The information linking unit 61 acquires operation information and control requests related to autonomous driving from the information linking unit 82.

[0036] The environment recognition unit 62 recognizes the driving environment of the host vehicle Am by combining the locator information and map data acquired from the locator 35 with the detection information acquired from the perimeter monitoring sensor 30. The environment recognition unit 62 may use the detection information received by the in-vehicle communication device 39 to recognize the driving environment. The environment recognition unit 62 acquires road information related to the road on which the host vehicle Am is traveling or the road on which the host vehicle Am is scheduled to travel, and grasps the size, relative position, relative speed, etc. of dynamic targets located on the road around the host vehicle. The environment recognition unit 62 provides the recognition result of the driving environment including the grasped road information and target information, and route information acquired from the navigation ECU 38 to the action determination unit 63.

[0037] The environment recognition unit 62 detects the approach of an emergency vehicle based on acoustic recognition technology using detection information from the exterior acoustic sensor 34. The environment recognition unit 62 further detects the type, relative position, and movement direction of the emergency vehicle whose approach has been recognized. Specifically, the environment recognition unit 62 determines whether the emergency vehicle is traveling on the same road and in the same direction as the host vehicle Am. Furthermore, the environment recognition unit 62 estimates whether the host vehicle Am is obstructing the travel of the emergency vehicle. As an example, if an emergency vehicle approaching the host vehicle Am from behind is unable to overtake (or pass) the host vehicle Am and continues to remain behind the host vehicle Am, the environment recognition unit 62 estimates that the host vehicle Am is obstructing the travel of the emergency vehicle. Specifically, the environment recognition unit 62 estimates that the host vehicle Am is obstructing the travel of the emergency vehicle when a voice announcement indicating that the vehicle should give way is detected or when the approaching state of the emergency vehicle continues for a predetermined time (e.g., approximately 10 to several tens of seconds) or more. The environment recognition unit 62 provides the recognition result related to the emergency vehicle to the HMI control device 100 via the information linking unit 61 .

[0038] When the autonomous driving ECU 50 has control of driving operations, the behavior determination unit 63 determines the behavior of the host vehicle Am based on the recognition result of the driving environment by the environment recognition unit 62. The behavior determination unit 63 generates a planned driving line along which the host vehicle Am will travel, and outputs the generated planned driving line to the control execution unit 64. The behavior determination unit 63 has a control switching unit 77 and an evacuation control unit 78 as sub-functional units.

[0039] The control switching unit 77 cooperates with the cruise control ECU 40 to switch the control state of the host vehicle Am between autonomous driving and manual driving. Additionally, the control switching unit 77 switches the autonomous driving level of the autonomous driving control implemented by the host vehicle Am. For example, when the information linking unit 61 detects an input of a transition operation to level 3 or level 4, the control switching unit 77 switches the cruise control state of the host vehicle Am from manual driving or driving assistance control of level 2 or lower to autonomous driving control of level 3 or higher. Furthermore, when the control switching unit 77 determines to terminate the autonomous driving control, it switches the cruise control state from autonomous driving control to driving assistance control or manual driving. Additionally, during an autonomous driving period of autonomous driving level 4 or higher, if the driver is not in a state suitable for manual driving, the control switching unit 77 restricts (prohibits) switching from autonomous driving to manual driving (override). In this case, the control switching unit 77 cooperates with the cruise control ECU 40 to prohibit the driver from operating the driving operation by not reflecting the driving operation input to the driving operator on the behavior of the host vehicle Am.

[0040] The avoidance control unit 78 performs adaptive travel control to accommodate the emergency vehicle when the environment recognition unit 62 detects an emergency vehicle approaching the host vehicle Am. The avoidance control unit 78 controls the behavior of the host vehicle Am to avoid the emergency vehicle while grasping the status of other vehicles traveling around the host vehicle Am through adaptive travel control. The avoidance control unit 78 performs at least one of stop control, low-speed travel control, departure control, and reroute control as adaptive travel control. The stop control is control to stop the host vehicle Am at a position at least a predetermined distance away from the emergency vehicle. The low-speed travel control is control to suppress the travel speed of the host vehicle Am. The departure control is control to change lanes to cause the host vehicle Am to leave the current lane when the host vehicle Am is traveling in the same lane as the emergency vehicle. The reroute control is control to reset the planned travel route of the host vehicle Am to avoid the estimated travel route of the emergency vehicle. The change of the planned driving route by the reroute control may be realized by a process in which the action determination unit 63 changes the planned driving line, or may be realized by a process in which a reroute request is output to the navigation ECU 38. Information indicating the start and end of such adaptive driving control and information indicating the type of adaptive driving control currently being performed are provided to the HMI control device 100 from the information linking unit 61 as part of the control status information.

[0041] When the autonomous driving ECU 50 has control of driving operations, the control execution unit 64 cooperates with the cruise control ECU 40 to execute acceleration / deceleration control, steering control, and the like of the host vehicle Am in accordance with the planned driving line generated by the action determination unit 63. Specifically, the control execution unit 64 generates control commands based on the planned driving line and sequentially outputs the generated control commands to the cruise control ECU 40.

[0042] [Configuration of HMI Control Device] The HMI control device 100 shown in Figures 1 to 3 constitutes an HMI system 10 together with a plurality of display devices, an audio device 24, ambient light 25, an operation device 26, etc. The display devices present information to the driver's vision by displaying images, etc. The display devices include a meter display 21, a center information display (hereinafter referred to as CID) 22, and a head-up display (hereinafter referred to as HUD) 23, etc. The CID 22 has a touch panel function and detects touch operations on the display screen by the driver, etc.

[0043] The audio device 24 has multiple speakers installed in the vehicle cabin. The audio device 24 reproduces, for example, notification sounds or voice messages through the speakers. The ambient lights 25 are provided on the instrument panel, center console, steering wheel, door trim, etc. The ambient lights 25 present information to the driver's peripheral vision through ambient displays that change the emitted light color.

[0044] The operation device 26 is an input unit that accepts user operations by the driver, etc. User operations related to, for example, activation and deactivation of the autonomous driving function and setting of a destination for route guidance are input to the operation device 26. The operation device 26 includes a steering switch provided on the spokes of the steering wheel, an operation lever provided on the steering column, and a voice input device that recognizes what the passengers are saying.

[0045] The HMI control device 100 functions as a presentation control device and comprehensively controls the presentation of information using a plurality of display devices, an audio device 24, an ambient light 25, etc. The HMI control device 100 presents information related to autonomous driving in cooperation with the autonomous driving ECU 50. For example, when the autonomous driving ECU 50 plans to end autonomous driving control, the HMI control device 100 issues a notification requesting the driver to take over driving.

[0046] The HMI control device 100 is a computer mainly including a processing unit 11, a RAM 12, a storage unit 13, an input / output interface 14, and a control circuit including a bus connecting these units. The processing unit 11 accesses the RAM 12 to execute various processes (instructions) for implementing the presentation control method and vehicle control method according to the present disclosure. The storage unit 13 stores various programs (such as a presentation control program and a vehicle control program) executed by the processing unit 11. As the processing unit 11 executes the programs, an information acquisition unit 81, an information linkage unit 82, a presentation control unit 88, and the like are configured in the HMI control device 100 as functional units for implementing the presentation control function and the vehicle control function.

[0047] The information acquisition unit 81 acquires vehicle information (e.g., vehicle speed information, etc.) indicating the state of the host vehicle Am from the communication bus 99. The information acquisition unit 81 determines whether the host vehicle Am is stopped or not based on the vehicle speed information. In addition, the information acquisition unit 81 acquires operation information indicating the content of the driver's operation from the CID 22, the operation device 26, etc. Furthermore, the information acquisition unit 81 acquires driver status information from the driver monitor 29. The information acquisition unit 81 continuously determines whether the driver is awake or asleep based on the alertness information included in the driver status information.

[0048] During the autonomous driving period of autonomous driving level 4 or higher, the passenger sitting in the driver's seat corresponds to the driver for convenience. The driver's seat is a seat suitable for operating the driving operation parts such as the accelerator pedal, brake pedal, and steering wheel, and is located near the driving operation parts.

[0049] The information linking unit 82 links with the information linking unit 61 of the automatic driving ECU 50, and enables information sharing between the automatic driving ECU 50 and the HMI control device 100. The information linking unit 82 acquires, from the automatic driving ECU 50, an implementation request for presenting information related to the automatic driving function, recognition results of the surroundings of the vehicle, control status information indicating the operating state of the automatic driving function, etc. The information linking unit 82 provides the acquired implementation request, recognition results, control status information, etc. to the presentation control unit 88. The information linking unit 82 provides, to the automatic driving ECU 50, information related to automatic driving from among the operation information grasped by the information acquisition unit 81. The information linking unit 82 has a control grasping unit 83 and a request output unit 84 as sub-functional units for information linking with the automatic driving ECU 50.

[0050] The control recognition unit 83 recognizes the autonomous driving level of the driving control implemented by the autonomous driving ECU 50 based on the control status information acquired by the information linkage unit 82. The control recognition unit 83 recognizes the approach of an emergency vehicle to the host vehicle Am based on the recognition results of the surroundings of the host vehicle acquired by the information linkage unit 82. The control recognition unit 83 further recognizes whether the emergency vehicle is traveling in the same direction as the host vehicle Am based on the recognition results. In the following description, a situation in which the approach of an emergency vehicle is recognized, but it is unclear whether the emergency vehicle is traveling in the same direction, is referred to as a "first situation." Furthermore, a situation in which the approach of an emergency vehicle is recognized and it is further recognized that the emergency vehicle is traveling in the same direction is referred to as a "second situation."

[0051] When the approach of an emergency vehicle is recognized, the control recognition unit 83 recognizes, based on the recognition result, the possibility that the emergency vehicle will not be able to pass by the host vehicle Am, in other words, whether the host vehicle Am is obstructing the travel of the emergency vehicle. In addition, when the approach of an emergency vehicle is recognized, the control recognition unit 83 recognizes, based on the control status information, whether adaptive cruise control is being performed by the evacuation control unit 78. When adaptive cruise control is being performed, the control recognition unit 83 further recognizes the type of adaptive cruise control being performed.

[0052] The request output unit 84 outputs control requests related to autonomous driving to the autonomous driving ECU 50. Specifically, when the driver is asleep during an autonomous driving period of autonomous driving level 4 or higher, the request output unit 84 outputs a control request to the autonomous driving ECU 50 requesting prohibition of transition to manual driving. The request output unit 84 continues to output a control request to the autonomous driving ECU 50 requesting prohibition of transition to manual driving even after the driver changes from asleep to awake. As described above, the request output unit 84 cooperates with the control switching unit 77 to prohibit (restrict) manual driving by a driver who is asleep and a driver who has just woken up. The request output unit 84 continues the state in which manual driving is restricted until the driver who has become awake assumes a position (driving posture) that allows him or her to take over driving, or until a preset time (e.g., approximately 10 seconds) has elapsed since the driver became awake. The request output unit 84 or the driver monitor 29 determines that the driver is ready to take over driving when the driver performs an action such as gripping the steering wheel or returning the driver's seat to its original position.

[0053] If the driver awakens during the period in which the evacuation control unit 78 is performing adaptive cruise control in response to an emergency vehicle, the request output unit 84 temporarily suspends the adaptive cruise control that is currently being performed. The request output unit 84 outputs a control request to the autonomous driving ECU 50 to request suspension of the adaptive cruise control. Based on the control request, the evacuation control unit 78 suspends the stop control, low-speed cruise control, and departure control that are currently being performed. Specifically, the evacuation control unit 78 performs control such as temporarily suspending lateral movement and temporarily suspending deceleration and continuing to travel at the current vehicle speed.

[0054] The presentation control unit 88 comprehensively provides information to the driver using each display device, the audio device 24, and the ambient light 25 (hereinafter, information presentation devices). The presentation control unit 88 provides content and presents information according to the operating state of autonomous driving based on the implementation request, recognition result, and control status information provided from the information linkage unit 82. When the presentation control unit 88 determines that autonomous driving control of autonomous driving level 3 or higher is being implemented, it enables the playback of video content, etc. When the end of autonomous driving control is scheduled, the presentation control unit 88 requests the driver to take over driving, etc. Furthermore, when the information linkage unit 82 determines that an emergency vehicle is approaching the host vehicle Am, the presentation control unit 88 performs an emergency approach alert to inform passengers such as the driver of the approaching emergency vehicle.

[0055] The presentation control unit 88 issues an external alert using the hazard lights 44 and the exterior display 45. In a first situation in which the approach of an emergency vehicle to the host vehicle Am is recognized, the presentation control unit 88 uses the exterior display 45 to notify vehicles outside the vehicle of the approach of the emergency vehicle. If the host vehicle Am is equipped with multiple exterior displays 45, the presentation control unit 88 notifies following vehicles of the approach of the emergency vehicle using at least the rear display. Additionally, in a second situation in which it is recognized that the emergency vehicle is traveling in the same direction as the host vehicle Am, the presentation control unit 88 not only issues an external alert using the exterior display 45 but also starts flashing the hazard lights 44 of the host vehicle Am. Note that the flashing of the hazard lights 44 may be started when the evacuation control unit 78 starts response driving control.

[0056] [Presenting Information About Approaching Emergency Vehicles at Autonomous Driving Level 4 or Higher] In the autonomous vehicles described so far, the driver in the driver's seat is allowed to sleep during the period of autonomous driving at autonomous driving level 4 or higher. Even if an emergency vehicle approaches the host vehicle Am, the driver can respond to the emergency vehicle through the adaptive cruise control implemented by the autonomous driving ECU 50. Therefore, if the emergency approach notification that notifies the driver of the approach of an emergency vehicle is implemented by voice using the audio device 24, the driver will be urged to wake up even when there is no need to be awake.

[0057] Therefore, the presentation control unit 88 performs a limited approach notification as a presentation of approach information when an emergency vehicle approaches the host vehicle Am while the driver is asleep during an automated driving period with automated driving at level 4 or higher. The limited approach notification is an emergency approach notification that is performed only when the driver is asleep. While a normal emergency approach notification is a notification indicating the approach of an emergency vehicle by both display and audio, the limited approach notification is a notification that indicates the approach of an emergency vehicle to the occupants by display only, out of the two methods of display and audio. Unlike a normal emergency approach notification, the limited approach notification limits (cancels) the awakening notification that encourages the occupants to wake up, specifically, the audio notification indicating the approach of an emergency vehicle.

[0058] In the limited approach notification, a text message such as "An emergency approach is approaching" and / or a still image or animation indicating the approach of the emergency approach are displayed by at least one display device. In addition, when the countermeasure driving control by the evacuation control unit 78 starts, at least one display device displays at least one still image or animation indicating the content (type) of the countermeasure driving control being performed.

[0059] More specifically, when an emergency vehicle approaches the host vehicle Am during an automated driving period at automated driving level 3 or lower, the presentation control unit 88 performs a normal emergency approach warning using both a visual and an audio signal, regardless of the driver's state of alertness (see FIG. 4, Lv3 and lower). Similarly, during a manual driving period, the presentation control unit 88 performs an emergency approach warning using both a visual and an audio signal.

[0060] On the other hand, in a first situation in which an approaching emergency vehicle to the vehicle Am is detected during an autonomous driving period using autonomous driving at level 4 or higher, the presentation control unit 88 provides a limited approach notification by display only, regardless of the driver's state of alertness (see Figure 4, Lv4, first situation).

[0061] In a second situation in which an emergency vehicle is detected traveling in the same direction as the host vehicle Am, the presentation control unit 88 changes the notification mode based on the driver's wakefulness (see FIG. 4 Lv4, second situation). If the driver is wakeful in the second situation, the presentation control unit 88 performs a normal emergency approach notification using both a visual and an audio message. On the other hand, if the driver is asleep in the second situation, the presentation control unit 88 continues the limited approach notification using only a visual.

[0062] Note that the awakening alert implemented in the normal emergency approach alert is not limited to the above-described audio alert. For example, the awakening alert may be an action such as vibrating the seat, forcibly returning the backrest of the driver's seat that has been reclined backward, generating acceleration that shakes the vehicle Am in the longitudinal direction, generating air conditioning airflow to promote awakening, or opening a window on the side of the driver's seat. In the limited approach alert, the implementation of these awakening alerts is stopped. Furthermore, the awakening alert does not have to be completely stopped in the limited approach alert, but may be implemented in a limited state. For example, in the limited approach alert, an audio message indicating the approach of an emergency vehicle may be played at a volume much lower than that of the normal emergency approach alert.

[0063] Hereinafter, the details of each process performed by the HMI control device 100 to respond to the approach of an emergency vehicle will be described based on FIGS. 5 to 7 and with reference to FIGS. 1 to 4.

[0064] <Emergency vehicle notification process> In the emergency vehicle notification process shown in Fig. 5, an emergency approach notification or a limited approach notification is performed based on the approach of an emergency vehicle. The emergency vehicle notification process is started by the HMI control device 100 based on the start of autonomous driving at level 4 or higher being recognized by the control recognition unit 83. The emergency vehicle notification process is repeatedly performed until the end of autonomous driving at level 4 or higher is recognized by the information linkage unit 82.

[0065] In S11 of the emergency vehicle notification processing, the control recognition unit 83 recognizes that an emergency vehicle is approaching the host vehicle Am based on the recognition result obtained from the automatic driving ECU 50. If the control recognition unit 83 has not recognized the approach of an emergency vehicle (S11: NO), the current emergency vehicle notification processing is temporarily terminated. On the other hand, if the control recognition unit 83 has recognized the approach of an emergency vehicle (S11: YES), in S12 the control recognition unit 83 further recognizes whether the emergency vehicle is traveling in the same direction as the host vehicle Am based on the recognition result obtained from the automatic driving ECU 50.

[0066] If it is unclear whether the emergency vehicle is traveling in the same direction as the host vehicle Am (S12: NO, first situation), the presentation control unit 88 performs a limited approach notification in S15, which notifies the passenger (driver) of the approach of the emergency vehicle only by display. The presentation control unit 88 also performs a limited approach notification in S15 when the emergency vehicle is not traveling in the same direction as the host vehicle Am. In these limited approach notifications, audio notifications are stopped (limited).

[0067] On the other hand, if an emergency vehicle is traveling in the same direction as the host vehicle Am (S12: YES, second situation), the information acquisition unit 81 acquires driver status information (alertness information) in S13. The information acquisition unit 81 determines (determines) whether the driver is asleep or not based on the acquired latest driver status information in S14. If the driver is asleep (S14: YES), the presentation control unit 88 performs a limited approach notification in S15. On the other hand, if the driver is not asleep but is awake (S14: NO), the presentation control unit 88 performs a normal emergency approach notification in S16. In this emergency approach notification, the approach of an emergency vehicle is notified to the driver both by voice and by display.

[0068] In S17, after the start of the limited approach notification, the control understanding unit 83 estimates, based on the recognition result, whether or not the host vehicle Am is obstructing the emergency vehicle. If it is not estimated that the host vehicle Am is obstructing the emergency vehicle (S17: NO), the current emergency vehicle notification process is temporarily terminated. On the other hand, if it is estimated that the host vehicle Am is obstructing the emergency vehicle (S17: YES), the presentation control unit 88 performs a wake-up notification in S18. In this case, the presentation control unit 88 prompts the sleeping driver to wake up by at least audio. Furthermore, in S19 after the start of the wake-up notification, the presentation control unit 88 restarts the display indicating the approaching emergency vehicle from the beginning. The presentation control unit 88 temporarily ends the display of the series of animations started in S15, and then displays the animations from the beginning in S19 to once again notify the awakened driver of the approaching emergency vehicle.

[0069] 6 is a process for providing an external notification of an approaching emergency vehicle. The external notification process is started by the HMI control device 100 when the control recognition unit 83 recognizes the start of autonomous driving control of level 3 or higher. The external notification process is repeatedly performed until the information linking unit 82 recognizes the end of autonomous driving control.

[0070] In steps S31 and S32 of the vehicle exterior notification process, similar to steps S11 and S12 of the emergency vehicle notification process (see FIG. 5), it is determined whether or not an approaching emergency vehicle has been detected and whether or not the emergency vehicle is traveling in the same direction as the host vehicle Am. If the approaching emergency vehicle has not been detected (S31: NO), the vehicle exterior notification process is temporarily terminated.

[0071] On the other hand, in the first situation where the approach of an emergency vehicle is detected but it is unclear whether the emergency vehicle is traveling in the same direction as the host vehicle Am (S31: YES, S32: NO), the presentation control unit 88 notifies the outside of the vehicle of the approach of the emergency vehicle in S33. In this case, the presentation control unit 88 uses at least the rear display to notify other vehicles around the host vehicle, particularly following vehicles, of the approach of the emergency vehicle.

[0072] In a second situation in which the approach of an emergency vehicle is detected and the emergency vehicle is traveling in the same direction as the host vehicle Am (S31: YES, S32: YES), the presentation control unit 88 cooperates with the body ECU 43 in S34 to start flashing the hazard lights 44. Furthermore, the presentation control unit 88 increases the number of exterior displays 45 that provide an emergency vehicle approach warning in S35. In this case, the presentation control unit 88 displays a text message or the like indicating the approach of an emergency vehicle on the front display and side displays in addition to the rear display.

[0073] 7 , information is presented and driving restrictions are imposed to accommodate a driver who has woken up after the start of adaptive driving control by the evacuation control unit 78. The awakening response process is started by the HMI control device 100 when the start of adaptive driving control is recognized by the control recognition unit 83.

[0074] In S51 of the awakening response processing, the information acquisition unit 81 acquires driver status information. Based on the acquired latest driver status information, the information acquisition unit 81 determines whether the driver's state has changed from a sleeping state to an awakening state in S52. If the state has changed from a sleeping state to an awakening state (S52: YES), the information acquisition unit 81 determines in S53 based on the driver status information whether the driver is in an appropriate driving posture immediately after awakening.

[0075] If the awake driver is in an appropriate driving posture (S53: YES), the request output unit 84 allows the driver to drive in S54. In this case, the request output unit 84 outputs a control request to the control switching unit 77 to cancel the prohibition on transition to manual driving. This makes it possible for the driver to take over driving. On the other hand, if the awake driver is not in an appropriate driving posture (S53: NO), the request output unit 84 prohibits the driver from driving in S55. In this case, the request output unit 84 outputs a control request to the control switching unit 77 to prohibit manual driving.

[0076] In S56, the presentation control unit 88 notifies the driver by voice that adaptive cruise control is being performed by the evacuation control unit 78. In addition to the voice notification, the presentation control unit 88 also displays a display to indicate to the driver the type of adaptive cruise control being performed.

[0077] In S57, the request output unit 84 outputs a control request to the autonomous driving ECU 50 to temporarily suspend the adaptive cruise control that is currently being performed. As a result, the evacuation control unit 78 suppresses the adaptive cruise control for a predetermined time period immediately after the driver wakes up so that the driver can easily grasp the situation.

[0078] After the control request is output in S57, or if there is no change from a sleep state to an awake state (S52: NO), the control grasping unit 83 determines in S58 based on the control status information whether the adaptive driving control has ended. If the adaptive driving control has ended (S58: YES), the current awakening adaptive processing ends. On the other hand, if the adaptive driving control is continuing (S58: NO), steps S51 to S57 (processing) are repeated.

[0079] (Summary of First Embodiment) In the first embodiment described so far, even if the approach of an emergency vehicle is detected, the driver can continue sleeping because the wake-up notification that prompts the driver to wake up is limited. In addition, since the approach of the emergency vehicle is indicated to the driver by a display, even if the driver suddenly wakes up, the driver can understand from the display that the emergency vehicle is approaching. As a result, it is possible to respond to an emergency vehicle in a convenient manner for passengers such as the driver.

[0080] Additionally, in the first embodiment, if the driver wakes up after the start of the limited approach notification, the presentation control unit 88 notifies the driver by voice that the host vehicle Am is currently performing emergency vehicle response driving control. As described above, the presentation control unit 88 can clearly notify the driver that an emergency vehicle is approaching and that emergency vehicle response driving control has already begun. As a result, a system that is convenient for the driver is realized.

[0081] In the first embodiment, if the driver awakens while the host vehicle Am is performing adaptive cruise control to respond to an emergency vehicle, the adaptive cruise control is temporarily suspended. This suspension of adaptive cruise control temporarily suppresses changes in the situation and the behavior of the host vehicle. As a result, the driver can easily grasp the situation of the host vehicle Am immediately after awakening.

[0082] Furthermore, in the first embodiment, when an emergency vehicle approaches the host vehicle Am while the driver is not asleep, the approaching emergency vehicle is notified to the driver by both audio and visual notification. Therefore, a driver who is awake can reliably recognize the approaching emergency vehicle. On the other hand, when an emergency vehicle approaches the host vehicle Am while the driver is asleep, the limited approach notification stops audio notification and only visual notification is provided to the driver. Therefore, a driver who is asleep can continue to sleep without being forced to wake up unnecessarily.

[0083] Additionally, in the first embodiment, it is further determined whether an emergency vehicle is traveling in the same direction as the host vehicle Am. Then, in a first situation in which the approach of an emergency vehicle is determined but it is unclear whether the emergency vehicle is traveling in the same direction, a limited approach notification is implemented. In contrast, in a second situation in which it is determined that the emergency vehicle is traveling in the same direction, whether or not to implement a limited approach notification is determined depending on whether the driver is asleep. As a result, it is possible to avoid waking the driver in the early stages of approach when the relative position of the emergency vehicle is unknown. This reduction in hassle can realize a convenient emergency vehicle response for the driver.

[0084] In the first embodiment, if the driver is asleep in the second situation, a limited approach notification is provided, whereas if the driver is not asleep in the second situation, an emergency approach notification is provided to notify the driver of the approach of an emergency vehicle by both audio and visual means. As described above, even if the driver is awake, audio notification is suspended until the second situation is reached. This reduces the annoyance to the driver who is performing another task.

[0085] In the first embodiment, if the host vehicle Am is obstructing the emergency vehicle after the start of the limited approach warning, the driver is notified of the alert. As a result, in a situation where it is difficult to appropriately respond using the emergency driving control, it is possible to leave the response to the emergency vehicle to the driver.

[0086] In addition, in the first embodiment, after the start of the wake-up notification, the display indicating the approach of the emergency vehicle is restarted from the beginning. As a result, the driver who has woken up can properly understand the situation of the host vehicle Am by checking the contents of the restarted display. As a result, the driver can easily start responding to the emergency vehicle promptly.

[0087] In the first embodiment, when an emergency vehicle approaches the host vehicle Am, the approaching emergency vehicle is notified to an outside vehicle using the exterior display 45 mounted on the host vehicle Am. As a result, information about the approaching emergency vehicle can be shared with other vehicles around the host vehicle. As a result, the host vehicle Am can more easily obtain cooperation from other vehicles through adaptive cruise control, allowing the host vehicle Am to smoothly avoid the emergency vehicle.

[0088] Furthermore, in the first embodiment, when it is determined that an emergency vehicle is traveling in the same direction as the host vehicle Am, in addition to providing an exterior warning via the exterior display 45, the hazard lights 44 of the host vehicle Am begin to flash. As a result, other vehicles around the host vehicle Am can prepare for the host vehicle Am's actions to avoid the emergency vehicle. As a result, the host vehicle Am can smoothly implement corresponding driving control.

[0089] Additionally, in the first embodiment, manual driving by a driver who has changed from a sleepy state to an awake state is prohibited, which prevents a situation in which a driver who has just woken up and is still half asleep inputs an erroneous manual driving operation, causing the behavior of the host vehicle Am to become unstable.

[0090] In the first embodiment, the information acquisition unit 81 corresponds to a "state recognition unit," and the control recognition unit 83 corresponds to an "approach recognition unit." Furthermore, the function of the request output unit 84 that performs the process of S55 corresponds to an "operation restriction unit," the function of the request output unit 84 that performs the process of S57 corresponds to a "response restriction unit," and the presentation control unit 88 corresponds to an "alert implementation unit." Furthermore, the HMI control device 100 corresponds to a "vehicle control device."

[0091] Second Embodiment A second embodiment of the present disclosure shown in FIG. 8 is a modified example of the first embodiment. The HMI control device 100 of the second embodiment is installed in a service vehicle that provides a mobility service. The mobility service provides service users with a means of transportation to their destination. The service vehicle is equipped with an autonomous driving ECU 50 and is capable of autonomous driving at level 4 or higher. An operator may or may not be on board the service vehicle. The operator sits in the driver's seat of the service vehicle so as to be able to operate the driving operation unit. The operator is an occupant who takes over control of the driving operation from the autonomous driving ECU 50 in a situation where it is difficult for the autonomous driving ECU 50 to respond.

[0092] The details of the emergency vehicle notification process performed by the HMI control device 100 installed in the service vehicle will be described below based on Fig. 8 and with reference to Figs. 1 to 3. The emergency vehicle notification process of the second embodiment is started by the HMI control device 100 when the service vehicle starts operating autonomously. The emergency vehicle notification process is repeatedly performed until the service vehicle stops operating.

[0093] In S211 of the emergency vehicle notification process, the control recognition unit 83 recognizes the approach of an emergency vehicle to the host vehicle Am based on the recognition result acquired from the autonomous driving ECU 50. If the control recognition unit 83 has not recognized the approach of an emergency vehicle (S211: NO), the current emergency vehicle notification process is temporarily terminated. On the other hand, if the control recognition unit 83 has recognized the approach of an emergency vehicle (S211: YES), the information acquisition unit 81 acquires driver status information in S212. Based on the acquired latest driver status information, the information acquisition unit 81 recognizes (determines) whether an operator is in the service vehicle in S213.

[0094] If an operator is on board (S213: YES), the presentation control unit 88 performs a limited approach notification in S214. In this way, if an emergency vehicle approaches the service vehicle while an operator is on board, the audio notification is stopped. Instead, the user (passenger) of the service vehicle is notified of the approach of the emergency vehicle only by a visual display.

[0095] On the other hand, if an operator is not on board (S213: NO), the presentation control unit 88 performs a normal emergency approach notification in S215. In this way, when an emergency vehicle approaches a service vehicle without an operator on board, the approach of the emergency vehicle is notified to passengers by both audio and visual notification.

[0096] In the second embodiment described above, when an operator is not on board, passengers in the service vehicle are notified of the approach of an emergency vehicle by voice. This allows passengers to be sure that the onboard system is aware of the approach of the emergency vehicle. Therefore, passengers can continue to use the service vehicle with peace of mind, even when an operator is not on board.

[0097] Furthermore, when an operator is on board, voice notifications are limited (stopped). By limiting bothersome notifications in this way, passengers can use the service vehicle comfortably. As a result, it is possible to provide an emergency vehicle response that is convenient for passengers using the service vehicle, depending on whether an operator is present.

[0098] Third Embodiment The third embodiment of the present disclosure is another modification of the first embodiment. In the third embodiment, when an emergency vehicle approaches the host vehicle Am while the driver is asleep during an autonomous driving period at autonomous driving level 4 or higher, the manner of notification varies depending on the road environment around the host vehicle and the content of the adaptive cruise control. In addition, when an emergency evacuation using adaptive cruise control requires the driver to control the host vehicle Am, i.e., when a driver handover request is made, an awakening notification is implemented to wake the driver up using at least audio. For example, when an emergency evacuation makes it impossible for the emergency vehicle to travel due to the influence of the host vehicle Am, or when it becomes difficult to continue evacuation using the autonomous driving function, an awakening notification is implemented. Details of the emergency vehicle notification process implemented by the HMI control device 100 of the third embodiment will be described below based on FIG. 9 and with reference to FIGS. 1 to 3. As in the first embodiment, the emergency vehicle notification process of the third embodiment is initiated when the control understanding unit 83 understands that autonomous driving of level 4 or higher has begun, and is repeatedly performed until the information linkage unit 82 understands that autonomous driving of level 4 or higher has ended.

[0099] In steps S11 to S14 of the emergency vehicle notification process, substantially the same processing as in the first embodiment is performed. In steps S11 and S14, the information acquisition unit 81 determines whether an emergency vehicle is approaching the host vehicle Am and whether the driver is asleep. If an emergency vehicle is approaching the host vehicle Am and the driver is awake (S14: NO), the presentation control unit 88 performs a normal emergency approach notification in step S16. In this emergency approach notification, the approach of the emergency vehicle is notified to the driver both audibly and visually.

[0100] On the other hand, if the driver is asleep (S14: YES), in S311 the control recognition unit 83 acquires road type information from the autonomous driving ECU 50. The road type information includes information indicating the type of road on which the host vehicle Am is traveling. The road type information is prepared by the environment recognition unit 62 based on locator information, map data, etc. In S312, the control recognition unit 83 determines whether the road on which the host vehicle Am is traveling is an ordinary road based on the road type information.

[0101] Here, a general road is a road that is open to pedestrians and all vehicles, including bicycles and motorcycles. Specifically, general roads include urban roads, regional roads, and rural roads. General roads are equipped with intersections, traffic signals, crosswalks, sidewalks, etc. In contrast, roads other than general roads (hereinafter referred to as designated roads) are roads that only certain types of vehicles can use and where pedestrians, bicycles, light vehicles, etc. are prohibited from using. Specifically, designated roads include expressways and motorways. The speed limits of designated roads are set higher than those of general roads. Designated roads are connected to general roads via interchanges, junctions, etc.

[0102] If the host vehicle Am is traveling on a specific road such as an expressway (S312: NO), the presentation control unit 88 performs a limited approach notification in S15, which notifies the driver of the approach of an emergency vehicle only by display. In the limited approach notification, the audio notification is stopped (limited). In this way, the intensity of the notification that notifies the driver of the approach of an emergency vehicle when the host vehicle Am is traveling on a specific road is set lower than the intensity of the notification when the emergency vehicle is approaching the host vehicle Am traveling on an ordinary road.

[0103] On the other hand, if the host vehicle Am is traveling on an ordinary road (S312: YES), in S313 the control grasping unit 83 grasps the content of the adaptive cruise control being performed or scheduled to be performed by the evacuation control unit 78. In S314, the control grasping unit 83 grasps (determines) whether the host vehicle Am will temporarily stop in adaptive cruise control corresponding to the emergency vehicle. If adaptive cruise control that temporarily stops the host vehicle Am is performed (S314: YES), in S15 the presentation control unit 88 implements a limited approach notification that notifies the driver of the approach of the emergency vehicle only by display.

[0104] On the other hand, when adaptive cruise control is performed to continue driving slowly without temporarily stopping the host vehicle Am (S314: NO), the presentation control unit 88 performs a normal emergency approach warning in S16 to notify the driver of the approach of an emergency vehicle by both audio and visual means. In this way, the intensity of the warning when adaptive cruise control is performed without temporarily stopping the host vehicle Am is set higher than the intensity of the warning when adaptive cruise control is performed to temporarily stop the host vehicle Am.

[0105] When a limited approach notification is issued because the driver is asleep, the control recognition unit 83 determines in S317 whether a driver change request has occurred in the adaptive cruise control (emergency evacuation) for the emergency vehicle. If adaptive cruise control using the autonomous driving function at autonomous driving level 4 or higher continues and the control recognition unit 83 has not acquired a driver change request (S317: NO), the current emergency vehicle notification process is temporarily terminated. On the other hand, if it is difficult to continue autonomous driving at level 4 or higher and the driver needs to monitor the surroundings or perform driving operations, the control recognition unit 83 acquires a driver change request from the autonomous driving ECU 50 (S317: YES). In this case, the presentation control unit 88 issues a wake-up notification in S18 to prompt the asleep driver to wake up by at least a sound. Furthermore, the presentation control unit 88 restarts the display indicating the approach of an emergency vehicle from the beginning in S19 after the wake-up notification has started. The presentation control unit 88 temporarily ends the display of the series of animations that began in S15, and then in S19 displays the animations from the beginning to alert the driver of the approaching emergency vehicle.

[0106] (Summary of the Third Embodiment) The third embodiment described so far also achieves the same effects as the first embodiment. In other words, if the driver is asleep, the wake-up notification that prompts the driver to wake up is limited even if the driver recognizes that an emergency vehicle is approaching. Furthermore, even if the driver suddenly wakes up while the emergency vehicle is approaching, the driver can recognize the approaching emergency vehicle from the display. Therefore, it is possible for passengers such as the driver to respond to the emergency vehicle in a convenient manner.

[0107] Additionally, in the third embodiment, a determination is made as to whether the road on which the host vehicle Am is traveling is a general road. Then, when the driver is asleep, the manner of notification to the driver of the approaching emergency vehicle changes depending on whether the emergency vehicle approaches the host vehicle Am on a general road or on a specific road other than a general road. As described above, when the approach of an emergency vehicle is detected, it is possible to appropriately determine whether to prompt the driver to wake up, depending on the driving environment around the host vehicle. As a result, a response to the emergency vehicle that is convenient for the driver and the like can be implemented.

[0108] In the third embodiment, when the driver is asleep, the intensity of the notification when an emergency vehicle approaches the host vehicle Am traveling on a general road is set higher than the intensity of the notification when an emergency vehicle approaches the host vehicle Am traveling on a specific road. Specifically, when an emergency vehicle approaches the host vehicle Am traveling on a general road, the driver is notified of the approach of the emergency vehicle by both audio and visual notification in the emergency approach notification. In contrast, when an emergency vehicle approaches the host vehicle Am traveling on a specific road, the audio notification is stopped and the driver is notified of the approach of the emergency vehicle by visual notification in the limited approach notification. The adaptive cruise control for responding to an emergency vehicle on a specific road is less affected by external disturbances than the adaptive cruise control performed on a general road. Therefore, the urgency of waking the driver is lower on a specific road than on a general road. Therefore, stopping the audio notification on a specific road can improve driver convenience.

[0109] Furthermore, in the third embodiment, the emergency vehicle response control determines whether the vehicle is temporarily stopped. The notification mode for informing the passengers of the approach of an emergency vehicle changes depending on whether the emergency vehicle response control is performed to temporarily stop the vehicle or to continue driving the vehicle while the driver is asleep. As described above, when the approach of an emergency vehicle is determined, it is possible to appropriately determine whether to prompt the driver to wake up, depending on the details of the emergency vehicle response control. As a result, an emergency vehicle response that is convenient for the driver and the like can be implemented.

[0110] Additionally, in the third embodiment, when the driver is asleep and adaptive cruise control is performed without temporarily stopping the host vehicle Am, the intensity of the notification to the driver of the approaching emergency vehicle is set higher than the intensity of the notification to the driver of the approaching emergency vehicle when adaptive cruise control is performed to temporarily stop the host vehicle Am. Specifically, when adaptive cruise control is performed to continue traveling of the host vehicle Am, the limited approach notification stops the audio notification and notifies the driver of the approaching emergency vehicle by display. When the adaptive cruise control temporarily stops the host vehicle, the driver can be given time to check the situation around the host vehicle. Therefore, the awakening of the driver may be postponed. On the other hand, when adaptive cruise control is performed to temporarily stop the host vehicle Am, the emergency approach notification notifies the driver of the approaching emergency vehicle by both audio and visual means. When the adaptive cruise control continues traveling at a slow speed, for example, the driver has less time to check the situation around the host vehicle than when the adaptive cruise control temporarily stops the vehicle. Therefore, it is desirable to provide audio and visual notifications.

[0111] In the third embodiment, when the driver is asleep and driving operation by the driver is required to respond to an emergency vehicle, an audio wake-up notification is implemented. Therefore, even if the notification to wake the driver is withheld, if a response by the driver is required, the driving control can be appropriately handed over from the automatic driving function to the driver.

[0112] (Other Embodiments) Although multiple embodiments of the present disclosure have been described above, the present disclosure should not be construed as being limited to the above-described embodiments, and can be applied to various embodiments and combinations within the scope that does not deviate from the gist of the present disclosure.

[0113] In the above embodiment, the determination of whether the driver is asleep is performed based on the driver status information generated by the driver monitor 29. However, sensing information having a configuration different from that of the driver monitor 29 may be used to determine whether the driver is asleep.

[0114] As an example, the information acquisition unit 81 according to the first modification acquires information about the reclining angle of the driver's seat. Based on this angle information, the information acquisition unit 81 determines the sleeping state of the driver when the backrest of the driver's seat is tilted backward. The information acquisition unit 81 may determine the sleeping state of the driver by combining the driver status information and the reclining angle information.

[0115] Furthermore, the information acquisition unit 81 according to the second modification acquires biological information of the driver, specifically, information such as pulse rate, body temperature, and brain waves. The biological information is measured by a sensor provided in the driver's seat or a wearable device worn by the driver. The information acquisition unit 81 combines the biological information with driver status information to grasp the sleeping state of the driver.

[0116] In a third modification of the above embodiment, no distinction is made between the first situation and the second situation. In the third modification, when an emergency vehicle approaches, if the driver is asleep, a limited approach notification is implemented, whereas if the driver is not asleep, the approach of the emergency vehicle is notified by both audio and visual means.

[0117] In the fourth modification of the first embodiment, in addition to or instead of the driver, it is determined whether a passenger (fellow passenger) seated in a seat other than the driver's seat is asleep. When an emergency vehicle approaches the host vehicle Am during an autonomous driving period at autonomous driving level 4 or higher, if the driver is awake, actions and notifications intended to wake the passenger may be limited (stopped).

[0118] In a fifth modification of the third embodiment, when the driver is asleep, the intensity of the notification when an emergency vehicle approaches the host vehicle Am traveling on an ordinary road is set lower than the intensity of the notification when an emergency vehicle approaches the host vehicle Am traveling on a specific road. Specifically, when the host vehicle Am is traveling on an ordinary road, a limited approach notification is implemented, and when the host vehicle Am is traveling on a highway or the like, a normal emergency approach notification is implemented. As in the fifth modification, the traveling speed of the host vehicle Am may be taken into consideration, and the intensity of the emergency vehicle approach notification may be adjusted to be higher in a traveling environment where the traveling speed is higher.

[0119] In a sixth modification of the third embodiment, when the driver is asleep, the intensity of the notification when adaptive cruise control is performed to prevent the host vehicle Am from temporarily stopping is set lower than the intensity of the notification when adaptive cruise control is performed to temporarily stop the host vehicle Am. Specifically, when a temporary stop is made, a normal emergency approach notification is implemented, and when slow-moving travel is continued, a limited approach notification is implemented. As in the sixth modification, it may be assumed that the more a scene requires a temporary stop, the more difficult the adaptive cruise control will be, and the intensity of the emergency vehicle approach notification may be adjusted higher to encourage the driver to wake up.

[0120] In a seventh modification of the above embodiment, some of the functions of the HMI control device 100 are implemented in the autonomous driving ECU 50. That is, in the seventh modification, a system including the HMI control device 100 and the autonomous driving ECU 50 corresponds to a "vehicle control device." In this seventh modification, the environment recognition unit 62 corresponds to an "approach recognition unit," the control switching unit 77 corresponds to an "operation restriction unit," and the evacuation control unit 78 corresponds to a "response restriction unit."

[0121] In addition, in the eighth modification of the above embodiment, a system including the HMI control device 100 and the driver monitor 29 corresponds to the "vehicle control device." In the eighth modification, the driver monitor 29 corresponds to the "state grasping unit."

[0122] Furthermore, in a ninth modification of the above embodiment, the functions of the HMI control device 100 are integrated into the autonomous driving ECU 50. In this ninth modification, the integrated autonomous driving ECU 50 corresponds to the "vehicle control device."

[0123] In the above embodiments, the functions provided by the HMI control device can be provided by software and hardware that executes the software, software alone, hardware alone, or a combination of these. Furthermore, when such functions are provided by electronic circuits as hardware, each function can also be provided by digital circuits including multiple logic circuits or analog circuits. Furthermore, the software for realizing such functions may include, at least in part, code automatically generated by a neural network or language model trained using a large amount of learning data.

[0124] Each processing unit in the above embodiments is hardware for arithmetic processing coupled to a RAM. The processing unit includes at least one arithmetic core, such as a central processing unit (CPU) and a graphics processing unit (GPU). The processing unit may further include a field-programmable gate array (FPGA), a neural network processing unit (NPU), and an IP core with other dedicated functions. Furthermore, the processing unit is not limited to being individually mounted on a printed circuit board. The processing unit may be mounted on an application-specific integrated circuit (ASIC), a system on chip (SoC), a chiplet integration, an FPGA, or the like.

[0125] In the above embodiments, the form of the storage medium (non-transitory tangible storage medium) that stores various programs and the like may be changed as appropriate. Furthermore, the storage medium is not limited to a configuration provided on a circuit board, but may be provided in the form of a memory card or the like, inserted into a slot, and electrically connected to a control circuit such as an autonomous driving ECU. Furthermore, the storage medium may be an optical disk, hard disk drive, solid state drive, or the like that serves as a source from which programs are copied or distributed to the autonomous driving ECU or the like.

[0126] Vehicles equipped with the above-described autonomous driving ECU and HMI control device are not limited to general private passenger cars (Personally Owned Vehicles, POVs) and service vehicles. Vehicles equipped with the HMI control device and the like may also be rental cars, manned taxis, ride-sharing vehicles, freight vehicles, buses, etc. The vehicle may be a right-hand drive vehicle or a left-hand drive vehicle. Furthermore, the traffic environment in which the vehicle travels may be a traffic environment based on left-hand traffic or a traffic environment based on right-hand traffic. Vehicle control according to the present disclosure may be optimized as appropriate according to the road traffic laws of each country and region, as well as the steering wheel position of the vehicle.

[0127] The controller and methods described herein may be implemented by a special-purpose computer comprising a processor programmed to perform one or more functions embodied in a computer program. Alternatively, the apparatus and methods described herein may be implemented by special-purpose hardware logic circuitry. Alternatively, the apparatus and methods described herein may be implemented by one or more special-purpose computers comprising a processor executing a computer program in combination with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0128] (Disclosure of Technical Ideas) This specification discloses multiple technical ideas described in the following multiple clauses. Some clauses may be described in a multiple dependent form, with the subsequent clause alternatively referring to the preceding clause. Furthermore, some clauses may be described in a multiple dependent form, with the subsequent clause referring to another multiple dependent clause. These multiple dependent clauses define multiple technical ideas.

[0129] (Technical Idea 1) A vehicle control device used in a host vehicle (Am) capable of traveling by autonomous driving in which a passenger seated in the driver's seat is allowed to sleep, comprising: an approach recognition unit (83) that recognizes the approach of an emergency vehicle to the host vehicle; a state recognition unit (81) that recognizes whether the passenger is asleep or not; and a notification implementation unit (88) that issues an emergency approach notification to the passenger of the approach of the emergency vehicle, wherein, when the emergency vehicle approaches the host vehicle while the passenger is asleep during a period of automatic traveling by the autonomous driving, the notification implementation unit limits an awakening notification to encourage the passenger to wake up, while implementing a limited approach notification that indicates the approach of the emergency vehicle to the passenger by a display. (Technical Idea 2) The vehicle control device according to Technical Idea 1, wherein, when the passenger wakes up after the start of the limited approach notification, the notification implementation unit issues a voice notification that the host vehicle is performing response driving control to accommodate the emergency vehicle. (Technical Idea 3) The vehicle control device according to Technical Idea 1 or 2, further comprising a response restriction unit (S57) that temporarily suspends the response travel control in the event that the occupant awakens during a period in which the host vehicle is implementing the response travel control in response to the emergency vehicle. (Technical Idea 4) The vehicle control device according to any one of Technical Ideas 1 to 3, wherein the notification implementation unit notifies the occupant of the approach of the emergency vehicle by both audio and visual means in the emergency approach notification in the event that the emergency vehicle approaches the host vehicle while the occupant is not in the sleep state, and cancels the audio notification and notifies the occupant of the approach of the emergency vehicle by visual means in the limited approach notification in the event that the emergency vehicle approaches the host vehicle while the occupant is in the sleep state. (Technical Idea 5) A vehicle control device described in any one of Technical Ideas 1 to 4, wherein the approach recognition unit further recognizes whether the emergency vehicle is traveling in the same direction as the host vehicle, and the notification implementation unit implements the limited approach notification in a first situation in which the approach of the emergency vehicle is recognized but it is unclear whether the emergency vehicle is traveling in the same direction, and determines whether to implement the limited approach notification depending on whether the occupant is in the sleeping state in a second situation in which it is recognized that the emergency vehicle is traveling in the same direction.(Technical Idea 6) The vehicle control device according to Technical Idea 5, wherein the notification implementation unit implements the limited approach notification when the occupant is in the sleeping state in the second situation, and implements the emergency approach notification that indicates the approaching emergency vehicle to the occupant by both audio and visual means when the occupant is not in the sleeping state in the second situation. (Technical Idea 7) The vehicle control device according to any one of Technical Ideas 1 to 6, wherein the approach recognition unit recognizes whether or not the host vehicle is obstructing the travel of the emergency vehicle, and the notification implementation unit implements the awakening notification when the host vehicle is obstructing the travel of the emergency vehicle after the start of the limited approach notification. (Technical Idea 8) The vehicle control device according to Technical Idea 7, wherein the notification implementation unit restarts the display indicating the approaching emergency vehicle from the beginning after the start of the awakening notification. (Technical Idea 9) The vehicle control device according to any one of Technical Ideas 1 to 8, wherein the approach grasping unit determines whether the road on which the vehicle is traveling is an ordinary road, and the notification implementation unit changes the manner of the notification indicating to the occupant of the approach of the emergency vehicle between a case in which the emergency vehicle approaches the vehicle on the ordinary road and a case in which the emergency vehicle approaches the vehicle on a specific road other than the ordinary road, while the occupant is in the sleeping state. (Technical Idea 10) The vehicle control device according to Technical Idea 9, wherein the notification implementation unit increases the intensity of the notification indicating to the occupant of the approach of the emergency vehicle when the emergency vehicle approaches the vehicle traveling on the ordinary road, while the occupant is in the sleeping state, compared to the intensity of the notification indicating to the occupant of the approach of the emergency vehicle when the emergency vehicle approaches the vehicle traveling on the specific road. (Technical Idea 11) A vehicle control device described in any one of Technical Ideas 1 to 10, wherein the approach recognition unit recognizes whether the vehicle will temporarily stop in response to the emergency vehicle using adaptive cruise control, and the notification execution unit changes the manner of notification to the occupant that the emergency vehicle is approaching depending on whether the adaptive cruise control is performed to temporarily stop the vehicle or not when the occupant is asleep.(Technical Idea 12) The vehicle control device according to Technical Idea 11, wherein the notification implementation unit increases the intensity of the notification indicating to the occupant that the emergency vehicle is approaching when the responsive travel control that does not cause the host vehicle to temporarily stop is performed in a situation where the occupant is asleep, compared to the intensity of the notification indicating to the occupant that the emergency vehicle is approaching when the responsive travel control that causes the host vehicle to temporarily stop is performed. (Technical Idea 13) The vehicle control device according to any one of Technical Ideas 1 to 12, wherein the notification implementation unit implements the awakening notification by voice in a situation where the occupant is asleep and driving operation by the occupant is required to respond to the emergency vehicle. (Technical Idea 14) The vehicle control device according to any one of Technical Ideas 1 to 13, wherein the host vehicle is a service vehicle that provides a mobility service, the state grasping unit grasps whether an operator is on board the service vehicle operated by autonomous driving, and the notification implementation unit, when the operator is not on board and the emergency vehicle approaches the service vehicle, notifies the approach of the emergency vehicle by both audio and visual notification, and when the operator is on board and the emergency vehicle approaches the service vehicle, stops the audio notification and notifies the approach of the emergency vehicle by visual notification. (Technical Idea 15) The vehicle control device according to any one of Technical Ideas 1 to 14, wherein, when the emergency vehicle approaches the host vehicle, the notification implementation unit notifies the outside of the vehicle that the emergency vehicle is approaching using an exterior display (45) mounted on the host vehicle. (Technical Idea 16) The vehicle control device according to Technical Idea 15, wherein the approach recognition unit further recognizes whether the emergency vehicle is traveling in the same direction as the host vehicle, and the notification execution unit, when it is recognized that the emergency vehicle is traveling in the same direction, starts flashing an operation of a hazard lamp (44) of the host vehicle in addition to providing an exterior notification by the exterior display. (Technical Idea 17) The vehicle control device according to any one of Technical Ideas 1 to 16, further comprising an operation restriction unit (S55) that prohibits the occupant who has changed from the sleep state to the wakefulness state from performing manual driving.(Technical Idea 18) A vehicle control program used in a host vehicle (Am) capable of traveling by autonomous driving in which a passenger seated in the driver's seat is allowed to sleep, the vehicle control program causing at least one processing unit (11) to execute processing including: determining whether an emergency vehicle is approaching the host vehicle (S11); determining whether the passenger is asleep or not (S13, S14); and, if the emergency vehicle approaches the host vehicle while the passenger is asleep during a period of autonomous traveling by the autonomous driving, implementing a limited approach notification that indicates to the passenger the approach of the emergency vehicle by displaying a message while limiting a wake-up notification to encourage the passenger to wake up (S15).

Claims

1. A vehicle control device used in a self-driving vehicle (Am) that is capable of driving by automatic driving, in which sleep is permitted for the occupant seated in the driver's seat, The aforementioned approach detection unit (83) detects the approach of an emergency vehicle to the vehicle, A state-aware unit (81) that determines whether the passenger is in a sleep state, The system includes a notification unit (88) that performs an emergency approach notification to inform the occupants of the approach of the emergency vehicle, The aforementioned notification implementation unit, During the automated driving period under the aforementioned autonomous driving system, if an emergency vehicle approaches the vehicle while the passenger is in the aforementioned sleep state, a limited proximity notification will be issued to show the passenger the approaching emergency vehicle, while limiting the wake-up notification to prompt the passenger to wake up. A vehicle control device that, if the state recognition unit determines that the occupant has changed from a sleeping state to an awake state after the commencement of the limited approach notification, announces by voice that the vehicle is performing corresponding driving control in response to the emergency vehicle.

2. The approach detection unit further determines whether the emergency vehicle is traveling in the same direction as the vehicle itself. The aforementioned notification implementation unit, In the first situation where the approach of the emergency vehicle has been detected, but it is unclear whether the emergency vehicle is traveling in the same direction, the limited approach notification is implemented. The vehicle control device according to claim 1, which determines whether or not to implement the limited proximity notification depending on whether or not the occupant is in a sleeping state in the second situation in which it is determined that the emergency vehicle is traveling in the same direction.

3. A vehicle control device used in a self-driving vehicle (Am) that is capable of driving by automatic driving, in which sleep is permitted for the occupant seated in the driver's seat, The aforementioned approach detection unit (83) detects the approach of an emergency vehicle to the vehicle, A state-aware unit (81) that determines whether the passenger is in a sleep state, The system includes a notification unit (88) that performs an emergency approach notification to inform the occupants of the approach of the emergency vehicle, The approach detection unit further determines whether the emergency vehicle is traveling in the same direction as the vehicle itself. The aforementioned notification implementation unit, During the automated driving period under the aforementioned autonomous driving system, if an emergency vehicle approaches the vehicle while the passenger is in the aforementioned sleep state, a limited proximity notification will be issued to show the passenger the approaching emergency vehicle, while limiting the wake-up notification to prompt the passenger to wake up. In the first situation where the approach of the emergency vehicle has been detected, but it is unclear whether the emergency vehicle is traveling in the same direction, the limited approach notification is implemented. A vehicle control device that determines whether or not to implement the restricted proximity notification in the second situation in which it is determined that the emergency vehicle is traveling in the same direction, depending on whether or not the occupant is in a sleeping state.

4. The aforementioned notification implementation unit, In the second situation described above, if the passenger is in the sleep state, the limited approach notification is issued. The vehicle control device according to claim 2 or 3, which, in the second situation, if the passenger is not in the sleep state, provides an emergency approach notification to the passenger by both voice and display to indicate the approach of the emergency vehicle.

5. The vehicle control device according to claim 1 or 3, further comprising: a response restriction unit (S57) that temporarily suspends the response driving control if the occupant becomes awake during the period in which the vehicle performs the response driving control corresponding to the emergency vehicle.

6. The aforementioned notification implementation unit, When an emergency vehicle approaches the vehicle while the passenger is not in the sleep state, the emergency approach notification shall inform the passenger of the approach of the emergency vehicle by both audio and visual means. The vehicle control device according to claim 1 or 3, which, when an emergency vehicle approaches the vehicle while the passenger is in the sleep state, discontinues the audible notification and indicates the approach of the emergency vehicle to the passenger via a display.

7. The approach detection unit determines whether the vehicle is obstructing the movement of the emergency vehicle. The vehicle control device according to claim 1 or 3, wherein the notification implementing unit performs the alert notification if the vehicle is obstructing the movement of the emergency vehicle after the commencement of the restricted approach notification.

8. The vehicle control device according to claim 7, wherein the notification unit restarts the display indicating the approach of the emergency vehicle from the beginning after the start of the alert notification.

9. The proximity detection unit determines whether the road on which the vehicle is traveling is a public road or not. The vehicle control device according to claim 1 or 3, wherein the notification unit changes the manner of notification to the passenger indicating the approach of the emergency vehicle when the emergency vehicle approaches the vehicle on a public road or when the emergency vehicle approaches the vehicle on a specific road other than a public road, while the passenger is in a sleeping state.

10. The vehicle control device according to claim 9, wherein the notification implementing unit, when the passenger is in the state of sleep, increases the intensity of the notification that indicates the approach of the emergency vehicle to the passenger when the emergency vehicle approaches the vehicle traveling on the general road, to a higher intensity than the notification that indicates the approach of the emergency vehicle to the passenger when the vehicle is traveling on the specific road.

11. The approach detection unit determines whether or not the vehicle will come to a temporary stop using the corresponding driving control for the emergency vehicle. The vehicle control device according to claim 1 or 3, wherein the notification implementing unit changes the manner of notification to the passenger that an emergency vehicle is approaching, depending on whether the corresponding driving control that causes the vehicle to temporarily stop is performed or whether the corresponding driving control that does not cause the vehicle to temporarily stop is performed while the passenger is in the sleep state.

12. The vehicle control device according to claim 11, wherein the notification implementing unit increases the intensity of the notification that indicates the approach of an emergency vehicle to the passenger when the corresponding driving control that does not temporarily stop the vehicle is performed while the passenger is in a sleeping state, to a higher intensity than the notification that indicates the approach of an emergency vehicle to the passenger when the corresponding driving control that temporarily stops the vehicle is performed.

13. The vehicle control device according to claim 1 or 3, wherein the notification unit provides an audible wake-up notification when the passenger is in a sleeping state and driving operations are required by the passenger to respond to an emergency vehicle.

14. The aforementioned vehicle is a service vehicle that provides mobility services. The status recognition unit determines whether or not an operator is on board the service vehicle operated by the automated driving system. The aforementioned notification implementation unit, If the emergency vehicle approaches the service vehicle while the operator is not on board, the approach of the emergency vehicle will be notified by both audio and display. The vehicle control device according to claim 1 or 3, wherein, while the operator is on board, if the emergency vehicle approaches the service vehicle, the audible notification is discontinued and the approach of the emergency vehicle is notified by display.

15. A vehicle control device used in a service vehicle (Am) that provides mobility services and is capable of autonomous driving, The aforementioned approach detection unit (83) detects the approach of an emergency vehicle to the vehicle, A status-aware unit (81) that determines whether or not an operator is on board the service vehicle operated by the aforementioned autonomous driving system, The system includes a notification unit (88) that performs an emergency approach notification to inform the passengers of the service vehicle of the approach of the emergency vehicle, The notification unit, during the automated driving period under the automated driving system, If the emergency vehicle approaches the service vehicle while the operator is not on board, the emergency approach notification will be implemented to notify the service vehicle of the approach of the emergency vehicle by both voice and display. A vehicle control device that, when the aforementioned operator is on board and the emergency vehicle approaches the service vehicle, discontinues audible notification and notifies the approach of the emergency vehicle by display.

16. The vehicle control device according to any one of claims 1, 3, or 15, wherein the notification unit, when an emergency vehicle approaches the vehicle, uses an external display (45) mounted on the vehicle to notify the outside that an emergency vehicle is approaching.

17. The vehicle control device according to claim 16, wherein, when the notification unit determines that the emergency vehicle is traveling in the same direction as the vehicle itself, the notification unit initiates the flashing of the vehicle's hazard lights (44) in addition to the external notification on the external display.

18. The vehicle control device according to claim 1 or 3, further comprising an operation restriction unit (S55) that prohibits the passenger from performing manual driving after changing from a sleep state to an awake state.

19. A vehicle control method used in a self-driving vehicle (Am) that is capable of being driven by an automated system in which sleep is permitted for the occupant seated in the driver's seat, The vehicle becomes aware of the approach of an emergency vehicle to its own vehicle (S11), Determine whether the passenger is asleep or not (S13, S14), During the automated driving period under the aforementioned autonomous driving system, if the passenger is in the aforementioned sleep state and an emergency vehicle approaches the vehicle, a limited approach notification is implemented to show the passenger the approach of the emergency vehicle, while limiting the wake notification to prompt the passenger to wake up (S15). If it is determined that the occupant has changed from a sleeping state to an awake state after the commencement of the aforementioned limited approach notification, the vehicle will be notified by voice that it is performing corresponding driving control to the emergency vehicle (S56). A vehicle control method that includes the following step in a process performed in at least one processing unit (11).

20. A vehicle control method used in a self-driving vehicle (Am) that is capable of being driven by an automated system in which sleep is permitted for the occupant seated in the driver's seat, The vehicle becomes aware of the approach of an emergency vehicle to its own vehicle (S11), Further determination is made as to whether the emergency vehicle is traveling in the same direction as the vehicle itself (S12), Determine whether the passenger is asleep or not (S13, S14), During the period of autonomous driving under the aforementioned autonomous driving system, In the first situation, where the approach of the emergency vehicle is known but it is unclear whether the emergency vehicle is traveling in the same direction, a limited approach notification is implemented to indicate the approach of the emergency vehicle to the passenger by display, while limiting the alert to wake the passenger. In the second situation, when it is determined that the emergency vehicle approaching the vehicle is traveling in the same direction, it is decided whether or not to implement the limited proximity notification depending on whether or not the occupant is asleep (S15, S16). A vehicle control method that includes the following step in a process performed in at least one processing unit (11).

21. A vehicle control method used in a service vehicle (Am) that provides mobility services and is capable of driving by autonomous driving, The vehicle becomes aware of the approach of an emergency vehicle to its own vehicle (S211), The system determines whether or not an operator is on board the service vehicle operated by the automated driving system (S213), During the automated driving period under the aforementioned automated driving system, if the emergency vehicle approaches the service vehicle while the operator is not on board, an emergency approach notification is issued to the occupants of the service vehicle, informing them of the approach of the emergency vehicle both by voice and display. If the emergency vehicle approaches the service vehicle while the operator is on board, the voice notification is discontinued, and the approach of the emergency vehicle is notified by display (S214, S215). A vehicle control method that includes the following step in a process performed in at least one processing unit (11).

22. A vehicle control program used in a self-driving vehicle (Am) that is capable of being driven by an autonomous vehicle in which sleep is permitted for the occupant seated in the driver's seat, The vehicle becomes aware of the approach of an emergency vehicle to its own vehicle (S11), Determine whether the passenger is asleep or not (S13, S14), During the automated driving period under the aforementioned autonomous driving system, if the passenger is in the aforementioned sleep state and an emergency vehicle approaches the vehicle, a limited approach notification is implemented to show the passenger the approach of the emergency vehicle, while limiting the wake notification to prompt the passenger to wake up (S15). If it is determined that the occupant has changed from a sleeping state to an awake state after the commencement of the aforementioned limited approach notification, the vehicle will be notified by voice that it is performing corresponding driving control to the emergency vehicle (S56). A vehicle control program that causes at least one processing unit (11) to perform a process including the above.

23. A vehicle control program used in a self-driving vehicle (Am) that is capable of being driven by an autonomous vehicle in which sleep is permitted for the occupant seated in the driver's seat, The vehicle becomes aware of the approach of an emergency vehicle to its own vehicle (S11), Further determination is made as to whether the emergency vehicle is traveling in the same direction as the vehicle itself (S12), Determine whether the passenger is asleep or not (S13, S14), During the period of autonomous driving under the aforementioned autonomous driving system, In the first situation, where the approach of the emergency vehicle is known but it is unclear whether the emergency vehicle is traveling in the same direction, a limited approach notification is implemented to indicate the approach of the emergency vehicle to the passenger by display, while limiting the alert to wake the passenger. In the second situation, when it is determined that the emergency vehicle approaching the vehicle is traveling in the same direction, it is decided whether or not to implement the limited proximity notification depending on whether or not the occupant is asleep (S15, S16). A vehicle control program that causes at least one processing unit (11) to perform a process including the above.

24. A vehicle control program used in a service vehicle (Am) that provides mobility services and is capable of being driven by autonomous driving that allows sleep for a passenger seated in the driver's seat, The vehicle becomes aware of the approach of an emergency vehicle to its own vehicle (S211), The system determines whether or not an operator is on board the service vehicle operated by the automated driving system (S213), During the automated driving period under the aforementioned automated driving system, if the emergency vehicle approaches the service vehicle while the operator is not on board, an emergency approach notification is issued to the occupants of the service vehicle, informing them of the approach of the emergency vehicle both by voice and display. If the emergency vehicle approaches the service vehicle while the operator is on board, the voice notification is discontinued, and the approach of the emergency vehicle is notified by display (S214, S215). A vehicle control program that causes at least one processing unit (11) to perform a process including the above.