Presentation control device, presentation control program, automatic driving control device, and automatic driving control program

Abstract

An HMI control device disclosed herein functions as a presentation control device that controls the presentation of information to a passenger boarding a host vehicle. The HMI control device acquires detection information of an emergency vehicle approaching the host vehicle, and further ascertains whether there is more than one emergency vehicle approaching the host vehicle. The HMI control device changes the method of emergency vehicle approach notification indicating the approach of emergency vehicles to the passenger according to whether there is one emergency vehicle or more than one emergency vehicle approaching the host vehicle.

Description

Presentation control device, presentation control method, automatic driving control device, and automatic driving control method CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Patent Application No. 2023-202048 filed in Japan on November 29, 2023, and the contents of the original application are incorporated by reference in their entirety.

[0002] The disclosure of this specification relates to a presentation control technology for controlling the presentation of information, and an automatic driving control technology for autonomously driving a vehicle.

[0003] Patent Document 1 discloses a blind spot vehicle sound detection system that can detect the direction of another vehicle that is in the blind spot of an obstruction.

[0004] JP 2012-146149 A

[0005] The inventors of the present disclosure have conceived of utilizing the other vehicle detection technology described in Patent Document 1 to recognize emergency vehicles. Such detection technology may enable detection of multiple emergency vehicles.

[0006] The present disclosure aims to provide a presentation control device, a presentation control method, an automatic driving control device, and an automatic driving control method that are convenient for passengers when multiple emergency vehicles are detected.

[0007] In order to achieve the above object, one disclosed aspect is a presentation control device that controls the presentation of information to passengers in a vehicle, and is equipped with an information acquisition unit that acquires detection information of emergency vehicles approaching the vehicle and further determines whether there are multiple emergency vehicles approaching the vehicle, and a notification control unit that changes the method of approach notification that notifies passengers of the approach of an emergency vehicle depending on whether there are multiple emergency vehicles approaching the vehicle or only one emergency vehicle.

[0008] Another disclosed aspect is a presentation control method for controlling the presentation of information to passengers in a vehicle, the presentation control method including, in processing performed by at least one processing unit, the steps of acquiring detection information of an emergency vehicle approaching the vehicle, determining whether there are multiple emergency vehicles approaching the vehicle, and changing the method of approach notification that notifies passengers of the approach of an emergency vehicle depending on whether there is multiple emergency vehicles approaching the vehicle or whether there is only one emergency vehicle.

[0009] In these aspects, the approach notification method is changed depending on whether there are multiple emergency vehicles approaching the vehicle or only one emergency vehicle. Therefore, the occupants of the vehicle can understand from the approach notification whether there are multiple emergency vehicles approaching the vehicle. Therefore, when multiple emergency vehicles are detected, it is possible to present information that is convenient for the occupants.

[0010] Another disclosed aspect is an automatic driving control device that is capable of driving a vehicle using autonomous driving control, and is equipped with an information grasping unit that acquires detection information of emergency vehicles approaching the vehicle and further grasps whether there are multiple emergency vehicles approaching the vehicle, and an action control unit that changes the content of the evasive action of the vehicle to avoid the emergency vehicle depending on whether there are multiple emergency vehicles approaching the vehicle or whether there is only one emergency vehicle.

[0011] Another disclosed aspect is an automated driving control method capable of driving a vehicle through autonomous driving control, the automated driving control method including, in processing performed by at least one processing unit, the steps of acquiring detection information of an emergency vehicle approaching the vehicle, further determining whether there are multiple emergency vehicles approaching the vehicle, and changing the content of the evasive action of the vehicle to avoid the emergency vehicle depending on whether there are multiple emergency vehicles approaching the vehicle or whether there is only one emergency vehicle.

[0012] In these aspects, the content of the evasive action to avoid the emergency vehicle changes depending on whether there are multiple emergency vehicles approaching the host vehicle or only one emergency vehicle. Therefore, the host vehicle can take evasive action corresponding to the number of approaching emergency vehicles. Therefore, when multiple emergency vehicles are detected, it is possible to realize automated driving control that is convenient for passengers.

[0013] It should be noted that the reference numbers in parentheses in the claims merely indicate an example of the correspondence with the specific configurations in the embodiments described below, and do not limit the technical scope in any way. Furthermore, claims not explicitly stated in the claims may be combined together if no particular problems arise in the combination.

[0014] Fig. 1 is a diagram showing an overall view of an in-vehicle network including an autonomous driving ECU and an HMI control device according to an embodiment of the present disclosure. Fig. 2 is a block diagram showing details of the autonomous driving ECU and the HMI control device together with related configurations. Fig. 3 is a table listing types of sirens sounded by emergency vehicles. Fig. 4 is a flowchart showing details of emergency vehicle recognition processing performed by the autonomous driving ECU. Fig. 5 is a flowchart showing details of emergency vehicle avoidance processing performed by the autonomous driving ECU. Fig. 6 is a flowchart showing details of rescue action processing performed by the autonomous driving ECU. Fig. 7 is a flowchart showing details of emergency vehicle notification processing performed by the HMI control device.

[0015] The functions of the autonomous driving control device according to one embodiment of the present disclosure are realized by an autonomous driving ECU (Electronic Control Unit) 50 shown in FIG. 1. The autonomous driving ECU 50 is installed in a vehicle (hereinafter, host vehicle Am). By installing the autonomous driving ECU 50, the host vehicle Am becomes an autonomous driving vehicle or an autonomously driven vehicle equipped with an autonomous driving function. The autonomous driving level in this disclosure is based on the standards defined by the Society of Automotive Engineers. In this disclosure, autonomous driving control of autonomous driving level 3 or higher by the autonomous driving ECU 50 is referred to as "autonomous driving control."

[0016] During an autonomous driving period in which the host vehicle Am is driven by autonomous driving control by the autonomous driving ECU 50, the driver may be permitted to perform a specific action other than predefined driving (hereinafter referred to as a second task). During an autonomous driving period under autonomous driving level 3, the driver is legally permitted to perform the second task until a request for a driver handover is made by the autonomous driving ECU 50 in cooperation with the HMI (Human Machine Interface) control device 100. For example, actions such as watching entertainment content such as videos, operating a device such as a smartphone, and eating are considered as second tasks.

[0017] The autonomous driving ECU 50 is communicatively connected to a communication bus 99 of an in-vehicle network 1 mounted in the host vehicle Am. The communication bus 99 is connected to an in-vehicle monitor device 29, a periphery monitoring sensor 30, a locator 35, a navigation ECU 38, an in-vehicle communication device 39, a driving control ECU 40, an HMI control device 100, and the like. These nodes connected to the communication bus 99 can communicate with each other. Certain nodes among these ECUs, etc. may be electrically connected directly to each other and be able to communicate without going through the communication bus 99.

[0018] The in-vehicle monitor device 29 includes at least one in-vehicle camera that captures images of the interior of the vehicle Am and a control unit that controls the in-vehicle camera. The in-vehicle camera may be a visible light camera or a near-infrared camera combined with a near-infrared light source. The at least one in-vehicle camera is installed so as to capture an image of the occupant (driver) seated in the driver's seat and functions as a driver monitor. The in-vehicle monitor device 29 provides the images captured by the in-vehicle camera or the results of image analysis of the captured images to the autonomous driving ECU 50 and the HMI control device 100 as in-vehicle monitor information. The in-vehicle monitor information includes occupant identification information that identifies the occupant (mainly the driver) currently in the vehicle Am from among pre-registered occupants, as well as driver status information that indicates the driver's eyepoint position, line of sight, etc. The occupant identification information may identify only the driver among the occupants, or may include information identifying occupants other than the driver.

[0019] The perimeter monitoring sensor 30 is an autonomous sensor mounted on the host vehicle Am and monitors the environment surrounding the host vehicle Am. The perimeter monitoring sensor 30 is capable of detecting moving objects and stationary objects within a detection range around the host vehicle. The perimeter monitoring sensor 30 provides detection information of objects around the host vehicle to the autonomous driving ECU 50, etc. The perimeter monitoring sensor 30 includes a camera unit 31 and an exterior acoustic sensor 32. The perimeter monitoring sensor 30 may further include a millimeter-wave radar, a lidar, a sonar, etc.

[0020] The camera unit 31 includes a front camera module, a rear camera module, a left side camera module, and a right side camera module. The camera unit 31 includes multiple camera modules, so that it can capture images of the entire surroundings of the vehicle Am. The camera unit 31 outputs image data captured by each camera module or analysis information of the image data as detection information to the communication bus 99.

[0021] The exterior acoustic sensor 32 is primarily composed of a microphone element that converts sound into an electrical signal. The microphone element functions as a condenser microphone that outputs an electrical signal based on changes in capacitance caused by sound pressure vibrating a thin diaphragm. A MEMS (Micro Electro Mechanical Systems) microphone or the like can be used as the microphone element. Alternatively, the exterior acoustic sensor 32 may be a piezoelectric microphone using a piezoelectric sensor instead of a condenser microphone. The piezoelectric sensor converts sound into an electronic signal using the piezoelectric element. The exterior acoustic sensor 32 is held on the exterior structure of the host vehicle Am with the sound collection surface of the microphone element or the like facing the exterior structure. Multiple exterior acoustic sensors 32 are provided on the front, rear, left and right sides, ceiling, etc. of the host vehicle Am. The exterior acoustic sensors 32 provided at each location can detect incoming sounds from all around the host vehicle Am. The exterior acoustic sensor 32 outputs the sound data generated by each microphone element or analysis information of the sound data as detection information to the communication bus 99 .

[0022] The locator 35 includes a GNSS (Global Navigation Satellite System) receiver, an inertial sensor, etc. The locator 35 sequentially determines the position and traveling direction of the host vehicle Am by combining positioning signals received from multiple positioning satellites by the GNSS receiver, measurement results from the inertial sensor, and vehicle speed information output to the communication bus 99. The locator 35 sequentially outputs position information and direction information of the host vehicle Am based on the positioning results to the communication bus 99 as locator information.

[0023] The locator 35 also has a map database that stores map data. The map database is primarily composed of a large-capacity storage medium that stores a large amount of three-dimensional map data and two-dimensional map data. The three-dimensional map data is a so-called HD (High Definition) map and includes road information necessary for autonomous driving. Specifically, the three-dimensional map data includes three-dimensional road shape information and detailed information about each lane. The locator 35 can update the three-dimensional map data and two-dimensional map data to the latest information through external communication via the on-board communication device 39. The locator 35 reads map data around the current location from the map database and provides it to the autonomous driving ECU 50, etc., along with locator information.

[0024] The navigation ECU 38 acquires information about a destination specified by a passenger such as a driver based on operation information acquired from the HMI control device 100. The navigation ECU 38 acquires vehicle position information and direction information from the locator 35, and sets a route from the current position to the destination. The navigation ECU 38 provides route information indicating the set route to the destination to the autonomous driving ECU 50, the HMI control device 100, etc. The navigation ECU 38 works in conjunction with the HMI system 10 (described later) to provide route guidance to the destination by combining screen displays and voice messages, etc., and notifying the driver of the direction of travel of the vehicle Am at intersections, branching points, etc.

[0025] Here, a user terminal such as a smartphone or tablet may be connected to the in-vehicle network 1 or the HMI control device 100. Such a user terminal may provide the autonomous driving ECU 50 with information such as vehicle position information, direction information, and map data in place of the locator 35. Furthermore, the user terminal may provide the autonomous driving ECU 50 and the HMI control device 100 with information such as route information to a destination in place of the navigation ECU 38. Furthermore, the user terminal may play video content in place of the HMI system 10 described below or together with the HMI system 10.

[0026] The in-vehicle communication device 39 is an external communication unit mounted on the host vehicle Am. The in-vehicle communication device 39 functions as a V2X (Vehicle to Everything) communication device. The in-vehicle communication device 39 transmits and receives information via wireless communication with roadside devices installed on the side of the road. As an example, the in-vehicle communication device 39 receives congestion information, traffic regulation information, and the like from the roadside devices. The congestion information and traffic regulation information are, for example, VICS (registered trademark) information. The in-vehicle communication device 39 may further receive signal information indicating the lighting patterns of traffic signals, as well as detection information of stopped vehicles, parked vehicles, pedestrians, cyclists, and the like from the roadside devices. The in-vehicle communication device 39 provides the received congestion information, traffic regulation information, signal information, detection information, and the like to the navigation ECU 38, the autonomous driving ECU 50, the HMI control device 100, and the like.

[0027] The cruise control ECU 40 is an electronic control device that mainly includes a microcontroller. The cruise control ECU 40 generates vehicle speed information indicating the current traveling speed of the host vehicle Am based on detection signals from wheel speed sensors provided at the hub portions of each wheel, and sequentially outputs the generated vehicle speed information to the communication bus 99. The cruise control ECU 40 has at least the functions of a brake control ECU, a drive control ECU, and a steering control ECU. The cruise control ECU 40 continuously controls the braking force of each wheel, the output of the powertrain, and the steering angle based on either an operation command based on the driver's driving operation or a control command from the automatic driving ECU 50.

[0028] The HMI control device 100, together with a plurality of display devices, an audio device 24, an ambient light 25, an operation device 26, etc., constitutes an HMI system 10. The HMI system 10 has an input interface function that accepts operations by a passenger such as a driver of the host vehicle Am, and an output interface function that presents information to the driver. The HMI control device 100 functions as a presentation control device according to an embodiment of the present disclosure.

[0029] The display devices present information to the driver's vision by displaying images or the like. The display devices include a meter display 21, a center information display (hereinafter referred to as CID) 22, and a head-up display (hereinafter referred to as HUD) 23. The meter display 21 and the HUD 23 are display devices that present information to the driver sitting in the driver's seat. The CID 22 is a display device that can present information to a passenger sitting in the passenger seat as well as the driver. The CID 22 has a touch panel function and detects touch operations on the display screen by the driver or the like.

[0030] The audio device 24 has multiple speakers installed in the vehicle cabin. The audio device 24 reproduces, for example, notification sounds or voice messages through the speakers. The ambient lights 25 are provided on the instrument panel, center console, steering wheel, door trim, etc. The ambient lights 25 present information to the driver's peripheral vision through ambient displays that change the emitted light color.

[0031] The operation device 26 is an input unit that accepts user operations by the driver, etc. User operations related to activation and deactivation of the autonomous driving function, user operations related to setting a destination for route guidance, user operations related to setting air conditioning, etc. The operation device 26 includes a steering switch provided on the spokes of the steering wheel, an operation lever provided on the steering column, and a voice input device that recognizes what the passengers are saying.

[0032] 1 and 2 is a computer that mainly includes a processing unit 51, a RAM 52, a storage unit 53, an input / output interface 54, and a control circuit that includes a bus connecting these components. The processing unit 51 accesses the RAM 52 to execute various processes (instructions) for implementing the autonomous driving control method according to the present disclosure. The storage unit 53 stores various programs (autonomous driving control programs, etc.) that are executed by the processing unit 51. As the processing unit 51 executes the programs, the autonomous driving ECU 50 is configured with an information linkage unit 61, an environment recognition unit 62, a behavior determination unit 63, a control execution unit 64, etc. as functional units for implementing the autonomous driving function.

[0033] The information linking unit 61 enables information linking between the in-vehicle monitoring device 29, the navigation ECU 38, the in-vehicle communication device 39, the HMI control device 100, etc. and the autonomous driving ECU 50. The information linking unit 61 acquires occupant identification information from the in-vehicle monitoring device 29. The information linking unit 61 determines the type of occupant aboard the vehicle Am based on the occupant identification information. Specifically, the information linking unit 61 determines whether a medical professional such as a doctor, nurse, or paramedic is aboard the vehicle Am. The information linking unit 61 acquires route information from the navigation ECU 38 and provides the acquired route information to the environment recognition unit 62 and the behavior determination unit 63. The information linking unit 61 requests the navigation ECU 38 to change (reset) the destination or the planned driving route by outputting a reroute request to the navigation ECU 38. The information linking unit 61 provides the detection information received by the in-vehicle communication device 39 to the environment recognition unit 62.

[0034] The information linking unit 61 enables the HMI control device 100 to issue a notification synchronized with the operating state of the autonomous driving function by outputting a request to issue an in-vehicle notification to the HMI control device 100. The information linking unit 61 outputs, to the HMI control device 100, information that triggers the implementation of notifications related to the approach of an emergency vehicle, such as an emergency vehicle approach notification, an inquiry notification, and a warning notification, which will be described later.

[0035] The environment recognition unit 62 recognizes the driving environment of the host vehicle Am by combining the locator information and map data acquired from the locator 35, the detection information acquired from the perimeter monitoring sensor 30, and the vehicle speed information acquired from the communication bus 99. The environment recognition unit 62 may acquire the detection information received by the in-vehicle communication device 39 from the information linkage unit 61 and use it to recognize the driving environment. The environment recognition unit 62 acquires road information related to the road on which the host vehicle Am is traveling or the road on which the host vehicle Am is scheduled to travel, and grasps the relative positions and relative speeds of dynamic targets around the host vehicle Am, such as other vehicles traveling around the host vehicle Am. The environment recognition unit 62 sequentially provides the action determination unit 63 with the recognition results of the driving environment, including the road information and the grasped information of the dynamic targets.

[0036] The environment recognition unit 62 recognizes approaching sounds around the host vehicle Am. The environment recognition unit 62 performs an emergency vehicle recognition process (see FIG. 4 ), which will be described later. In the emergency vehicle recognition process, the environment recognition unit 62 recognizes the approach of emergency vehicles, including police vehicles, fire engines, ambulances, and the like, to the host vehicle Am by combining detection information based on the image data of the camera unit 31 with detection information based on the sound data of the exterior acoustic sensor 32. Unless otherwise specified, an emergency vehicle in the present disclosure refers to an emergency vehicle traveling for emergency purposes.

[0037] By using the sound data, the environment recognition unit 62 can recognize the approach of an emergency vehicle sounding a siren earlier than when using only image data. Additionally, by using the sound data, the environment recognition unit 62 identifies the type of siren being sounded by the emergency vehicle. The environment recognition unit 62 determines the tone of the siren, the length of each siren sound, and whether or not a warning bell is sounding (see FIG. 3 ). The environment recognition unit 62 can distinguish between a comfort siren and a harmonic siren, for example. Based on the determined type of siren, the environment recognition unit 62 determines the type of emergency vehicle approaching the vehicle Am and the state of emergency business.

[0038] When the environment recognition unit 62 detects that an emergency vehicle is approaching the host vehicle Am, it detects the relative position and movement direction of the emergency vehicle. As a result, the environment recognition unit 62 determines whether the emergency vehicle whose approach has been detected is moving away from the host vehicle Am (hereinafter, referred to as a "separating state"). Additionally, the environment recognition unit 62 further determines whether there are multiple emergency vehicles approaching the host vehicle Am based on the emergency vehicle detection information. Situations in which the host vehicle Am encounters multiple emergency vehicles include, for example, PA cooperation and the occurrence of a large-scale disaster. PA cooperation is a situation in which a fire engine (Pumper) and an ambulance (Ambulance) cooperate to carry out emergency response activities. Large-scale disasters include, for example, a building fire and a forest fire.

[0039] Here, when multiple emergency vehicles of the same type, in other words, emergency vehicles sounding the same siren, are approaching, it can be difficult to determine whether there are multiple vehicles. In this way, when there is a possibility that multiple emergency vehicles are approaching the host vehicle Am, but it cannot be determined that there are multiple vehicles, the environment recognition unit 62 generates estimation information that estimates the presence of multiple emergency vehicles. The emergency vehicle detection information, emergency vehicle departure information, number of emergency vehicles, estimated number of emergency vehicles, siren type information, and other information obtained by the environment recognition unit 62 are provided to the HMI control device 100 from the information linkage unit 61 as the above-mentioned emergency vehicle-related information.

[0040] When the autonomous driving ECU 50 has control of driving operations, the behavior judgment unit 63 generates a planned driving line for the vehicle Am to travel based on the results of recognition of the driving environment by the environment recognition unit 62, and outputs the generated planned driving line to the control execution unit 64.

[0041] When the action determination unit 63 recognizes that an emergency vehicle is approaching the host vehicle Am, it controls the action of the host vehicle Am (hereinafter, referred to as avoidance action) to avoid the emergency vehicle. When the action determination unit 63 recognizes that an emergency vehicle is approaching, it performs at least one of stop control, low-speed travel control, and reroute control based on an emergency vehicle avoidance process (see FIG. 5 ) described later. The stop control is control to stop the host vehicle Am at a position at least a predetermined distance away from the emergency vehicle. The low-speed travel control is control to suppress the travel speed of the host vehicle Am. The reroute control is control to reset the planned travel route of the host vehicle Am so as to avoid the estimated travel route of the emergency vehicle. The change of the planned travel route by the reroute control may be realized by the action determination unit 63 changing the planned travel line, or may be realized by the information linkage unit 61 outputting a reroute request to the navigation ECU 38.

[0042] When the autonomous driving ECU 50 has control of driving operations, the control execution unit 64 cooperates with the cruise control ECU 40 to execute acceleration / deceleration control, steering control, and the like of the host vehicle Am in accordance with the planned driving line generated by the action determination unit 63. Specifically, the control execution unit 64 generates control commands based on the planned driving line and sequentially outputs the generated control commands to the cruise control ECU 40.

[0043] <Details of HMI Control Device> The HMI control device 100 is a computer mainly including a control circuit equipped with a processing unit 11, a RAM 12, a storage unit 13, an input / output interface 14, and a bus connecting these. The HMI control device 100 functions as a presentation control device. The HMI control device 100 presents information related to autonomous driving in cooperation with the autonomous driving ECU 50. The processing unit 11 accesses the RAM 12 to execute various processes (instructions) for realizing the presentation control method according to the present disclosure. The storage unit 13 stores various programs (presentation control programs, etc.) executed by the processing unit 11. By executing the programs by the processing unit 11, an information linking unit 82, a presentation control unit 83, etc. are configured in the HMI control device 100 as functional units related to presentation control.

[0044] The information linking unit 82 links with the autonomous driving ECU 50, the navigation ECU 38, etc. The information linking unit 82 enables information sharing between the autonomous driving ECU 50, the navigation ECU 38, and the HMI control device 100. The information linking unit 82 provides operation information of user operations input to the operation device 26, etc., to the autonomous driving ECU 50, the navigation ECU 38, etc. The information linking unit 82 acquires control status information indicating the control state of the autonomous driving function, and a request for implementing notifications related to the autonomous driving function, etc., from the autonomous driving ECU 50. The information linking unit 82 acquires route information indicating a set route to the destination from the navigation ECU 38.

[0045] The information-coordination unit 82 sequentially acquires emergency vehicle-related information provided by the information-coordination unit 61 of the autonomous driving ECU 50. Based on the acquired detection information, the information-coordination unit 82 identifies emergency vehicles approaching the host vehicle Am. Based on the acquired departure information, the information-coordination unit 82 identifies that an approaching emergency vehicle has left the host vehicle Am, for example, that the emergency vehicle has overtaken the host vehicle Am. Based on the acquired emergency vehicle count information, the information-coordination unit 82 identifies whether there are multiple emergency vehicles approaching the host vehicle Am. Furthermore, based on the acquired estimation information, the information-coordination unit 82 identifies a situation in which it cannot be determined that there are multiple emergency vehicles approaching the host vehicle Am, and therefore, there is a possibility that there are multiple emergency vehicles. Based on the acquired siren type information, the information-coordination unit 82 identifies the type of siren sounded by the emergency vehicle, thereby identifying the type of approaching emergency vehicle and the status of the emergency mission (see FIG. 3 ).

[0046] The presentation control unit 83 comprehensively controls the presentation of information using multiple display devices, the audio device 24, the ambient light 25, etc. For example, when the autonomous driving ECU 50 plans to end autonomous driving control, the presentation control unit 83 issues a notification requesting the driver to take over driving, based on an implementation request acquired by the information linkage unit 82. The presentation control unit 83 also issues notifications related to route guidance, based on route information generated by the navigation ECU 38.

[0047] In an emergency vehicle notification process (see FIG. 7 ) described later, the presentation control unit 83 performs in-vehicle notifications related to emergency vehicles based on the emergency vehicle-related information acquired by the information linking unit 82. The presentation control unit 83 performs notifications related to emergency vehicles (hereinafter referred to as emergency vehicle notifications), such as an approaching emergency vehicle notification, an inquiry notification, a warning notification, and a control duration notification.

[0048] The emergency vehicle approaching notification is a notification that notifies occupants of the approach of an emergency vehicle. As the emergency vehicle approaching notification, the presentation control unit 83 displays an image (emergency vehicle icon) and a text message indicating the approaching emergency vehicle on the meter display 21 (or HUD 23) and the CID 22. When the information linking unit 82 acquires siren type information, the presentation control unit 83 adds detailed information to the emergency vehicle approaching notification and presents information related to the siren type to the occupants. Specifically, the detailed information notifies the occupants of the type of approaching emergency vehicle and the details of the emergency business by an icon or a text message. In addition, as the emergency vehicle approaching notification, the presentation control unit 83 plays an audio message or notification sound indicating the approaching emergency vehicle inside the vehicle cabin via the speaker of the audio device 24.

[0049] The inquiry notification and the attention alert notification are notifications that are made when estimated information is acquired by the information linking unit 82. The inquiry notification is a notification that inquires the passenger about whether or not multiple emergency vehicles are present. The inquiry notification is made using, for example, the CID 22. In the inquiry notification, an inquiry message such as "Can you see multiple emergency vehicles?" is displayed on the CID 22 together with touch icons such as "Yes" and "No." In response to such an inquiry notification, the passenger, such as the driver, inputs a selection operation to the touch icon on the CID 22.

[0050] The attention alert is an alert that warns the driver that multiple emergency vehicles may be approaching. The attention alert is issued, for example, using the CID 22. In the inquiry alert, a text message such as "Multiple emergency vehicles may be approaching" is displayed on the meter display 21 (or the HUD 23) and the CID 22. The presentation control unit 83 may issue only one of the inquiry alert and the attention alert. Alternatively, the presentation control unit 83 may issue an audio message in the vehicle cabin using the audio device 24 as the inquiry alert and the attention alert.

[0051] The control duration notification is a notification that shows time information related to stop control or low-speed traveling control to the occupants of the host vehicle Am. The control duration notification is performed, for example, using the CID 22, when the autonomous driving ECU 50 performs stop control or low-speed traveling control in order to take evasive action to avoid an emergency vehicle. The control duration notification shows the occupants the approximate remaining time for which the evasive action by stop control or low-speed traveling control will continue, in other words, the predicted timing for the control to end. The remaining time presented in the control duration notification may be increased or decreased sequentially depending on the behavior of the emergency vehicle.

[0052] At least some of the notifications included in the emergency vehicle notification, such as the emergency vehicle approach notification, the inquiry notification, the attention-call notification, and the control duration notification, may be controlled by the information linking unit 61. In this embodiment, based on an implementation request output from the information linking unit 61 to the information linking unit 82, the information linking unit 82 implements the notification corresponding to the implementation request.

[0053] <Response Control to Emergency Vehicles During Level 3 Autonomous Driving Period> The autonomously driven vehicles described so far need to respond to emergency vehicles approaching their own vehicle Am during the autonomous driving period, regardless of the driver's judgment. When an emergency vehicle is detected, it is more convenient for the passengers if the response control to the emergency vehicle is changed depending on the number of emergency vehicles, the detection timing, etc., rather than performing a uniform response control. Details of the emergency vehicle recognition process, emergency vehicle avoidance process, and emergency vehicle notification process performed to perform response control to emergency vehicles in the system disclosed herein will be described below based on FIGS. 4 to 7 and with reference to FIGS. 1 and 2.

[0054] [Emergency Vehicle Recognition Process] In the emergency vehicle recognition process shown in Figure 4, the approach of an emergency vehicle is recognized by the environment recognition unit 62. The environment recognition unit 62 can detect the approach of multiple emergency vehicles by using acoustic recognition technology. The emergency vehicle recognition process is started when the autonomous driving ECU 50 is activated, and is repeatedly performed until the autonomous driving ECU 50 stops operating.

[0055] In S11 of the emergency vehicle recognition processing, the environment recognition unit 62 acquires sound data of external vehicle sounds generated by the external vehicle acoustic sensor 32. The environment recognition unit 62 may acquire analysis information of the sound data, instead of or together with the sound data, from a data processing unit associated with the external vehicle acoustic sensor 32. In S12, the environment recognition unit 62 determines whether or not a siren sound indicating the presence of an emergency vehicle is present, based on the acquired sound data.

[0056] If there is no siren sound from an emergency vehicle (S12: NO), the current external sound recognition process is temporarily terminated. On the other hand, if there is a siren sound from an emergency vehicle (S12: YES), the environment recognition unit 62 acquires detection information of an emergency vehicle approaching the host vehicle Am. Then, in S13, the environment recognition unit 62 analyzes the sound data to identify the type of siren sound being emitted by the emergency vehicle approaching the host vehicle Am.

[0057] Specifically, the environment recognition unit 62 determines the type of emergency vehicle approaching the vehicle Am and the state of emergency business, and generates siren type information based on the determination result. The emergency vehicle-related information generated by the above emergency vehicle recognition process is sequentially provided to the behavior determination unit 63 and the HMI control device 100, and is referenced in the emergency vehicle avoidance process and emergency vehicle notification process.

[0058] In S14, the environment recognition unit 62 analyzes the sound data to determine the number of emergency vehicles approaching the host vehicle Am. Specifically, the environment recognition unit 62 determines whether one or more emergency vehicles are approaching the host vehicle Am, or whether the number of emergency vehicles cannot be determined. If the environment recognition unit 62 determines that multiple emergency vehicles are approaching the host vehicle Am, it generates number information indicating the number of host vehicles Am approaching the host vehicle Am. In addition, if the number of emergency vehicles cannot be determined, the environment recognition unit 62 generates estimation information that estimates the presence of multiple emergency vehicles.

[0059] [Emergency vehicle avoidance processing] The emergency vehicle avoidance processing shown in Figure 5 is processing for controlling the behavior of the host vehicle Am so as to avoid an emergency vehicle. In the emergency vehicle avoidance processing, the content of the avoidance processing action of the host vehicle Am for avoiding the emergency vehicle changes depending on whether there are multiple emergency vehicles approaching the host vehicle Am or whether there is only one emergency vehicle. As with the emergency vehicle recognition processing (see Figure 4), the emergency vehicle avoidance processing is started based on the activation of the autonomous driving ECU 50 and is repeatedly performed until the autonomous driving ECU 50 stops operating.

[0060] In S31 of the emergency vehicle avoidance process, emergency vehicle-related information generated by acoustic sensing performed in the emergency vehicle recognition process is acquired. Then, in S32, the emergency vehicle-related information is referenced to determine whether or not the sound of an emergency vehicle, such as a siren or a warning bell, has been detected. If the sound of an emergency vehicle is not detected and the approach of an emergency vehicle is not recognized (S32: NO), the state of waiting for the approach of an emergency vehicle is maintained.

[0061] On the other hand, if the sound of an emergency vehicle is detected and the approach of the emergency vehicle is recognized (S32: YES), it is determined in S33 whether or not the approach of multiple emergency vehicles has been detected. If the approach of only one emergency vehicle has been detected (S33: NO), the behavior determination unit 63 performs behavior control in S34 to cause the host vehicle Am to take normal avoidance action. Specifically, the behavior determination unit 63 performs stop control or low-speed travel control. Note that normal avoidance action may be taken even when the number of emergency vehicles is not determined.

[0062] The behavior determination unit 63 performs stop control to cause the host vehicle Am to leave the host vehicle's lane and move to the shoulder, and stops the host vehicle Am at a shelter that is found on the shoulder. The behavior determination unit 63 also performs low-speed travel control to offset the position of the host vehicle Am within the host vehicle's lane so as to give way to an emergency vehicle while continuing to travel at a low speed.

[0063] On the other hand, if the approach of multiple emergency vehicles is detected (S33: YES), the behavior determination unit 63 performs behavior control in S35 to cause the host vehicle Am to perform evasive action to avoid the multiple emergency vehicles (hereinafter, multiple-vehicle avoidance action). Specifically, the behavior determination unit 63 performs reroute control. In the reroute control, the behavior determination unit 63 resets the planned traveling route of the host vehicle Am so that the host vehicle Am avoids the estimated traveling routes of the multiple emergency vehicles and travels in a direction where no emergency vehicles are present.

[0064] In S36, the behavior determination unit 63 determines whether emergency vehicles have been detected consecutively. If the behavior determination unit 63 detects another emergency vehicle within a predetermined time after the first emergency vehicle previously detected has left the host vehicle Am, the behavior determination unit 63 determines that multiple emergency vehicles have been detected consecutively. The predetermined time used in the emergency vehicle avoidance processing is set to, for example, several seconds to several tens of seconds from the time when information about the departure of the first emergency vehicle is acquired or from the time when normal driving control is resumed after the completion of evasive action against the first emergency vehicle.

[0065] If it is determined that emergency vehicles have been detected consecutively (S36: YES), the behavior determination unit 63 performs behavioral control in S37 to cause the host vehicle Am to perform a second avoidance action. In the second avoidance action, in order to avoid the second emergency vehicle, an avoidance action that is different in content from the initial (normal) avoidance action to avoid the first emergency vehicle is performed. Specifically, the behavior determination unit 63 performs reverse control or reroute control. On the other hand, if it is determined that emergency vehicles have not been detected consecutively (S36: NO), the current emergency vehicle avoidance process is terminated.

[0066] The behavior determination unit 63 uses reverse control to move the host vehicle Am back so that the host vehicle Am returns to the evacuation location where the host vehicle Am was stopped in the first evasive action. The behavior determination unit 63 stops the host vehicle Am at the evacuation location until the second emergency vehicle passes the host vehicle Am. Furthermore, the behavior determination unit 63 uses reroute control to reset the planned driving route of the host vehicle Am so that the host vehicle Am leaves the road where the two emergency vehicles are heading.

[0067] 6 is a process for controlling the behavior of the host vehicle Am so that the host vehicle Am follows an emergency vehicle. Similar to the emergency vehicle avoidance process (see FIG. 5), the rescue action process is started based on activation of the autonomous driving ECU 50 and is repeatedly performed until the autonomous driving ECU 50 stops operating.

[0068] In S41 of the rescue action processing, emergency vehicle-related information generated by acoustic sensing performed in the emergency vehicle recognition processing is acquired. Then, in S42, it is determined whether the approach of multiple emergency vehicles has been detected and whether emergency vehicles have been detected consecutively. If there are not multiple emergency vehicles approaching the host vehicle Am and no consecutive emergency vehicles have been detected (S42: NO), the state of waiting for the detection of multiple or consecutive emergency vehicles is maintained.

[0069] On the other hand, if the approach of multiple emergency vehicles is detected, or if emergency vehicles are detected consecutively (S42: YES), in S43, the information linking unit 61 acquires occupant identification information from the in-vehicle monitoring device 29. Based on the acquired occupant identification information, in S44, the information linking unit 61 determines whether a specific type of occupant, specifically, a medical professional, is aboard the host vehicle Am. If a medical professional is not aboard the host vehicle Am (S44: NO), in S45, the behavior determining unit 63 performs behavioral control to cause the host vehicle Am to perform normal avoidance behavior. On the other hand, if a medical professional is aboard the host vehicle Am (S44: YES), in S46, the behavior determining unit 63 causes the host vehicle Am to perform an action to approach the emergency vehicle. Specifically, the behavior determining unit 63 controls the traveling of the host vehicle Am to pursue the emergency vehicle.

[0070] [Emergency Vehicle Notification Process] The emergency vehicle notification process shown in Figure 7 is a process for notifying occupants of an approaching emergency vehicle. In the emergency vehicle notification process, the content of the emergency vehicle approach notification changes depending on whether there are multiple (two or more) emergency vehicles approaching the host vehicle Am or whether there is only one emergency vehicle. The emergency vehicle avoidance process is started based on the activation of the autonomous driving ECU 50 and the HMI control device 100, and is repeatedly performed until these operations are stopped.

[0071] In S51 of the emergency vehicle notification processing, the information linking unit 82 acquires emergency vehicle-related information, including emergency vehicle detection information, from the HMI control device 100. In S52, the information linking unit 82 determines whether or not an emergency vehicle is approaching the host vehicle Am based on the acquired detection information. If there is no detection information indicating an approaching emergency vehicle (S52: NO), the information acquisition processing continues in S51.

[0072] On the other hand, if there is detection information indicating the approach of an emergency vehicle (S52: YES), in S53 the information linking unit 82 determines whether there are multiple emergency vehicles approaching the host vehicle Am based on the acquired number information and estimation information, etc. If there is only one emergency vehicle approaching the host vehicle Am (S53: NO, only one vehicle), in S54 the information linking unit 82 sets the emergency vehicle notification implementation mode to the normal notification mode.

[0073] On the other hand, if there are multiple emergency vehicles approaching the vehicle Am (YES at S53), the notification control unit 83 sets the emergency vehicle notification mode to a multiple vehicle notification mode at S55. In the multiple vehicle notification mode, the approaching emergency vehicle notification is emphasized more than in the normal notification mode.

[0074] Specifically, the background color or highlight frame color of the emergency vehicle icon, text message, etc. displayed on the display device is changed from an attention-grabbing color (e.g., yellow or amber) in the normal alert mode to a warning color (e.g., red) in the multiple vehicle alert mode. Furthermore, in the multiple vehicle alert mode, the display size of the emergency vehicle icon, text message, etc. is enlarged compared to the normal alert mode. Furthermore, the volume of the voice message played by the audio device 24 is increased in the multiple vehicle alert mode compared to the normal alert mode. Additionally, in the multiple vehicle alert mode, the number of information presentation devices used to notify approaching emergency vehicles may be increased compared to the normal alert mode. Furthermore, in the multiple vehicle alert mode, a text message or voice message notifying that multiple vehicles are approaching may be added.

[0075] Furthermore, in the multiple vehicle notification mode, a control duration notification is implemented. When multiple emergency vehicles are approaching the vehicle Am, the vehicle Am will be stopped or traveling at a low speed for a long period of time, and the control duration notification informs the occupants of the remaining duration of the avoidance action. In contrast, when there is only one approaching emergency vehicle, the stopped time or the slow-travel time is unlikely to be long. Therefore, in the normal notification mode, a control duration notification is not implemented.

[0076] If the number of emergency vehicles approaching the host vehicle Am cannot be determined (S53: NO, there is a possibility that there are multiple vehicles), the presentation control unit 83 issues at least one of an inquiry notification and a warning notification in S56. Note that if the occupant performs a touch operation indicating that multiple emergency vehicles are approaching in response to the inquiry notification issued in S56, it may be determined in S33 of the emergency vehicle avoidance processing (see FIG. 5) that multiple emergency vehicles are approaching.

[0077] In S57, the information linking unit 82 determines whether siren type information is present. If siren type information is present, the presentation control unit 83 references the siren type information to determine the type of siren sounded by the emergency vehicle. The presentation control unit 83 then determines whether a specific (rare) emergency vehicle sound is present. If a specific emergency vehicle sound is present (S57: YES), the presentation control unit 83 adds detailed information (type description) related to the siren type, specifically, information indicating the type of emergency vehicle and the status of the emergency mission, to the emergency vehicle approach notification in S58. In this case, the emergency vehicle icon is changed to an image corresponding to the type of approaching emergency vehicle. Furthermore, a text message indicating the nature of the emergency mission, such as "En route to the fire scene," is additionally displayed. Note that if there is no siren type information or no specific emergency vehicle sound (S57: NO), S58 is skipped.

[0078] In S59, the information linking unit 82 determines whether emergency vehicles have been detected consecutively. If the information linking unit 82 detects another emergency vehicle within a predetermined time after the first emergency vehicle previously detected has left the host vehicle Am, the information linking unit 82 determines that multiple emergency vehicles have been detected consecutively. The predetermined time used in the emergency vehicle notification process is different from the predetermined time used in the emergency vehicle avoidance process (see FIG. 5 ). Specifically, the predetermined time is set to, for example, about one minute from the time when information about the departure of the first emergency vehicle is acquired or from the time when normal driving control is resumed after the completion of evasive action against the first emergency vehicle.

[0079] If it is determined that emergency vehicles have been detected consecutively (S59: YES), the information linking unit 82 determines that multiple emergency vehicles are approaching the host vehicle Am. In this case, the presentation control unit 83 changes the amount of information in the approaching emergency vehicle notification in S60. Specifically, the presentation control unit 83 reduces the amount of information in the approaching emergency vehicle notification. For example, the presentation control unit 83 reduces the amount of information in the approaching emergency vehicle notification by continuing to display an emergency vehicle icon while hiding a text message. Furthermore, the presentation control unit 83 reduces the amount of information in the approaching emergency vehicle notification by playing only a notification sound instead of a voice message. Note that if it is determined that emergency vehicles have not been detected consecutively (S59: NO), S60 is skipped.

[0080] Summary of the embodiment In the embodiment described so far, the method of notifying the approaching emergency vehicle changes depending on whether multiple emergency vehicles are approaching the host vehicle Am or only one emergency vehicle. Therefore, the occupants of the host vehicle Am can determine from the approaching emergency vehicle notification whether multiple emergency vehicles are approaching the host vehicle Am. Therefore, when multiple emergency vehicles are detected, it is possible to present information that is convenient for the occupants.

[0081] In addition, in this embodiment, the more the number of emergency vehicles approaching the vehicle, the more emphasized the notification of approaching emergency vehicles becomes. Therefore, the notification of approaching emergency vehicles can clearly notify the passengers that multiple emergency vehicles are approaching the vehicle.

[0082] In this embodiment, the type of siren used by the emergency vehicle is also identified. Additional information related to the type of siren is then provided to the passenger. Therefore, even if the passenger does not fully understand the meaning of each of the multiple types of sirens, they can still correctly understand the status of the approaching emergency vehicle by recognizing the additional information.

[0083] Furthermore, in this embodiment, when the number of emergency vehicles approaching the host vehicle Am is not determined, estimation information is obtained to estimate the presence of multiple emergency vehicles. Then, based on the estimation information, at least one of an inquiry notification is issued to inquire about the presence of multiple emergency vehicles and a warning notification to alert the occupant to the possibility that multiple emergency vehicles are approaching is issued. Therefore, even if the number of emergency vehicles may not be correctly detected, for example, in a situation where multiple approaching emergency vehicles with the same siren are sounding, the autonomous driving ECU 50 can accurately explain the detection status to the occupant. Additionally, the autonomous driving ECU 50 can more accurately grasp the status of approaching emergency vehicles based on the occupant's input in response to the inquiry notification. Furthermore, the warning notification can correctly alert the occupant to the possibility that multiple emergency vehicles are approaching.

[0084] Additionally, in this embodiment, when the autonomous driving ECU 50 takes evasive action to avoid the approaching emergency vehicles due to multiple approaching emergency vehicles, the control duration notification notifies the occupants of the time when the evasive action will end. Therefore, even if the host vehicle Am is stopped or traveling at a low speed for a long period of time due to evasive action against multiple emergency vehicles, the occupants' anxiety about whether the autonomous driving ECU 50 is functioning normally can be alleviated.

[0085] In this embodiment, if detection information of another emergency vehicle approaching the host vehicle Am is acquired after the emergency vehicle has left the host vehicle Am, it is determined that there are multiple emergency vehicles approaching the host vehicle Am. As a result, even in a situation where multiple emergency vehicles approach the host vehicle Am at different times, it is possible to issue a warning equivalent to that issued in a situation where multiple emergency vehicles approach the host vehicle Am at the same time.

[0086] Furthermore, in this embodiment, if the approach of a second emergency vehicle is detected within a predetermined time after the emergency vehicle has left the vehicle Am, the amount of information presented to the occupant in the emergency vehicle approach notification changes. In this way, when successive emergency vehicles approach, the occupant is prepared for the emergency vehicle by the time the occupant detects the approach of the first emergency vehicle. Therefore, by modifying the mode, such as by reducing the amount of information in the approach notification for the second emergency vehicle, it is possible to present information that is easy for the occupant to understand and that is not bothersome.

[0087] In addition, in this embodiment, the content of the avoidance action to avoid the emergency vehicle changes depending on whether there are multiple emergency vehicles approaching the host vehicle Am or only one emergency vehicle. Therefore, the host vehicle Am can take an avoidance action corresponding to the number of approaching emergency vehicles. Therefore, when multiple emergency vehicles are detected, it is possible to realize automated driving control that is convenient for occupants.

[0088] In this embodiment, when multiple emergency vehicles approach the host vehicle Am, reroute control is implemented to reset the planned driving route of the host vehicle Am so as to avoid the estimated driving routes of the emergency vehicles. Therefore, when a large-scale disaster occurs in a nearby area and it is estimated that multiple emergency vehicles may approach, the implementation of reroute control makes it possible to move the host vehicle Am away from the area where multiple emergency vehicles are gathering. As a result, it is possible to avoid a situation in which the host vehicle Am repeatedly takes evasive action, such as stopping, every time an approaching emergency vehicle is detected.

[0089] Furthermore, in this embodiment, if the approach of another emergency vehicle is detected within a predetermined time after the emergency vehicle has left the host vehicle Am, a second avoidance action that is different from the avoidance action taken to avoid the first emergency vehicle is taken to avoid the second emergency vehicle. As a result of the above, the autonomous driving ECU 50 can select an appropriate avoidance action for the second emergency vehicle.

[0090] In addition, in this embodiment, as a second avoidance action to avoid the second emergency vehicle, reverse control, which reverses the host vehicle Am, or reroute control, which resets the planned driving route of the host vehicle Am so as to avoid the estimated driving route of the emergency vehicle, is performed. If reverse control is performed, the host vehicle Am can quickly give way to the emergency vehicle. Furthermore, if reroute control is performed, the host vehicle Am can leave the area where multiple emergency vehicles are gathering.

[0091] In this embodiment, the type of passenger aboard the host vehicle Am is identified. If multiple emergency vehicles are approaching the host vehicle Am and a specific type of passenger is aboard the host vehicle Am, control is implemented to move the vehicle closer to the emergency vehicle. Based on the above, if the passenger has the skills required for a disaster site where an emergency vehicle will rush to, the autonomous driving ECU 50 can switch the vehicle control of the host vehicle Am to have the passenger head to the disaster site. As a result, it is possible to contribute to saving lives.

[0092] In the above embodiment, the autonomous driving ECU 50 corresponds to the "autonomous driving control device," the information linking unit 61 corresponds to the "vehicle interior recognition unit," the environment recognition unit 62 corresponds to the "information recognition unit," and the behavior determination unit 63 corresponds to the "behavior control unit." Also, the information linking unit 82 corresponds to the "information acquisition unit," the presentation control unit 83 corresponds to the "notification control unit," and the HMI control device 100 corresponds to the "presentation control device."

[0093] (Other Embodiments) Although one embodiment of the present disclosure has been described above, the present disclosure should not be construed as being limited to the above embodiment, and can be applied to various embodiments and combinations within the scope that does not deviate from the gist of the present disclosure.

[0094] In the above embodiment, the method of notifying the occupants and the policy of driving control for avoiding the emergency vehicle were both changed depending on whether a single emergency vehicle was detected or whether multiple emergency vehicles were detected. In contrast, in Modification 1 of the above embodiment, the method of notifying the occupants of the approaching emergency vehicle is changed depending on whether multiple emergency vehicles are detected, but the avoidance action of the host vehicle Am is not changed. Furthermore, in Modification 2 of the above embodiment, the avoidance action of the host vehicle Am is changed depending on whether multiple emergency vehicles are detected, but the method of notifying the occupants of the approaching emergency vehicle is not changed.

[0095] In Modification 3 of the above embodiment, the host vehicle Am equipped with the HMI control device 100 does not have an automatic driving ECU 50. In other words, the host vehicle Am does not have an automatic driving function. The HMI control device 100 has a function equivalent to the environment recognition unit 62, and acquires detection information of an emergency vehicle approaching the host vehicle Am based on sound data acquired from the exterior acoustic sensor 32. As in Modification 3, the HMI control device 100 can present information that is convenient for occupants even in a manually driven vehicle.

[0096] The autonomous driving ECU 50 of the above embodiment mainly performs autonomous driving control at autonomous driving level 3. On the other hand, the autonomous driving ECU 50 of modified example 4 of the above embodiment performs autonomous driving control at autonomous driving level 4 or autonomous driving level 5, where the driver does not have to take over driving. Furthermore, the autonomous driving ECU 50 of modified example 5 of the above embodiment causes the host vehicle Am to autonomously drive without a driver on board, under remote monitoring by a remote monitor. In the host vehicle Am of modified examples 4 and 5 above, it is also possible to provide information presentation and autonomous driving control that is convenient for passengers.

[0097] In the above embodiment, the method of notification of approaching emergency vehicles and the content of the avoidance action were switched depending on whether there was one emergency vehicle or two or more. In a sixth modification of the above embodiment, at least one of the method of notification of approaching emergency vehicles and the content of the avoidance action is further changed as multiple emergency vehicles are detected and the number of detected emergency vehicles increases to two, three, four, and so on. Furthermore, in the above embodiment, the greater the number of emergency vehicles, the stronger the emphasis of the approach notification. In contrast, in a seventh modification of the above embodiment, the greater the number of emergency vehicles, the weaker the emphasis of the approach notification may be.

[0098] The information linking unit 61 in the above embodiment acquires passenger identification information from the in-vehicle monitor device 29. Such passenger identification information is not limited to the photographed information obtained by the in-vehicle monitor device 29. For example, various pre-registered information can be used as passenger identification information, such as information pre-registered by the owner of the vehicle Am, information set by the user by touching the CID 22 (navigation screen, etc.), and registration information for rental cars and car dispatch services.

[0099] In the above embodiment, when emergency vehicles are detected consecutively, the amount of information provided to the passenger in the emergency vehicle approach notification is reduced. In contrast, in Modification 8 of the above embodiment, when a second or subsequent emergency vehicle raises a different warning from the first emergency vehicle that has already been notified, the amount of information provided to the passenger in the emergency vehicle approach notification is increased. Furthermore, the rescue action process (see FIG. 6 ) that directs specific passengers, such as medical personnel, to a disaster site, etc., may not be performed. Additionally, the type explanation (see S58 in FIG. 7 ) when a special emergency vehicle sound is detected may also be omitted.

[0100] The autonomous driving ECU 50 in the above embodiment can perform stop control, low-speed driving control, reroute control, reverse control, etc. In contrast, the autonomous driving ECU 50 according to the ninth modification of the above embodiment can perform only some of these behavior controls.

[0101] At least some of the functions of the autonomous driving ECU 50 may be implemented in another control device provided in the in-vehicle network 1. For example, in a modification 10 of the above embodiment, a dedicated ECU (hereinafter referred to as an acoustic analysis ECU) that performs analysis processing of sound data and outputs the analysis information to the environment recognition unit 62 is provided in the in-vehicle network 1. The function of recognizing emergency vehicles is implemented in the acoustic analysis ECU. In this modification 10, a system including the autonomous driving ECU 50 and the acoustic analysis ECU corresponds to an "autonomous driving control device."

[0102] At least some of the functions of the HMI control device 100 may be implemented in another control device provided in the in-vehicle network 1. For example, in Modification 11 of the above embodiment, the HMI control device 100 cooperates with the autonomous driving ECU 50 to perform emergency vehicle alerts. In this Modification 11, a system including the HMI control device 100 and the autonomous driving ECU 50 corresponds to a "presentation control device." Furthermore, the processing functions of the HMI control device 100 may be implemented in control circuits of display devices such as the meter display 21 and the CID 22.

[0103] In a twelfth modification of the above embodiment, the functions of the HMI control device 100 are integrated into the autonomous driving ECU 50. In a thirteenth modification of the above embodiment, the functions of the cruise control ECU 40 are integrated into the autonomous driving ECU 50. In these twelfth and thirteenth modifications, the autonomous driving ECU 50 integrated with other ECUs, in other words, the integrated ECU, corresponds to the "presentation control device" and the "autonomous driving control device."

[0104] In the above embodiments, the functions provided by the HMI control device and the autonomous driving ECU can be provided by software and hardware that executes the software, software alone, hardware alone, or a combination of these. Furthermore, when such functions are provided by electronic circuits as hardware, each function can also be provided by digital circuits including multiple logic circuits or analog circuits. Furthermore, the software for realizing the autonomous driving function may include, at least in part, code automatically generated by, for example, a neural network or language model trained using real-world camera footage.

[0105] Each processing unit in the above embodiments is hardware for arithmetic processing coupled to a RAM. The processing unit includes at least one arithmetic core, such as a central processing unit (CPU) and a graphics processing unit (GPU). The processing unit may further include a field-programmable gate array (FPGA), a neural network processing unit (NPU), and other IP cores with dedicated functions. Such processing units may be individually implemented on a printed circuit board, or may be implemented together with RAM in a system on a chip (SoC). The SoC may also include an application-specific integrated circuit (ASIC), an FPGA, an NPU, and other IP cores.

[0106] In the above embodiments, the form of the storage medium (non-transitory tangible storage medium) that stores various programs and the like may be changed as appropriate. Furthermore, the storage medium is not limited to a configuration provided on a circuit board, but may be provided in the form of a memory card or the like, inserted into a slot, and electrically connected to a control circuit such as an autonomous driving ECU. Furthermore, the storage medium may be an optical disk, hard disk drive, solid state drive, or the like that serves as a source from which programs are copied or distributed to the autonomous driving ECU or the like.

[0107] The vehicle equipped with the above-described HMI control device and autonomous driving ECU is not limited to a typical private passenger car (Personally Owned Vehicle, POV). The vehicle equipped with the HMI control device and autonomous driving ECU may be a rental car vehicle, a manned taxi vehicle, a ride-sharing vehicle, a freight vehicle, a bus, etc. The vehicle equipped with the HMI control device and autonomous driving ECU may be a right-hand drive vehicle or a left-hand drive vehicle. Furthermore, the traffic environment in which the vehicle travels may be a traffic environment based on left-hand traffic or a traffic environment based on right-hand traffic. The presentation control and autonomous driving control according to the present disclosure may be optimized as appropriate according to the road traffic laws of each country and region, as well as the steering wheel position of the vehicle.

[0108] The controller and methods described herein may be implemented by a special-purpose computer comprising a processor programmed to perform one or more functions embodied in a computer program. Alternatively, the apparatus and methods described herein may be implemented by special-purpose hardware logic circuitry. Alternatively, the apparatus and methods described herein may be implemented by one or more special-purpose computers comprising a processor executing a computer program in combination with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0109] (Disclosure of Technical Ideas) This specification discloses multiple technical ideas described in the following multiple clauses. Some clauses may be described in a multiple dependent form, with the subsequent clause alternatively referring to the preceding clause. Furthermore, some clauses may be described in a multiple dependent form, with the subsequent clause referring to another multiple dependent clause. These multiple dependent clauses define multiple technical ideas.

[0110] (Technical Idea 1) A presentation control device that controls the presentation of information to a passenger in a host vehicle (Am), comprising: an information acquisition unit (82) that acquires detection information of emergency vehicles approaching the host vehicle and further determines whether there are multiple emergency vehicles approaching the host vehicle; and a notification control unit (83) that changes a method of approach notification that notifies the passenger of the approach of the emergency vehicles when there are multiple emergency vehicles approaching the host vehicle and when there is only one emergency vehicle. (Technical Idea 2) The presentation control device according to Technical Idea 1, wherein the notification control unit changes the approach notification to a more emphasized manner as the number of emergency vehicles approaching the host vehicle increases. (Technical Idea 3) The presentation control device according to Technical Idea 1 or 2, wherein the information acquisition unit further determines the type of siren sounded by the emergency vehicle, and the notification control unit presents information related to the type of siren to the passenger. (Technical Idea 4) The presentation control device according to any one of Technical Ideas 1 to 3, wherein, when the number of emergency vehicles approaching the host vehicle has not been determined, the information acquisition unit grasps estimated information that estimates the presence of a plurality of emergency vehicles, and the notification control unit issues at least one of a notification inquiring the occupant as to whether a plurality of emergency vehicles are present and a notification calling attention to the possibility that a plurality of emergency vehicles are approaching based on the estimated information. (Technical Idea 5) The presentation control device according to any one of Technical Ideas 1 to 4, wherein, when a plurality of emergency vehicles are approaching and an autonomous driving function of the host vehicle takes evasive action to avoid the emergency vehicles, the notification control unit notifies the occupant of a time when the evasive action will end. (Technical Idea 6) The presentation control device according to any one of Technical Ideas 1 to 5, wherein, when the emergency vehicle has moved away from the host vehicle and the information acquisition unit acquires the detection information of another emergency vehicle approaching the host vehicle, the information acquisition unit determines that a plurality of emergency vehicles are approaching the host vehicle. (Technical Idea 7) A presentation control device according to any one of Technical Ideas 1 to 6, wherein the notification control unit changes the amount of information shown to the passenger in the approach notification if it is determined that the next emergency vehicle is approaching within a predetermined time after the emergency vehicle has left the vehicle.(Technical Idea 8) A presentation control program that controls the presentation of information to passengers in a host vehicle (Am), the presentation control program causing at least one processing unit (11) to execute processing including: acquiring detection information of emergency vehicles approaching the host vehicle (S52); determining whether there are multiple emergency vehicles approaching the host vehicle (S53); and changing a method of approach notification that notifies the passengers of the approach of the emergency vehicle depending on whether there is multiple emergency vehicles approaching the host vehicle or if there is only one emergency vehicle (S54, S55). (Technical Idea 9) An automatic driving control device capable of driving a host vehicle (Am) by autonomous driving control, comprising: an information grasping unit (62) that acquires detection information of an emergency vehicle approaching the host vehicle and further grasps whether there are multiple emergency vehicles approaching the host vehicle, and an action control unit (63) that changes the content of the avoidance action of the host vehicle to avoid the emergency vehicle when there are multiple emergency vehicles approaching the host vehicle and when there is only one emergency vehicle. (Technical Idea 10) The automatic driving control device according to Technical Idea 9, wherein the action control unit performs reroute control to reset the planned driving route of the host vehicle so as to avoid the estimated driving routes of the emergency vehicles when multiple emergency vehicles are approaching the host vehicle. (Technical Idea 11) The automatic driving control device according to Technical Idea 9 or 10, wherein the action control unit, when it is determined that the next emergency vehicle is approaching within a predetermined time after the emergency vehicle has left the host vehicle, implements the avoidance action to avoid a second emergency vehicle, the avoidance action being different from the avoidance action to avoid a first emergency vehicle. (Technical Idea 12) The automatic driving control device according to Technical Idea 11, wherein the action control unit implements, as the avoidance action to avoid the second emergency vehicle, reverse control to reverse the host vehicle, or reroute control to reset a planned driving route of the host vehicle so as to avoid an estimated driving route of the emergency vehicle.(Technical Idea 13) The autonomous driving control device according to any one of Technical Ideas 9 to 12, further comprising an interior recognition unit (61) that recognizes the type of passenger aboard the host vehicle, and wherein the behavior control unit performs control to move toward the emergency vehicles when a plurality of the emergency vehicles are approaching the host vehicle and a specific type of passenger is aboard the host vehicle. (Technical Idea 14) An autonomous driving control program capable of driving a host vehicle (Am) by autonomous driving control, the autonomous driving control program causing at least one processing unit (51) to execute processing including: acquiring detection information of emergency vehicles approaching the host vehicle (S12), further determining whether there are multiple emergency vehicles approaching the host vehicle (S14), and changing the content of the evasive action of the host vehicle to avoid the emergency vehicle depending on whether there is a plurality of emergency vehicles approaching the host vehicle or a single emergency vehicle (S34, S35).

Claims

1. A presentation control device that controls presentation of information to a passenger in a vehicle (Am), an information acquisition unit (82) that acquires detection information of an emergency vehicle approaching the host vehicle and further determines whether there are multiple emergency vehicles approaching the host vehicle; a notification control unit (83) that changes a method of notifying the passenger of the approach of the emergency vehicle depending on whether there are multiple emergency vehicles approaching the vehicle or whether there is one emergency vehicle; The notification control unit changes the approach notification to a more emphasized mode as the number of emergency vehicles approaching the host vehicle increases.

2. the information acquisition unit, when the number of the emergency vehicles approaching the host vehicle is not determined, grasps estimated information that estimates the presence of a plurality of the emergency vehicles; The presentation control device described in claim 1, wherein the notification control unit issues at least one of an alert to the passenger asking whether multiple emergency vehicles are present based on the estimated information, and an alert to warn the passenger that multiple emergency vehicles may be approaching.

3. A presentation control device that controls presentation of information to a passenger in a vehicle (Am), an information acquisition unit (82) that acquires detection information of an emergency vehicle approaching the host vehicle and further determines whether there are multiple emergency vehicles approaching the host vehicle; a notification control unit (83) that changes a method of notifying the passenger of the approach of the emergency vehicle depending on whether there are multiple emergency vehicles approaching the vehicle or whether there is one emergency vehicle; the information acquisition unit, when the number of the emergency vehicles approaching the host vehicle is not determined, grasps estimated information that estimates the presence of a plurality of the emergency vehicles; The notification control unit is a presentation control device that, based on the estimated information, issues at least one of an alert to the passenger asking whether multiple emergency vehicles are present, and an alert to warn the passenger that multiple emergency vehicles may be approaching.

4. The presentation control device according to claim 1 or 3, wherein the notification control unit notifies the passenger of the time when the autonomous driving function of the vehicle takes evasive action to avoid the emergency vehicles due to the approach of multiple emergency vehicles.

5. A presentation control device that controls presentation of information to a passenger in a vehicle (Am), an information acquisition unit (82) that acquires detection information of an emergency vehicle approaching the host vehicle and further determines whether there are multiple emergency vehicles approaching the host vehicle; a notification control unit (83) that changes a method of notifying the passenger of the approach of the emergency vehicle depending on whether there are multiple emergency vehicles approaching the vehicle or whether there is one emergency vehicle; The notification control unit When the autonomous driving function of the vehicle performs an avoidance action to avoid the emergency vehicles due to the approach of a plurality of the emergency vehicles, the autonomous driving function notifies the passenger of the time when the avoidance action will end; A presentation control device that does not notify the user of the time when the avoidance action will end if there is only one emergency vehicle approaching the user's vehicle.

6. The presentation control device described in any one of claims 1, 3, and 5, wherein the information acquisition unit determines that there are multiple emergency vehicles approaching the vehicle when the information acquisition unit acquires detection information of another emergency vehicle approaching the vehicle after the emergency vehicle has left the vehicle.

7. A presentation control device that controls presentation of information to a passenger in a vehicle (Am), an information acquisition unit (82) that acquires detection information of an emergency vehicle approaching the host vehicle and further determines whether there are multiple emergency vehicles approaching the host vehicle; a notification control unit (83) that changes a method of notifying the passenger of the approach of the emergency vehicle depending on whether there are multiple emergency vehicles approaching the vehicle or whether there is one emergency vehicle; The presentation control device determines that there are multiple emergency vehicles approaching the vehicle when the information acquisition unit acquires detection information of another emergency vehicle approaching the vehicle after the emergency vehicle has left the vehicle.

8. The presentation control device according to any one of claims 1, 3, 5, and 7, wherein the notification control unit changes the amount of information shown to the passenger in the approach notification when it detects that the next emergency vehicle is approaching within a predetermined time after the emergency vehicle has left the vehicle.

9. A presentation control device that controls presentation of information to a passenger in a vehicle (Am), an information acquisition unit (82) that acquires detection information of an emergency vehicle approaching the host vehicle and further determines whether there are multiple emergency vehicles approaching the host vehicle; a notification control unit (83) that changes a method of notifying the passenger of the approach of the emergency vehicle depending on whether there are multiple emergency vehicles approaching the vehicle or whether there is one emergency vehicle; The notification control unit is a presentation control device that changes the amount of information shown to the passenger in the approach notification when the approach of the next emergency vehicle is detected within a predetermined time after the emergency vehicle has left the vehicle.

10. The information acquisition unit further grasps the type of siren sounded by the emergency vehicle, The presentation control device according to claim 1 , wherein the notification control unit presents information related to the type of the siren to the passenger.

11. A presentation control program that controls presentation of information to a passenger in a vehicle (Am), Obtaining detection information of an emergency vehicle approaching the vehicle (S52); It is determined whether there are a plurality of emergency vehicles approaching the subject vehicle (S53). A method of notifying the passenger of the approach of the emergency vehicle is changed depending on whether there are a plurality of emergency vehicles approaching the vehicle or there is only one emergency vehicle (S54, S55). The process to be executed by at least one processing unit (11) includes the steps of: The presentation control program includes a step of changing the approach notification method, wherein the approach notification method is changed to a more emphasized mode as the number of emergency vehicles approaching the host vehicle increases.

12. A presentation control program that controls presentation of information to a passenger in a vehicle (Am), Obtaining detection information of an emergency vehicle approaching the vehicle (S52); It is determined whether there are a plurality of emergency vehicles approaching the subject vehicle (S53). A method of notifying the passenger of the approach of the emergency vehicle is changed depending on whether there are a plurality of emergency vehicles approaching the vehicle or there is only one emergency vehicle (S54, S55). The process to be executed by at least one processing unit (11) includes the steps of: In the step of determining whether there are a plurality of emergency vehicles, when the number of the emergency vehicles approaching the host vehicle is not determined, estimation information for estimating the presence of a plurality of the emergency vehicles is determined; In the step of changing the method of approach notification, a presentation control program is provided that, based on the estimated information, issues at least one of an alert to inquire about the presence of multiple emergency vehicles and an alert to warn the passengers that multiple emergency vehicles may be approaching.

13. A presentation control program that controls presentation of information to a passenger in a vehicle (Am), Obtaining detection information of an emergency vehicle approaching the vehicle (S52); It is determined whether there are a plurality of emergency vehicles approaching the subject vehicle (S53). A method of notifying the passenger of the approach of the emergency vehicle is changed depending on whether there are a plurality of emergency vehicles approaching the vehicle or there is only one emergency vehicle (S54, S55). The process to be executed by at least one processing unit (11) includes the steps of: In the step of changing the approach notification method, When the autonomous driving function of the vehicle performs an avoidance action to avoid the emergency vehicles due to the approach of a plurality of the emergency vehicles, the autonomous driving function notifies the passenger of the time when the avoidance action will end; A presentation control program that does not notify the time when the avoidance action will end if there is only one emergency vehicle approaching the host vehicle.

14. A presentation control program that controls presentation of information to a passenger in a vehicle (Am), Obtaining detection information of an emergency vehicle approaching the vehicle (S52); It is determined whether there are a plurality of emergency vehicles approaching the subject vehicle (S53). A method of notifying the passenger of the approach of the emergency vehicle is changed depending on whether there are a plurality of emergency vehicles approaching the vehicle or there is only one emergency vehicle (S54, S55). The process to be executed by at least one processing unit (11) includes the steps of: In the step of determining whether there are multiple emergency vehicles, if the detection information of another emergency vehicle approaching the vehicle is obtained after the emergency vehicle has left the vehicle, the presentation control program determines that there are multiple emergency vehicles approaching the vehicle.

15. A presentation control program that controls presentation of information to a passenger in a vehicle (Am), Obtaining detection information of an emergency vehicle approaching the vehicle (S52); It is determined whether there are a plurality of emergency vehicles approaching the subject vehicle (S53). A method of notifying the passenger of the approach of the emergency vehicle is changed depending on whether there are a plurality of emergency vehicles approaching the vehicle or there is only one emergency vehicle (S54, S55). The process to be executed by at least one processing unit (11) includes the steps of: In the step of changing the method of approach notification, a presentation control program changes the amount of information shown to the passenger in the approach notification if the approach of the next emergency vehicle is detected within a predetermined time after the emergency vehicle has left the vehicle.

16. An automatic driving control device capable of driving a vehicle (Am) by autonomous driving control, an information grasping unit (62) that acquires detection information of an emergency vehicle approaching the host vehicle and further grasps whether or not there are multiple emergency vehicles approaching the host vehicle; an action control unit (63) that changes the content of the avoidance action of the host vehicle to avoid the emergency vehicle depending on whether there are multiple emergency vehicles approaching the host vehicle or whether there is only one emergency vehicle; The behavior control unit is an automatic driving control device that, when the approach of the next emergency vehicle is detected within a predetermined time after the emergency vehicle has left the vehicle, performs an avoidance action to avoid the second emergency vehicle that is different from the avoidance action to avoid the first emergency vehicle.

17. The automatic driving control device according to claim 16, wherein the behavior control unit performs, as the avoidance action to avoid the second emergency vehicle, reverse control to cause the vehicle to reverse, or reroute control to reset the planned driving route of the vehicle so as to avoid the estimated driving route of the emergency vehicle.

18. The vehicle further includes an in-vehicle recognition unit (61) that recognizes the type of passengers riding in the vehicle, The automatic driving control device according to claim 16, wherein the behavior control unit performs control to approach the emergency vehicle when multiple emergency vehicles are approaching the vehicle and a specific type of passenger is on board the vehicle.

19. An automatic driving control device capable of driving a vehicle (Am) by autonomous driving control, an interior recognition unit (61) that recognizes the type of passengers riding in the vehicle; an information grasping unit (62) that acquires detection information of an emergency vehicle approaching the host vehicle and further grasps whether or not there are multiple emergency vehicles approaching the host vehicle; an action control unit (63) that changes the content of the avoidance action of the host vehicle to avoid the emergency vehicle depending on whether there are multiple emergency vehicles approaching the host vehicle or whether there is only one emergency vehicle; The behavior control unit is an automatic driving control device that performs control to approach the emergency vehicles when multiple emergency vehicles are approaching the vehicle and a specific type of passenger is on board the vehicle.

20. An automatic driving control program capable of driving a vehicle (Am) by autonomous driving control, Obtaining detection information of an emergency vehicle approaching the vehicle (S12); It is further determined whether there are a plurality of emergency vehicles approaching the subject vehicle (S14). The content of the avoidance action of the host vehicle to avoid the emergency vehicle is changed depending on whether there are a plurality of emergency vehicles approaching the host vehicle or there is only one emergency vehicle (S34, S35). The process to be executed by at least one processing unit (51) includes the steps of: In the step of changing the content of the evasive action, if the approach of the next emergency vehicle is detected within a predetermined time after the emergency vehicle has left the vehicle, an automatic driving control program is implemented to avoid the second emergency vehicle by taking evasive action with content different from the evasive action taken to avoid the first emergency vehicle.

21. An automatic driving control program capable of driving a vehicle (Am) by autonomous driving control, Obtaining detection information of an emergency vehicle approaching the vehicle (S12); It is further determined whether there are a plurality of emergency vehicles approaching the subject vehicle (S14). The content of the avoidance action of the host vehicle for avoiding the emergency vehicle is changed depending on whether there are a plurality of emergency vehicles approaching the host vehicle or whether there is only one emergency vehicle (S34, S35). The type of passenger riding in the vehicle is identified (S43). The process to be executed by at least one processing unit (51) includes the steps of: In the step of changing the content of the evasive action, an automatic driving control program performs control to approach the emergency vehicle when multiple emergency vehicles are approaching the vehicle and a specific type of passenger is on board the vehicle.