Message notification device, message notification method, and computer program for message notification

Abstract

To provide a message notification device that can properly notify an operator of a message even in a vehicle that is controlled to travel with an automatic operation mode.SOLUTION: When such a degree of change in peripheral environment of a vehicle that an operator pays attention to forward is detected, such a degree of change in vehicle behavior that the operator can feel is detected, or operator's operation to an on-vehicle device installed on a front side of the vehicle is detected while the vehicle is controlled to travel with an automatic operation mode that eliminates the need of drive operation by the operator of the vehicle and a visual line direction of the operator is not oriented in a direction of the drive operation, a message notification device displays a message to announce to the operator on a display part disposed on the front side of a driver seat on which the operator is seated.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a message notification device that notifies a driver of a vehicle of a message. [Background technology]

[0002] A message notification device mounted on a vehicle notifies the driver of various messages. Patent Document 1 describes a head-up display device that determines the display content or display position for each driver based on the driver's attribute information, and projects image light onto a projection member located in front of the driver to display a virtual image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2019-059245 Summary of the Invention [Problem to be solved by the invention]

[0004] Drivers of vehicles operating in automated driving modes that do not require driver operation, particularly automated driving level 3 where the driver is not required to monitor their surroundings, do not necessarily look ahead of the vehicle. Therefore, even if a message is displayed on a display located in front of the driver, the driver may not be able to properly recognize the message.

[0005] The present disclosure aims to provide a message notification device that can appropriately notify a driver of a message even in a vehicle whose driving is controlled in an autonomous driving mode. [Means for solving the problem]

[0006] The message notification device of the present disclosure includes a gaze detection unit that detects the gaze direction of the driver of the vehicle, and a notification unit that displays a message to be made known to the driver on a display unit located in front of the driver's seat when a change in the vehicle's surrounding environment that causes the driver to pay attention to the road ahead is detected, a change in the vehicle's behavior that the driver can feel is detected, or an operation by the driver of on-board equipment mounted in front of the driver's seat where the driver is seated is detected, when the vehicle's driving is controlled in an autonomous driving mode that does not require driving operation by the driver and the driver's gaze direction is not directed in a direction related to driving operation.

[0007] It is preferable that the message notification device of the present disclosure further includes an identification unit that identifies a message to be notified to the driver in response to a change in the environment inside the vehicle cabin or a change in the environment around the vehicle to the extent that the driver is not paying attention to the road ahead.

[0008] In the message notification device of the present disclosure, it is preferable that the notification unit displays a message on the display unit within a predetermined time after detecting a change in the surrounding environment, a change in the vehicle's behavior, or a driver's operation, and if it is detected that the driver's line of sight is directed in a direction related to the driving operation.

[0009] In the message notification device of the present disclosure, it is preferable that the change in the surrounding environment includes at least one of a change in illuminance around the vehicle, a change in air pressure around the vehicle, and a change in the distance from the vehicle to a preceding vehicle traveling in front of the vehicle.

[0010] In the message notification device of the present disclosure, the change in the vehicle behavior preferably includes at least one of starting or stopping the operation of the wipers, turning on or off the headlights, and stopping or slowing down the vehicle.

[0011] In the message notification device of the present disclosure, the operation of the in-vehicle device preferably includes the operation of at least one of a car navigation device, an audio device, and an air conditioning device.

[0012] The message notification method of the present disclosure includes displaying a message to be communicated to the driver on a display unit located in front of the driver's seat when a change in the vehicle's surrounding environment that causes the driver to pay attention to the road ahead is detected, a change in the vehicle's behavior that the driver can feel is detected, or an operation by the driver of on-board equipment mounted in front of the driver's seat where the driver is seated is detected, while the vehicle's driving is controlled in an autonomous driving mode that does not require driving operation by the driver, and the driver's line of sight is not directed in a direction related to driving operation.

[0013] It is preferable that the message notification method of the present disclosure further includes identifying a message to be communicated to the driver in response to a change in the environment inside the vehicle's cabin or a change in the environment surrounding the vehicle to the extent that the driver is not paying attention to the road ahead.

[0014] In the message notification method of the present disclosure, it is preferable that a message is displayed on the display unit within a predetermined time after detecting a change in the surrounding environment, a change in vehicle behavior, or a driver operation, and when it is detected that the driver's line of sight is directed in a direction related to the driving operation.

[0015] In the message notification method of the present disclosure, it is preferable that the change in the surrounding environment includes at least one of a change in illuminance around the vehicle, a change in air pressure around the vehicle, and a change in the distance from the vehicle to a preceding vehicle traveling in front of the vehicle.

[0016] In the message notification method of the present disclosure, the change in vehicle behavior preferably includes at least one of starting or stopping the operation of the wipers, turning on or off the headlights, and stopping or slowing down the vehicle.

[0017] In the message notification method of the present disclosure, the operation of the in-vehicle equipment preferably includes the operation of at least one of a car navigation device, an audio device, and an air conditioning device.

[0018] The computer program for message notification disclosed herein causes a computer mounted on a vehicle to display a message that should be made known to the driver on a display unit located in front of the driver's seat when the vehicle's driving is controlled in an autonomous driving mode that does not require driving operation by the driver of the vehicle, and the driver's line of sight is not directed in a direction related to driving operation, and when a change in the vehicle's surrounding environment that causes the driver to pay attention to the road ahead is detected, a change in the vehicle's behavior that the driver can feel is detected, or an operation by the driver of on-board equipment mounted in front of the driver's seat where the driver is seated is detected.

[0019] It is preferable that the computer program for message notification of the present disclosure further causes the computer to identify a message to be communicated to the driver in response to a change in the environment inside the vehicle's cabin or a change in the vehicle's surrounding environment to the extent that the driver is not paying attention to the road ahead.

[0020] In the message notification computer program of the present disclosure, it is preferable that, in displaying, a message be displayed on the display unit within a predetermined time after detecting a change in the surrounding environment, a change in the vehicle's behavior, or a driver's operation, and when it is detected that the driver's line of sight is directed in a direction related to the driving operation.

[0021] In the computer program for message notification of the present disclosure, it is preferable that the change in the surrounding environment includes at least one of a change in illuminance around the vehicle, a change in air pressure around the vehicle, and a change in the distance from the vehicle to a preceding vehicle traveling in front of the vehicle.

[0022] In the message notification computer program of the present disclosure, the change in vehicle behavior preferably includes at least one of starting or stopping the operation of the wipers, turning on or off the headlights, and stopping or slowing down the vehicle.

[0023] In the message notification computer program of the present disclosure, the operation of the in-vehicle equipment preferably includes the operation of at least one of a car navigation device, an audio device, and an air conditioning device.

[0024] According to the message notification device of the present disclosure, it is possible to appropriately notify the driver of a message even in a vehicle whose driving is controlled in an autonomous driving mode. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a schematic configuration diagram of a vehicle in which a message notification device is installed. [Figure 2] FIG. 1 is a hardware schematic diagram of an ECU. [Figure 3] FIG. 2 is a functional block diagram of a processor included in the ECU. [Figure 4] 10 is a flowchart of a message notification process. DETAILED DESCRIPTION OF THE INVENTION

[0026] A message notification device that can appropriately notify a driver of a vehicle whose driving is controlled in an autonomous driving mode will be described in detail below with reference to the drawings. The message notification device determines whether the vehicle's driving is controlled in an autonomous driving mode, which does not require the driver to perform driving operations. The message notification device also determines whether the driver's line of sight is directed in a direction related to driving operations. When the vehicle's driving is controlled in the autonomous driving mode and the driver's line of sight is not directed in a direction related to driving operations, the message notification device further determines whether a change in the vehicle's surrounding environment that would cause the driver to pay attention to the road ahead is detected, a change in vehicle behavior that the driver can perceive is detected, or a driver operation on an on-board device mounted at the front of the vehicle is detected. When a change in the surrounding environment, a change in vehicle behavior, or a driver operation is detected, the message notification device displays a message that should be communicated to the driver on a display unit located in front of the driver's seat where the driver is seated.

[0027] FIG. 1 is a schematic diagram of a vehicle in which a message notification device is installed.

[0028] The vehicle 1 includes a peripheral camera 2, a driver monitor camera 3, a GNSS receiver 4, a storage device 5, an in-vehicle device 6, a meter display 7, and an ECU 8. The ECU 8 is an example of a message notification device. The peripheral camera 2, the driver monitor camera 3, the GNSS receiver 4, the storage device 5, the in-vehicle device 6, the meter display 7, and the ECU 8 are communicatively connected via an in-vehicle network that complies with a standard such as a controller area network.

[0029] The surrounding camera 2 is an example of a surrounding image capturing unit for generating a surrounding image that represents the surrounding environment of the vehicle. The surrounding camera 2 has a two-dimensional detector configured with an array of photoelectric conversion elements, such as a CCD or C-MOS, that are sensitive to visible light, and an imaging optical system that forms an image of the area to be captured on the two-dimensional detector. The surrounding camera 2 is disposed, for example, at the front upper part of the vehicle interior, facing forward, captures images of the surroundings of the vehicle 1 through the front windshield glass at predetermined capture intervals (for example, 1 / 30 to 1 / 10 seconds), and outputs a surrounding image that represents the surroundings.

[0030] The driver monitor camera 3 is an example of a driver imaging unit for generating a driver image showing the state of the head of the vehicle driver. The driver monitor camera 3 has a two-dimensional detector configured with an array of photoelectric conversion elements, such as a CCD or C-MOS, that are sensitive to infrared light, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. The driver monitor camera 3 also has a light source that emits infrared light. The driver monitor camera 3 is mounted, for example, at the front of the vehicle interior, facing the face of the driver seated in the driver's seat. The driver monitor camera 3 irradiates the driver with infrared light at a predetermined photographing interval (for example, 1 / 30 to 1 / 10 seconds) and outputs an image showing the driver's face.

[0031] The GNSS receiver 4 receives GNSS signals from GNSS (Global Navigation Satellite System) satellites at predetermined intervals, and determines the own position of the vehicle 1 based on the received GNSS signals. The GNSS receiver 4 outputs, at predetermined intervals, a positioning signal representing the positioning result of the own position of the vehicle 1 based on the GNSS signals to the ECU 8, etc. via the in-vehicle network.

[0032] The storage device 5 is an example of a storage device, and includes, for example, a hard disk drive or a non-volatile semiconductor memory. The storage device 5 stores map information including routes between points included in a predetermined range.

[0033] The on-board device 6 is mounted in front of the driver's seat in the vehicle 1. The on-board device 6 is an information processing device with a navigation function, including a communication interface, a touch panel display, a memory storing map information, and a processor. The on-board device 6 searches for a route from the vehicle's own position, which is determined based on the positioning signal acquired from the GNSS receiver 4, to a destination specified by operating the touch panel display, and displays the searched route on the touch panel display.

[0034] The in-vehicle device 6 may be any device that is located in front of the driver's seat and that can be operated by the driver, and may be any device that has a function other than the navigation function, such as an audio device or an air conditioning device.

[0035] The meter display 7 is an example of a display unit and includes, for example, a liquid crystal display. The meter display 7 is disposed in front of the driver's seat in the vehicle 1, and displays messages to be notified to the driver in accordance with signals received from the ECU 8 via the in-vehicle network.

[0036] The ECU 8 is an ECU (Electronic Control Unit) having a communication interface, a memory, and a processor. The ECU 8 detects objects around the vehicle 1 from the peripheral image received from the peripheral camera 2, and outputs control signals to driving mechanisms (not shown) of the vehicle 1, such as the engine, brakes, and steering, so as to maintain a predetermined distance from the detected objects, thereby executing automatic driving control of the vehicle 1. The ECU 8 also detects the driver's line of sight from the driver image received from the driver monitor camera 3 via the communication interface. The ECU 8 also detects changes in the surrounding environment, detects changes in the behavior of the vehicle 1, and detects the driver's operation of the in-vehicle device 6. The ECU 8 then displays a message on the meter display 7 that should be notified to the driver.

[0037] 2 is a hardware schematic diagram of the ECU 8. The ECU 8 includes a communication interface 81, a memory 82, and a processor 83.

[0038] The communication interface 81 is an example of a communication unit, and includes a communication interface circuit for connecting the ECU 8 to an in-vehicle network. The communication interface 81 supplies received data to the processor 83. The communication interface 81 also outputs data supplied from the processor 83 to the outside.

[0039] The memory 82 is an example of a storage unit and includes a volatile semiconductor memory and a nonvolatile semiconductor memory. The memory 82 stores various data used in processing by the processor 83, such as a group of parameters for defining a neural network that operates as an object identifier for detecting objects from a peripheral image, a template representing the pupil and the corneal reflection image of a light source for detecting the driver's line of sight by matching with a driver image, a criterion for determining whether a change in the surrounding environment will cause the driver to pay attention to the road ahead, and a criterion for determining whether the driver can sense a change in vehicle behavior. The memory 82 also stores various application programs, such as a message notification program for executing a message notification process.

[0040] The processor 83 is an example of a control unit and includes one or more processors and their peripheral circuits. The processor 83 may further include other arithmetic circuits such as a logic unit, a numerical calculation unit, or a graphics processing unit.

[0041] FIG. 3 is a functional block diagram of the processor 83 included in the ECU 8. As shown in FIG.

[0042] The processor 83 of the ECU 8 has, as functional blocks, a gaze detection unit 831, a periphery detection unit 832, a driving control unit 833, a behavior detection unit 834, an operation detection unit 835, an identification unit 836, and a notification unit 837. Each of these units in the processor 83 is a functional module implemented by a program executed on the processor 83. A computer program that realizes the functions of each unit in the processor 83 may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. Alternatively, each of these units in the processor 83 may be implemented in the ECU 8 as an independent integrated circuit, a microprocessor, or firmware.

[0043] The gaze detection unit 831 detects the gaze direction of the driver from the driver image received from the driver monitor camera 3 via the communication interface 81. The gaze detection unit 831 detects the pupil and the corneal reflection image of the light source by, for example, performing template matching between the driver image and a template representing the pupil and the corneal reflection image of the light source, and detects the gaze direction based on the positional relationship between them. The gaze direction is represented by the horizontal and vertical angles between the traveling direction of the vehicle 1 and the driver's gaze.

[0044] The surroundings detection unit 832 detects changes in the surrounding environment of the vehicle 1.

[0045] The periphery detection unit 832 detects the distance from the vehicle 1 to a preceding vehicle traveling ahead of the vehicle as the surrounding environment of the vehicle 1. In this case, the periphery detection unit 832 inputs the peripheral image to a first classifier that has been trained in advance to detect objects, thereby identifying the position of other vehicles included in the peripheral image. The first classifier can be, for example, a convolutional neural network (CNN) having multiple convolution layers connected in series from the input side to the output side. By inputting peripheral images including other vehicles as training data into the CNN and performing training in advance, the CNN operates as a first classifier that identifies the position of other vehicles.

[0046] The periphery detection unit 832 detects the inter-vehicle distance from the vehicle 1 to the other vehicle by comparing the width of the other vehicle in the periphery image with the width of a standard vehicle stored in advance in the memory 82.

[0047] Furthermore, the surroundings detection unit 832 detects the illuminance around the vehicle 1 as the surrounding environment of the vehicle 1. In this case, the surroundings detection unit 832 inputs the surrounding image to a second classifier that has been trained in advance to perform semantic segmentation, thereby classifying multiple pixels included in the surrounding image into one of multiple classes representing objects corresponding to the pixels. The second classifier can be a CNN that has multiple convolutional layers connected in series from multiple input sides to the output side but does not have a fully connected layer. By inputting surrounding images in which each pixel is assigned a class such as vehicle, road, or sky into the CNN as training data and training it in advance, the CNN operates as a second classifier that performs semantic segmentation.

[0048] The periphery detection unit 832 estimates the illuminance in the periphery of the vehicle 1 using the brightness of the pixels classified as roads and the aperture value and exposure time used to capture the periphery image.

[0049] The periphery detection unit 832 may perform class classification for each pixel of the peripheral image and may use, as the first classifier and the second classifier, classifiers that perform identification of individual objects corresponding to the class into which the pixel is classified.

[0050] Furthermore, the surroundings detection unit 832 detects the atmospheric pressure around the vehicle 1 as the surrounding environment of the vehicle 1 based on atmospheric pressure information acquired from an atmospheric pressure sensor (not shown) mounted on the vehicle 1 and connected to the in-vehicle network.

[0051] The driving control unit 833 controls the driving of the vehicle 1 in an autonomous driving mode that does not require driving operation by the driver. The driving control unit 833 creates a driving route that maintains a predetermined distance or more to an object detected from the peripheral image by the peripheral detection unit 832, and outputs a control signal to a driving mechanism (not shown) of the vehicle 1 so that the vehicle 1 drives along the driving route. The driving mechanism includes, for example, an engine or motor that supplies power to the vehicle 1, a brake that reduces the driving speed of the vehicle 1, and a steering mechanism that steers the vehicle 1.

[0052] The behavior detection unit 834 detects the behavior of the vehicle 1.

[0053] The behavior detection unit 834 detects, as the behavior of the vehicle 1, behaviors such as the start or stop of operation of windshield wipers (not shown) mounted on the vehicle 1 and connected to the in-vehicle network. To detect the behavior of the wipers, the behavior detection unit 834 acquires a wiper behavior signal that instructs the wipers to behave. The wiper behavior signal is transmitted to the wipers in response to the driver's operation of the wiper switch or a raindrop signal output by a raindrop sensor (not shown) mounted on the vehicle 1.

[0054] Furthermore, the behavior detection unit 834 detects behaviors of the vehicle 1, such as turning on or off headlights (not shown) mounted on the vehicle 1 and connected to the in-vehicle network. To detect the behavior of the headlights, the behavior detection unit 834 acquires a headlight behavior signal that instructs the headlights on how to behave. The headlight behavior signal is transmitted to the headlights in response to the driver's operation of a headlight switch or an illuminance signal output by an illuminance sensor (not shown) mounted on the vehicle 1.

[0055] Furthermore, the behavior detection unit 834 detects behaviors such as stopping or deceleration of the vehicle 1 as behaviors of the vehicle 1. To detect stopping or deceleration of the vehicle 1, the behavior detection unit 834 acquires a vehicle speed signal indicating the vehicle speed of the vehicle 1 from a vehicle speed sensor (not shown) mounted on the vehicle 1 and connected to the in-vehicle network. The vehicle speed of the vehicle 1 is changed in response to the driver's operation of the brake pedal or a control signal output by the traveling control unit 833 to the traveling mechanism.

[0056] The operation detection unit 835 detects the driver's operation of the in-vehicle device 6. The operation detection unit 835 detects the driver's operation of the in-vehicle device 6 by receiving, via the in-vehicle network, an in-vehicle device operation signal transmitted by the in-vehicle device 6 in response to the driver's operation, such as touching the touch panel display of the in-vehicle device 6. The operation detection unit 835 may also detect the driver's operation of the in-vehicle device 6 by detecting an overlap between the in-vehicle device 6 and the driver's hand from an in-vehicle image output by an in-vehicle camera (not shown) positioned so as to capture the in-vehicle device 6. In this case, the in-vehicle device 6 does not need to be connected to the in-vehicle network.

[0057] The specifying unit 836 specifies a message to be communicated to the driver in response to a change in the environment inside the vehicle 1 or a change in the environment around the vehicle 1 to the extent that the driver is not paying attention to the road ahead.

[0058] The identification unit 836 receives a temperature and humidity signal indicating the temperature and humidity inside the vehicle cabin from an interior temperature and humidity sensor (not shown) that is provided inside the vehicle cabin and connected to the in-vehicle network. If a change in the environment inside the vehicle cabin corresponds to an interior environment change pattern stored in the memory 82, the identification unit 836 identifies a message (for example, "Please wipe off any fog from the inside of the windshield") associated with the interior environment change pattern.

[0059] The identification unit 836 also acquires traffic information indicating the traffic conditions ahead of the current location on the travel route from a beacon installed on the road or a traffic information receiver (not shown) that acquires traffic information via FM multiplex broadcasting. If a change in the traffic conditions ahead corresponds to a traffic condition change pattern stored in the memory 82, the identification unit 836 identifies a message (e.g., "There is a traffic jam ahead") associated with the traffic condition change pattern. The change in the traffic conditions ahead corresponds to a change in the surrounding environment that the driver cannot directly perceive and therefore does not direct the driver's attention ahead.

[0060] Furthermore, when the current position of vehicle 1 indicated in the positioning signal acquired from GNSS receiver 4 is approaching within a predetermined distance from the end point of autonomous driving, the identification unit 836 identifies a message associated with approaching the end point of autonomous driving (for example, "Autonomous driving will end in a short while"). Approaching the end point of autonomous driving corresponds to a change in the surrounding environment that does not cause the driver to pay attention to what is ahead, as the driver cannot directly perceive the end point of autonomous driving.

[0061] The notification unit 837 displays a message to be notified to the driver identified by the identification unit 836 on the meter display 7 when all of the following conditions are met:

[0062] (Condition 1) Vehicle 1 is being driven in autonomous mode. The notification unit 837 inquires of the driving control unit 833 to determine whether the driving of the vehicle 1 is being controlled in the autonomous driving mode.

[0063] (Condition 2) The driver's gaze must not be directed in a direction related to driving operations (eyes off). The direction related to driving operation is the direction in which the driver is expected to look when performing driving operations. The direction related to driving operation is, for example, a direction within a range of ±15 degrees up and down and ±45 degrees left and right with respect to the traveling direction of the vehicle 1, and corresponds to the direction forward from the driver sitting in the driver's seat. The notification unit 837 acquires the driver's line of sight from the line of sight detection unit 831 and determines whether the driver is in an eyes-off state.

[0064] (Condition 3) At least one of the following conditions 3-1 to 3-3 must be satisfied. (Condition 3-1) Changes in the surrounding environment that cause the driver to pay attention to the road ahead must be detected. The memory 82 stores a change threshold for each type of surrounding environment to determine whether the driver needs to pay attention to the road ahead. The notification unit 837 acquires, in time series, values ​​of the surrounding environment, such as illuminance, atmospheric pressure, and distance to the preceding vehicle, from the surroundings detection unit 832. If the difference between a past value and a current value is greater than the change threshold, the notification unit 837 detects a change in the surrounding environment that requires the driver to pay attention to the road ahead. The past value may be a value at the most recent past point in time in the time series, or may be a representative value (e.g., an arithmetic mean) of values ​​over a predetermined period of time including the most recent past point in time.

[0065] (Condition 3-2) A change in the behavior of Vehicle 1 that is noticeable to the driver must be detected. The memory 82 stores patterns of changes in the behavior of the vehicle 1 that can be felt by the driver. The notification unit 837 acquires a change in the behavior of the vehicle 1 from the behavior detection unit 834. Then, if the acquired change in behavior corresponds to one of the patterns of changes in behavior stored in the memory 82, the notification unit 837 detects a change in the behavior of the vehicle 1 that can be felt by the driver.

[0066] (Condition 3-3) The driver's operation of the in-vehicle device 6 must be detected. The notification unit 837 acquires the driver's operation on the in-vehicle device 6 detected by the operation detection unit 835.

[0067] In this way, the message identified by the identification unit 836 and displayed on the meter display 7 by the notification unit 837 is not directly related to the events detected under the above-mentioned conditions 3-1 to 3-3. The identification unit 836 can identify the message independently of the detection of these events by the notification unit 837.

[0068] The notification unit 837 may display a message on the meter display 7 when the following condition 4 is satisfied in addition to the above-mentioned conditions 1, 2, and 3. (Condition 4) Within a predetermined time after satisfying Condition 3, the driver's line of sight is detected to be directed in a direction related to driving operations.

[0069] 4 is a flowchart of the message notification process. The ECU 8 repeatedly executes the message notification process at predetermined time intervals (for example, every 1 / 10 seconds) while the vehicle 1 is traveling.

[0070] First, the specifying unit 836 of the ECU 8 specifies a message to be notified to the driver (step S1).

[0071] Next, the notification unit 837 of the ECU 8 determines whether the traveling of the vehicle 1 is controlled in the autonomous driving mode (step S2). If it is determined that the traveling of the vehicle 1 is not controlled in the autonomous driving mode (step S2: N), the ECU 8 ends the message notification process.

[0072] If it is determined that the vehicle 1 is being controlled in the autonomous driving mode (step S2: Y), the notification unit 837 determines whether the driver has their eyes off (step S3). If it is determined that the driver has not their eyes off (step S3: N), the ECU 8 ends the message notification process.

[0073] If it is determined that the driver is in an eyes-off state (step S3: Y), the notification unit 837 determines whether or not a change in the surrounding environment that would cause the driver to pay attention to the road ahead is detected (step S4). If a change in the surrounding environment that would cause the driver to pay attention to the road ahead is not detected (step S4: N), the ECU 8 ends the message notification process.

[0074] Furthermore, if it is determined that the driver is in an eyes-off state (step S3: Y), the notification unit 837 determines whether or not a change in the behavior of the vehicle 1 that the driver can perceive is detected (step S5). If a change in the behavior of the vehicle 1 that the driver can perceive is not detected (step S5: N), the ECU 8 ends the message notification process.

[0075] Furthermore, if it is determined that the driver is in an eyes-off state (step S3: Y), the notification unit 837 determines whether or not an operation by the driver to the in-vehicle device 6 is detected (step S6). If an operation by the driver to the in-vehicle device 6 is not detected (step S6: N), the ECU 8 ends the message notification process.

[0076] If a change in the surrounding environment that causes the driver to pay attention to the road ahead is detected (step S4: Y), if a change in the behavior of the vehicle 1 that the driver can feel is detected (step S5: Y), or if an operation by the driver on the in-vehicle equipment 6 is detected (step S6: Y), the notification unit 837 displays a message that should be communicated to the driver on the meter display 7 (step S7), and terminates the message notification process.

[0077] By executing the message notification process in this manner, the ECU 8 can appropriately notify the driver of a message even in a vehicle whose driving is controlled in an automatic driving mode.

[0078] In the message notification process, the message identification by the identification unit 836 (step S1) does not need to be performed first, but may be performed before the message is displayed (step S7).

[0079] Also, if it is determined in the message notification process that the driver is not in an eyes-off state (step S2: N), the ECU 8 may transition to step S6, and the notification unit 837 may display a message on the meter display 7 that should be communicated to the driver.

[0080] Furthermore, the in-vehicle device 6 may have a display, and the notification unit 837 may display the message to be notified to the driver on the display of the in-vehicle device 6 in step S7 of the message notification process. That is, the in-vehicle device that detects an operation by the driver and the display unit that displays the message to be notified to the driver do not need to be separate entities.

[0081] It should be understood that those skilled in the art can make various changes, substitutions, and alterations thereto without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0082] 1 vehicle 8 ECU 831 Gaze detection unit 837 Notification Department

Claims

1. a gaze detection unit that detects the gaze direction of a driver of the vehicle; When the vehicle is being driven in an automatic driving mode that does not require a driving operation by the driver, and the driver's line of sight is not directed in a direction related to the driving operation, When a change in the behavior of the vehicle that can be felt by the driver is detected, or when an operation by the driver of an in-vehicle device mounted in front of the driver's seat where the driver is seated is detected, a notification unit that displays a message to be notified to the driver on a display unit disposed in front of the driver's seat; A message notification device comprising:

2. a specifying unit that specifies a message to be notified to the driver in response to a change in the environment inside the vehicle or a change in the environment around the vehicle that causes the driver to not pay attention to the road ahead; the notification unit displays the message identified by the identification unit on the display unit. The message notification device according to claim 1 .

3. 2. The message notification device according to claim 1, wherein the notification unit displays the message on the display unit when, within a predetermined time after detecting a change in the vehicle's behavior or the driver's operation, it is detected that the driver's line of sight is directed in a direction related to the driving operation.

4. The message notification device according to claim 1 , wherein the change in the behavior of the vehicle includes at least one of starting or stopping the operation of windshield wipers, turning on or off headlights, and stopping or slowing down the vehicle.

5. The message notification device according to claim 1 , wherein the operation of the in-vehicle device includes an operation of at least one of a car navigation device, an audio device, and an air conditioning device.

6. When the vehicle is being controlled in an automated driving mode that does not require a driver to perform a driving operation, and when the driver's line of sight is not directed in a direction related to the driving operation, a change in the behavior of the vehicle that is perceptible to the driver is detected, or when an operation by the driver of an in-vehicle device mounted in front of the driver's seat is detected, a message that should be communicated to the driver is displayed on a display unit located in front of the driver's seat. A message notification method including:

7. The method further includes: identifying a message to be notified to the driver in response to a change in the environment inside the vehicle or a change in the environment around the vehicle to the extent that the driver is not paying attention to the road ahead; The message notification method according to claim 6 , wherein the step of displaying the message comprises displaying the specified message on the display unit.

8. 7. The message notification method according to claim 6, wherein the message is displayed on the display unit when, within a predetermined time after detecting a change in the vehicle's behavior or the driver's operation, it is detected that the driver's line of sight is directed in a direction related to the driving operation.

9. The message notification method according to claim 6 , wherein the change in the behavior of the vehicle includes at least one of starting or stopping wipers, turning on or off headlights, and stopping or slowing down the vehicle.

10. The message notification method according to claim 6 , wherein the operation of the in-vehicle device includes an operation of at least one of a car navigation device, an audio device, and an air conditioning device.

11. When the vehicle is being controlled in an automated driving mode that does not require a driver to perform a driving operation, and when the driver's line of sight is not directed in a direction related to the driving operation, a change in the behavior of the vehicle that is perceptible to the driver is detected, or when an operation by the driver of an in-vehicle device mounted in front of the driver's seat is detected, a message that should be communicated to the driver is displayed on a display unit located in front of the driver's seat. A computer program for message notification that causes a computer installed in the vehicle to execute the above.

12. Identifying a message to be communicated to the driver in response to a change in the environment inside the vehicle or a change in the environment around the vehicle that causes the driver to not pay attention to the road ahead; The message notification computer program according to claim 11 , further causing the computer to execute the following in the displaying of the message: displaying the specified message on the display unit.

13. 12. The computer program for message notification according to claim 11, wherein the message is displayed on the display unit when, within a predetermined time after detecting a change in the behavior of the vehicle or the driver's operation, it is detected that the driver's line of sight is further directed in a direction related to the driving operation.

14. The message notification computer program according to claim 11 , wherein the change in the behavior of the vehicle includes at least one of starting or stopping wipers, turning on or off headlights, and stopping or slowing down the vehicle.

15. The message notification computer program according to claim 11, wherein the operation of the in-vehicle device includes an operation of at least one of a car navigation device, an audio device, and an air conditioning device.

Images