Information display method and information display device

Abstract

To control information display to a driver by a display device such that the immersion level of the driver to a dynamic traffic element in the periphery of the own vehicle becomes the immersion level suitable to the congestion level of the dynamic traffic element.SOLUTION: An information display method detects a dynamic traffic element in the periphery of the won vehicle (S1), measures the brain activity of a driver (S4, S13), determines whether or not the driver visually recognizes the detected dynamic traffic element (S6, S15), calculates the immersion level of the driver to the dynamic traffic element on the basis of the brain activity when the driver visually recognizes the detected dynamic traffic element (S7, S16), and controls a display device such that the immersion level of the driver to the dynamic traffic element gets close to a target range when the calculated immersion level deviates from the target range set according to the congestion level of the dynamic traffic element (S9, S11, S18-S20).SELECTED DRAWING: Figure 5A

Description

Technical Field

[0001] The present invention relates to an information display method and an information display device.

Background Art

[0002] In Patent Document 1 below, a vehicle display device that performs display for assisting driving has been proposed. This vehicle display device acquires information related to traffic regulations and instructions corresponding to the road in front of the host vehicle, determines the degree of attention that the user should pay to the road in front of the host vehicle, and displays a display image showing areas of the road with different display modes according to the degree of attention on the windshield.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Elements constituting the traffic environment around the host vehicle include not only static elements such as traffic regulations and instructions but also dynamic traffic elements that are movable elements. It is preferable for the driver to maintain an appropriate degree of immersion according to the degree of congestion of the dynamic traffic elements with respect to the dynamic traffic elements. However, the degree of immersion of the driver with respect to the dynamic traffic elements outside the host vehicle may be affected by the display device provided inside the host vehicle. An object of the present invention is to control the information display to the driver by the display device so that the degree of immersion of the driver with respect to the dynamic traffic elements becomes a degree of immersion suitable for the degree of congestion of the dynamic traffic elements.

Means for Solving the Problems

[0005] In the information display method of the embodiment, dynamic traffic elements, which are movable elements among the components of the traffic environment around the vehicle, are detected, the brain activity of the vehicle's driver is measured, it is determined whether or not the driver is visually viewing the detected dynamic traffic elements, the degree of the driver's immersion in the dynamic traffic elements is calculated based on the brain activity measured when the driver is visually viewing the detected dynamic traffic elements, a target range for the driver's immersion in the dynamic traffic elements is set according to the degree of congestion of the dynamic traffic elements, and if the calculated degree of immersion falls outside the set target range, the display on the display device visible to the driver is controlled so that the driver's degree of immersion in the dynamic traffic elements approaches the target range. [Effects of the Invention]

[0006] According to the present invention, the display device can control the information displayed to the driver so that the driver's level of immersion in the dynamic traffic elements is appropriate to the level of congestion of the dynamic traffic elements. [Brief explanation of the drawing]

[0007] [Figure 1] This is a schematic diagram of an example of a vehicle equipped with an information display device according to the embodiment. [Figure 2] This is a block diagram of an example of the functional configuration of an information display device according to the first embodiment. [Figure 3A] This is a schematic diagram illustrating an example of a display when the congestion level of dynamic traffic elements is low and the level of immersion is within the target range. [Figure 3B] This is a schematic diagram illustrating an example of display when the congestion level of dynamic traffic elements is low and the level of immersion is below the target range. [Figure 3C] This is a schematic diagram illustrating an example of a display when the congestion level of dynamic traffic elements is low and the immersion level is higher than the target range. [Figure 4A] This is a schematic diagram illustrating an example of a display when the level of congestion of dynamic traffic elements is high and the level of immersion is within the target range. [Figure 4B] This is a schematic diagram illustrating an example of a display when the congestion level of dynamic traffic elements is high and the immersion level is lower than the target range. [Figure 4C]This is a schematic diagram illustrating an example of how to display information when the congestion level of dynamic traffic elements is high and the level of immersion exceeds the target range. [Figure 5A] This is a flowchart of an example of an information display method according to the first embodiment. [Figure 5B] This is a flowchart of a modified example of the information display method of the first embodiment. [Figure 6] This is a block diagram of an example of the functional configuration of an information display device according to the second embodiment. [Figure 7] This is a flowchart of an example of an information display method according to the second embodiment. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below with reference to the drawings. Note that the drawings are schematic and may differ from actual ones. Furthermore, the embodiments of the present invention described below are illustrative examples of devices and methods for realizing the technical concept of the present invention, and the technical concept of the present invention is not limited to the structure, arrangement, etc., of the components described below. The technical concept of the present invention can be modified in various ways within the technical scope defined by the claims described in the patent claims.

[0009] (First Embodiment) (composition) Figure 1 shows an example of a schematic configuration of a vehicle equipped with an information display device according to the embodiment. The vehicle 1 is equipped with a driving support device 10 that assists in the driving of the vehicle 1, and an information display device 20 that displays various information to the driver of the vehicle 1. The driving mode of the vehicle 1 can be switched between automatic driving mode and manual driving mode. In automatic driving mode, the driver assistance device 10 detects the driving environment around the vehicle 1. Based on the detected driving environment, the driver assistance device 10 controls the acceleration, deceleration, and steering angle of the vehicle 1 to perform autonomous driving control, driving the vehicle 1 without driver intervention.

[0010] The driver can switch the driving mode of their vehicle 1 between automatic driving mode and manual driving mode by operating a mode selector switch (not shown). Furthermore, the driver assistance device 10 may automatically switch the driving mode from automatic driving mode to manual driving mode when it detects an override operation by the driver. An override operation refers to the driver intervening in autonomous driving control by operating the steering wheel, accelerator pedal, or brake pedal in automatic driving mode. Furthermore, the driver assistance device 10 may automatically switch the driving mode from manual driving mode to automatic driving mode in accordance with the mode switching signal from the information display device 20.

[0011] The driver assistance system 10 includes a positioning device 11, a map database (map DB) 12, an object sensor 13, a vehicle sensor 14, a driver assistance controller 15, and an actuator 16. The positioning device 11 measures the current position of the vehicle 1. The positioning device 11 may include, for example, a Global Navigation Satellite System (GNSS) receiver. The GNSS receiver is, for example, a GPS receiver, and measures the current position of the vehicle 1 by receiving radio waves from multiple navigation satellites. The map database 12 stores road map data. For example, the map database 12 may store high-precision map data suitable for use as map information for autonomous driving (hereinafter simply referred to as "high-precision map"). The map database 12 may also store map data for navigation (hereinafter simply referred to as "navigation map").

[0012] The object sensor 13 acquires various information (surrounding environment information) about the driving environment around the host vehicle 1. For example, the object sensor 13 detects objects around the host vehicle 1. The object sensor 13 detects the surrounding environment of the host vehicle 1, such as objects existing around the host vehicle 1, the relative position between the host vehicle 1 and the object, the distance between the host vehicle 1 and the object, and the direction in which the object exists. The object sensor 13 outputs the surrounding environment information, which is the detected surrounding environment information, to the driving support controller 15 and the information display device 20. The object sensor 13 may include a camera, LiDAR (Light Detection and Ranging), radar, millimeter-wave radar, laser rangefinder, sonar, etc.

[0013] The vehicle sensor 14 detects various information (vehicle information) obtained from the host vehicle 1. The vehicle sensor 14 includes, for example, a vehicle speed sensor that detects the traveling speed (vehicle speed) of the host vehicle 1, a wheel speed sensor that detects the rotational speed of each tire of the host vehicle 1, a three-axis acceleration sensor (G sensor) that detects the acceleration (including deceleration) in the three-axis directions of the host vehicle 1, a steering angle sensor that detects the steering angle of the steering wheel, a steering angle sensor that detects the steering angle of the steering wheel, a gyro sensor that detects the angular velocity generated in the host vehicle 1, a yaw rate sensor that detects the yaw rate, an accelerator sensor that detects the operation of the accelerator pedal of the host vehicle 1, and a brake sensor that detects the operation of the brake pedal by the driver. The vehicle sensor 14 outputs the vehicle information to the driving support controller 15.

[0014] The driving support controller 15 is an electronic control unit (ECU: Electronic Control Unit) that performs autonomous driving control of the host vehicle 1. The driving support controller 15 includes a processor and peripheral components such as a storage device. The processor may be, for example, a CPU (Central Processing Unit) or a MPU (Micro-Processing Unit). The memory device may include a semiconductor memory device, a magnetic memory device, an optical memory device, etc. The memory device may include memories such as registers, cache memories, ROM (Read Only Memory) and RAM (Random Access Memory) used as main memory devices. The functions of the driving support controller 15 described below are realized, for example, when a processor executes a computer program stored in the memory device.

[0015] The driving support controller 15 executes autonomous driving control of the host vehicle 1 based on the positioning result of the positioning device 11, the map information of the map database 12, the surrounding environment information from the object sensor 13, and the vehicle information from the vehicle sensor 14. For example, the driving support controller 15 calculates a target driving trajectory for the host vehicle 1 to travel based on the current position and attitude of the host vehicle 1, the target route to the destination set by a navigation system or the like (not shown), the map information, and the surrounding environment of the host vehicle 1. For example, a route space map representing the presence or absence of routes and objects around the host vehicle 1 and a risk map in which the degree of danger of the driving area is quantified are generated, and based on the motion characteristics of the host vehicle 1, the vehicle information, the route space map, and the risk map, a target driving trajectory for the host vehicle 1 to travel is generated. The driving support controller 15 drives the actuator 16 so that the host vehicle 1 travels along the generated target driving trajectory.

[0016] The actuator 16 operates the steering wheel, the accelerator opening, and the braking device of the host vehicle 1 in response to a control signal from the driving support controller 15 to generate the vehicle behavior of the host vehicle 1. The actuator 16 includes a steering actuator, an accelerator opening actuator, and a brake control actuator. The steering actuator controls the steering direction and the amount of steering of the steering of the host vehicle 1. The accelerator opening actuator controls the accelerator opening of the host vehicle 1. The brake control actuator controls the braking operation of the braking device of the host vehicle 1.

[0017] Next, the information display device 20 will be described. The information display device 20 comprises a display device 21, a display controller 22, a brain activity sensor 23, and a occupant sensor 24. The display device 21 is installed in the passenger compartment of the vehicle 1 and in a position visible to the driver, and is capable of displaying visual information such as characters, diagrams, pictures, images, and videos. In the following description, visual information such as characters, diagrams, pictures, images, and videos may be collectively referred to as "images." In the following description, the display device 21 is described as a digital meter located on the instrument panel in front of the driver's seat. However, the present invention is not limited to such a digital meter and is broadly applicable to any display device located inside the vehicle 1 that is visible to the driver. For example, the display device 21 may be a navigation system or a graphical user interface (GUI) device for in-vehicle infotainment (IVI) equipment.

[0018] The display controller 22 is an electronic control unit that controls the visual information displayed on the display device 21. The display controller 22 includes a processor 25 and peripheral components such as a storage device 26. The processor 25 may be, for example, a CPU or an MPU. The storage device 26 may include semiconductor storage devices, magnetic storage devices, optical storage devices, etc. The storage device may include registers, cache memory, and memory such as ROM and RAM used as main memory.

[0019] The display controller 22 may be formed by dedicated hardware for executing each information processing. For example, the display controller 22 may include functional logic circuits set in a general-purpose semiconductor integrated circuit. For example, the display controller 22 may have a programmable logic device (PLD) such as a field-programmable gate array (FPGA).

[0020] Figure 2 is a block diagram of an example of the functional configuration of the information display device 20 of the first embodiment. The functions of the dynamic traffic element detection unit 30, the gaze detection unit 31, the viewing target determination unit 32, the immersion level calculation unit 33, and the display control unit 34 may be realized, for example, by the processor 25 of the display controller 22 shown in Figure 1 executing a computer program stored in the storage device 26, or they may be realized by dedicated hardware as exemplified above.

[0021] The brain activity sensor 23 is a sensor that detects the amount of brain activity of the driver. The brain activity sensor 23 comprises a brain activity measurement unit 23a and a blink detection unit 23b. The brain activity measurement unit 23a measures the state of the driver's brain activity and outputs brain activity information, which includes the measured state of brain activity, to the immersion level calculation unit 33. The state of brain activity is one of the following: the driver's electroencephalogram (EEG), the driver's magnetic field, the driver's electroencephalogram (EEG), or the activity state of the driver's brain surface. In other words, the brain activity measurement unit 23a measures the driver's brain activity using one of the following: the driver's EEG, magnetic field, electroencephalogram, or the activity state of the brain surface.

[0022] Specifically, the brain activity measurement unit 23a can be formed using means for measuring radio waves caused by brain activity, means for measuring magnetic fields caused by brain activity, means for measuring electric potentials placed on the surface of the driver's scalp, electroencephalograph, magnetoencephalograph, electroencephalograph, and optical brain function imaging device (fNIRS) based on functional near-infrared spectroscopy. Furthermore, the brain activity measurement unit 23a is preferably capable of measuring the target with high temporal resolution. In addition, the brain activity measurement unit 23a is preferably non-contact, but may also be contact-type. As a non-contact brain activity measurement unit 23a, for example, the "Neurobiomonitor Headrest" manufactured by Freer Logic can be used. As a non-contact brain activity measurement unit 23a, for example, the "RICOH MEG" magnetoencephalography system can be used.

[0023] The blink detection unit 23b detects the blinking motion of the driver's eyes. The blink detection unit 23b may be implemented as at least one of hardware and software. The blink detection unit 23b can, for example, detect the driver's blinking motion based on the driver's electrooculography (EOG). Furthermore, the blink detection unit 23b may be configured to estimate the potential generated by the driver's blinks, for example, by applying the blink potential removal technology of the electroencephalogram (EEG) analysis system (BESA Research 7.0). In the configuration that applies the blink potential removal technology of the EEG analysis system, the potential generated by the driver's blinks is estimated based on a pre-created model of what kind of scalp potential is generated by blinking. Based on the above, the blink detection unit 23b detects the blinking motion of the driver's eyes and outputs blink information, which includes the detected blinking motion, to the immersion level calculation unit 33. Furthermore, the term "blinking action" includes at least one of the following: "immediately after blinking," "immediately after closing the eyes during blinking," "immediately before blinking," "before and after blinking," "during blinking," "the moment of blinking," "the period during which blinking occurs," and "the period of time elapsed from closing the eyes after blinking until they are fully opened."

[0024] The occupant sensor 24 is, for example, located inside the vehicle and detects the driver's condition. The occupant sensor 24 may be, for example, an imaging device (such as a stereo camera or monocular camera) capable of capturing images of an object. The gaze detection unit 31 recognizes the driver's face orientation, the position of the inner corner of the eye, the iris, and the pupil, etc., based on the detection results from the occupant sensor 24. Based on the face orientation, the position of the inner corner of the eye, the iris, and the pupil, etc., the gaze detection unit 31 detects gaze information including the position coordinates of the driver's viewpoint and the direction of their gaze. The gaze detection unit 31 outputs the detected gaze information to the target of observation determination unit 32.

[0025] The dynamic traffic element detection unit 30 detects dynamic traffic elements present around the vehicle 1. In this specification, "dynamic traffic element" means a movable element among the components of the traffic environment around the vehicle 1. Dynamic traffic elements are elements excluding static traffic elements such as traffic regulations and instructions (road signs, road markings, traffic signals, etc.) corresponding to the roads around the vehicle 1. Examples of dynamic traffic elements include preceding vehicles, following vehicles, oncoming vehicles, pedestrians, and bicycles.

[0026] The dynamic traffic element detection unit 30 detects, for example, the position, posture, size, and speed of objects around the vehicle 1, such as vehicles, motorcycles, pedestrians, bicycles, and obstacles, based on the detection signal from the object sensor 13. The dynamic traffic element detection unit 30 tracks the detected object. Specifically, it verifies (matches) the identity of the object between different time points based on the behavior of the object output at different time points, and predicts the behavior of the object, such as its velocity and attitude (e.g., yaw angle), based on that matching. The dynamic traffic element detection unit 30 may determine moving objects among the objects surrounding the vehicle 1 as dynamic traffic elements and detect their positions. It may also determine the type of dynamic traffic element (for example, a preceding vehicle, a following vehicle, an oncoming vehicle, a pedestrian, a bicycle, etc.) based on the size, shape, position, and speed of the moving object.

[0027] Alternatively, for example, the dynamic traffic element detection unit 30 may perform image recognition processing on the surrounding image obtained when the camera of the object sensor 13 captures the area around the vehicle 1, and detect the position of dynamic traffic elements by recognizing vehicles, pedestrians, and bicycles from the surrounding image. In this case, the dynamic traffic element detection unit 30 may determine the type of detected dynamic traffic element by comparing a predetermined image pattern of vehicles, pedestrians, bicycles, etc. with the surrounding image. The dynamic traffic element detection unit 30 outputs information about the detected dynamic traffic elements to the visibility target determination unit 32 and the display control unit 34.

[0028] The sight target determination unit 32 determines whether the driver's sight target is a dynamic traffic element based on the driver's gaze information output from the gaze detection unit 31 and the dynamic traffic element information output from the dynamic traffic element detection unit 30. For example, the sight target determination unit 32 determines that the driver's sight target is a dynamic traffic element if the driver's gaze detected by the gaze detection unit 31 is directed towards the detection location of a dynamic traffic element detected by the dynamic traffic element detection unit 30, and determines that the driver's sight target is not a dynamic traffic element if the driver's gaze is not directed towards the detection location of a dynamic traffic element. The sight target determination unit 32 outputs the determination result to the immersion level calculation unit 33.

[0029] The immersion level calculation unit 33 calculates the driver's level of immersion in dynamic traffic elements based on brain activity information and blinking information output from the brain activity sensor 23 and the determination result from the visual target determination unit 32. For example, the driver's level of immersion is calculated based on the brain activity information and blinking information obtained when it is determined that the driver's visual target is a dynamic traffic element. The immersion level calculation unit 33 outputs the immersion level calculation information, which includes the calculated immersion level, to the display control unit 34. When calculating the level of immersion, for example, the level of immersion is calculated based on the magnitude of brain activity during blinking. Specifically, if the brain potential immediately after blinking is high, the level of immersion is calculated to be low, and if the brain potential immediately after blinking is low, the level of immersion is calculated to be high.

[0030] The immersion level calculation unit 33 may, for example, calculate the average value of brain activity during multiple blinking movements, and further calculate the immersion level based on the calculated average value. This can improve the accuracy of the immersion level calculation. In this case, the immersion level calculation unit 33 calculates the immersion level using blink evoked potentials by using, for example, a blinking motion detected by the blink detection unit 23b as a trigger, and then averaging the electroencephalograms measured by the brain activity measurement unit 23a. A "blink evoked potential" is a negative potential on the scalp that reflects a reset of the brain.

[0031] Here, we will explain why blink-evoked potentials are used to calculate the degree of immersion. It is known that blinking resets the brain by increasing the activity of networks that are activated during a resting state. Therefore, it is possible to obtain "blink-evoked potentials" by using methods such as averaging brain waves with blinking as a trigger. For example, when a driver is highly engrossed in dynamic traffic elements, the degree to which blinking resets the brain decreases, resulting in a smaller "blink-evoked potential." Conversely, when a driver is less engrossed in dynamic traffic elements, the degree to which blinking resets the brain increases, resulting in a larger "blink-evoked potential."

[0032] Therefore, the immersion level calculation unit 33 calculates a lower immersion level when the amount of brain activity is large, based on the magnitude of brain activity during blinking (for example, the maximum value of the electroencephalogram waveform for a certain period immediately after blinking). On the other hand, it calculates a higher immersion level when the amount of brain activity is small. Furthermore, if the blinking motion detected by the blink detection unit 23b is the "point in time," the immersion level calculation unit 33 may extract the state of brain activity for a certain period of time before and after the "point in time," or immediately after the "point in time," and calculate the immersion level based on the extracted state of brain activity.

[0033] The display control unit 34 controls the display on the display device 21 based on the dynamic traffic element information output from the dynamic traffic element detection unit 30 and the immersion level calculation information output from the immersion level calculation unit 33. Specifically, when there are many dynamic traffic elements around the vehicle (for example, when the degree of congestion of dynamic traffic elements is high), the driver needs to be highly immersed in driving. In other words, the driver needs to be highly immersed in the dynamic traffic elements. Therefore, for example, the display control unit 34 reduces the amount of information displayed on the display device 21 when there are many dynamic traffic elements (when the degree of congestion of dynamic traffic elements is high) compared to when there are few dynamic traffic elements (when the degree of congestion of dynamic traffic elements is low). For example, the amount of information displayed on the display device 21 is reduced as the degree of congestion of dynamic traffic elements increases. This increases the driver's immersion in dynamic traffic elements and improves the driving safety of the vehicle 1.

[0034] The display control unit 34 may calculate the degree of congestion of dynamic traffic elements according to the number of dynamic traffic elements detected by the dynamic traffic element detection unit 30 at approximately the same time (for example, detected within the same control cycle). For example, if the degree of congestion of a dynamic traffic element is less than a predetermined threshold, it may be determined that the congestion level is "low," and if the degree of congestion of a dynamic traffic element is equal to or greater than a predetermined threshold, it may be determined that the congestion level is "high."

[0035] Figures 3A to 3C are schematic diagrams of display examples on the display device 21 when the degree of congestion of dynamic traffic elements is small. When the degree of congestion of dynamic traffic elements is low, the display control unit 34 can improve the driver's user experience by displaying an image on the display device 21 in a "high-quality display mode" that displays a relatively large amount of information. Reference numeral 40 is a vehicle speed image showing the vehicle speed of the vehicle 1, reference numeral 41 is a mileage image showing the distance traveled by the vehicle 1, reference numeral 42 is a 3D map of the area around the vehicle, and reference numeral 43 is an animation representing an artificial intelligence (AI) agent.

[0036] For example, the vehicle speed image 40 and the mileage image 41 may be animated images that mimic an analog drum-type meter. Such video displays make it easier for the driver to understand that the vehicle speed and mileage have changed. Furthermore, by displaying the 3D map 42, the driver can more easily understand the surrounding conditions and facilities of their vehicle 1. Furthermore, displaying animations of the AI ​​agent 43 can improve the driver's acceptance, trust, and affinity for autonomous driving control and various information processing performed by the vehicle 1.

[0037] On the other hand, Figures 4A and 4B are schematic diagrams of display examples on the display device 21 when the degree of congestion of dynamic traffic elements is high. When the degree of congestion of dynamic traffic elements is high, the display control unit 34 displays an image on the display device 21 in a "simplified display mode" in which the amount of information displayed is relatively small.

[0038] Reference numeral 50 is a vehicle speed image showing the vehicle speed of the vehicle 1, reference numeral 51 is a tachometer image, and reference numeral 52 is an animation of icons for the vehicle 1 and other vehicles showing the presence or absence of other vehicles around the vehicle 1. The images in the simplified display mode (Figures 4A-4B) display less information than the images in the high-quality display mode (Figures 3A-3C). For example, the vehicle speed image 50 and the tachometer image 51 are animated images that mimic conventional needle-type meters, and compared to the vehicle speed image 40 and mileage image 41 in Figure 3A, the image changes are less and the amount of information is reduced. Also, the icons 52 for the own vehicle 1 and other vehicles show less image change compared to the 3D map 42. Furthermore, the display of the AI ​​agent as shown in Figure 3A is omitted.

[0039] In this way, the display control unit 34 reduces the amount of information displayed on the display device 21 when the degree of congestion of dynamic traffic elements is high, thereby lowering the driver's level of immersion in the display device 21. As a result, the driver's level of immersion in dynamic traffic elements can be increased, and the driving safety of the vehicle 1 can be improved.

[0040] Furthermore, the display control unit 34 determines whether the current driver's level of immersion in the dynamic traffic elements, calculated by the immersion calculation unit 33, is appropriate for the level of congestion of the dynamic traffic elements. If the current driver's level of immersion in the dynamic traffic elements is not appropriate for the level of congestion, the display control unit 34 controls the display on the display device 21 to guide the driver's level of immersion in the dynamic traffic elements to an appropriate level for the level of congestion.

[0041] For example, the display control unit 34 may set a target range for driver immersion in dynamic traffic elements, which is appropriate for the calculated degree of congestion of the dynamic traffic elements. The target range for immersion may be, for example, a single threshold without a width, or it may be a range where the difference between the upper limit and the lower limit is greater than 0. As described above, when the degree of congestion of dynamic traffic elements is high compared to when the degree of congestion is low, drivers are required to be more immersed in the dynamic traffic elements. Therefore, for example, the display control unit 34 may set a higher target range for immersion when the degree of congestion is high compared to when the degree of congestion is low.

[0042] The display control unit 34 then determines whether the current driver's level of immersion in dynamic traffic elements, calculated by the immersion calculation unit 33, falls within the set target range. (Note that if the target range is a single threshold without a width, the display control unit 34 determines whether the level of immersion calculated by the immersion calculation unit 33 is equal to the target range.) If the immersion level calculated by the immersion level calculation unit 33 is within the target range and the degree of congestion of dynamic traffic elements is low, the normal screen in the high-quality display mode described above is displayed. Figure 3A is an example of the normal screen in the high-quality display mode.

[0043] Furthermore, if the immersion level calculated by the immersion level calculation unit 33 is within the target range and the degree of congestion of dynamic traffic elements is high, the normal screen in the simplified display mode described above will be displayed. Figure 4A is an example of the normal screen in the simplified display mode.

[0044] On the other hand, if the degree of congestion of dynamic traffic elements is low (i.e., the dynamic traffic elements around the vehicle 1 are sparse), and the immersion level calculated by the immersion level calculation unit 33 (the driver's current level of immersion in dynamic traffic elements) is too low, it is considered that the driver's level of alertness is low. For example, it is considered that the driver is dozing off. Therefore, when the degree of congestion of dynamic traffic elements is low and the level of immersion is lower than the target range, the display control unit 34 adds content that enhances the driver's alertness to the normal screen (for example, Figure 3A) in high-quality display mode and displays it on the display device 21.

[0045] Figure 3B is a schematic diagram of an example of display when the congestion level of dynamic traffic elements is low and the level of immersion is below the target range. For example, content that increases the driver's alertness may be content 44a and 44b in which the AI ​​agent 43 presents a quiz to the driver. As content that increases the driver's alertness, the AI ​​agent 43 may also display an animation in which it tells a joke.

[0046] Furthermore, if the degree of congestion of dynamic traffic elements is low, and the driver's current level of immersion in those elements is too high, it can be assumed that the driver is preoccupied with thoughts other than driving. Therefore, if the degree of congestion of dynamic traffic elements is low and the level of immersion is higher than the target range, the display control unit 34 adds an instruction to the normal screen in high-quality display mode that instructs the driver to concentrate on driving their own vehicle (i.e., an instruction to stop thinking about anything other than driving) and displays it on the display device 21.

[0047] Figure 3C is a schematic diagram of an example of display when the congestion level of dynamic traffic elements is low and the level of immersion is higher than the target range. For example, as an instruction to concentrate on driving, an image 45a of the driver's family may be displayed to encourage safe driving. As an instruction to concentrate on driving, the AI ​​agent 43 may display a message 45b to the driver encouraging safe driving.

[0048] On the other hand, when the degree of congestion of dynamic traffic elements is high (i.e., when dynamic traffic elements around vehicle 1 are congested), if the driver's current level of engagement with the dynamic traffic elements is too low, it can be considered that the driver's attention to the dynamic traffic elements is insufficient. For example, it can be considered that the driver's attention is scattered. Therefore, if the degree of congestion of dynamic traffic elements is high and the level of immersion is lower than the target range, the display control unit 34 adds an instruction to the normal screen in simplified display mode (for example, Figure 4A) to draw the driver's attention to the dynamic traffic elements and displays it on the display device 21.

[0049] In this case, in order to suppress an increase in the amount of information displayed on the display device 21, it is preferable that the instructions to attract the driver's attention be concise. Figure 4B is a schematic diagram of an example of a display when the degree of congestion of dynamic traffic elements is high and the degree of immersion is lower than the target range. For example, the display control unit 34 may add only a text message 53 to the normal screen in the simplified display mode to attract the driver's attention.

[0050] Furthermore, if the degree of congestion in dynamic traffic elements is high, and the driver's current level of engagement with those elements is too high, then the driving burden in the current situation of congested dynamic traffic elements is considered to be too high for the driver. Therefore, if the degree of congestion of dynamic traffic elements is high and the level of immersion is lower than the target range, the display control unit 34 generates a mode switching signal to switch the driving mode of the vehicle 1 from manual driving mode to automatic driving mode and outputs it to the driving support device 10. The driving support controller 15 switches the driving mode from manual driving mode to automatic driving mode according to the mode switching signal.

[0051] Figure 4C is a schematic diagram of an example of a display when the degree of congestion of dynamic traffic elements is high and the level of immersion is higher than the target range. The image in Figure 4C is a normal screen displayed on the display device 21 during autonomous driving, and includes characters 60 indicating the vehicle speed of the vehicle 1, and animations 61 of icons for the vehicle 1 and other vehicles indicating the presence or absence of other vehicles around the vehicle 1. The display control unit 34 may display a message 62 to notify the change in operating mode when it outputs a mode switching signal and switches the operating mode to automatic operating mode.

[0052] Instead of outputting a mode switching signal, the display control unit 34 may display a message on the display device 21 prompting the user to switch from manual driving mode to automatic driving mode (i.e., to operate the mode switching switch for the driving mode of the vehicle 1). For example, the display control unit 34 may display a message on the display device 21 such as "The driving load is too high, so we recommend switching to automatic driving mode."

[0053] (operation) Figure 5A is a flowchart of an example of an information display method according to the first embodiment. In step S1, the dynamic traffic element detection unit 30 detects dynamic traffic elements around the vehicle 1. In step S2, the display control unit 34 calculates the congestion level of the dynamic traffic elements. The display control unit 34 determines whether the congestion level is above a threshold. If the congestion level is above the threshold (step S2:Y), the process proceeds to step S3. If the congestion level is not above the threshold (step S2:N), the process proceeds to step S12.

[0054] In step S3, the display control unit 34 displays an image in simplified display mode (for example, the image in Figure 4A) on the display device 21. In step S4, the brain activity sensor 23 measures the activity level of the driver's brain. In step S5, the gaze detection unit 31 detects the driver's gaze. In step S6, the visibility target determination unit 32 determines whether the driver's visible object is a dynamic traffic element. If the driver's visible object is a dynamic traffic element (step S6:Y), the process proceeds to step S7. If the driver's visible object is not a dynamic traffic element (step S6:N), the process returns to step S4.

[0055] In step S7, the immersion calculation unit 33 calculates the driver's level of immersion in dynamic traffic elements based on the measurement results from the brain activity sensor 23 and the determination results from the visual target determination unit 32. In step S8, the display control unit 34 determines whether the driver's level of immersion is within the target range set according to the degree of congestion of the dynamic traffic elements. If the driver's level of immersion is within the target range (step S8:Y), the display control unit 34 determines that the level of immersion is appropriate for the congestion level. The display control unit 34 terminates the process without changing the amount of information displayed on the display device 21.

[0056] If the driver's level of immersion is not within the target range (step S8:N), the process proceeds to step S9. In step S9, the display control unit 34 determines whether the driver's level of immersion is higher than the target range. If the driver's level of immersion is higher than the target range (step S9:Y), the process proceeds to step S10. If the driver's level of immersion is not higher than the target range (step S9:N), the process proceeds to step S11.

[0057] In step S10, the display control unit 34 switches the driving mode of the vehicle 1 from manual driving mode to automatic driving (AD) mode. The process then ends. In step S11, the display control unit 34 displays instructions on the display device 21 to increase the driver's immersion in dynamic traffic elements (for example, instructions to draw the driver's attention to dynamic traffic elements). The process then ends.

[0058] On the other hand, in step S12, the display control unit 34 displays an image in high-quality display mode (for example, the image in Figure 3A) on the display device 21. In step S13, the brain activity sensor 23 measures the activity level of the driver's brain. In step S14, the gaze detection unit 31 detects the driver's gaze. In step S15, the visibility target determination unit 32 determines whether the driver's visible object is a dynamic traffic element. If the driver's visible object is a dynamic traffic element (step S15:Y), the process proceeds to step S16. If the driver's visible object is not a dynamic traffic element (step S15:N), the process returns to step S13.

[0059] In step S16, the immersion calculation unit 33 calculates the driver's level of immersion in dynamic traffic elements based on the measurement results from the brain activity sensor 23 and the determination results from the visual target determination unit 32. In step S17, the display control unit 34 determines whether the driver's level of immersion is within the target range. If the driver's level of immersion is within the target range (step S17:Y), the display control unit 34 determines that the level of immersion is appropriate for the congestion level. The display control unit 34 terminates the process without changing the amount of information displayed on the display device 21.

[0060] If the driver's level of immersion is not within the target range (step S17:N), the process proceeds to step S18. In step S18, the display control unit 34 determines whether the driver's level of immersion is higher than the target range. If the driver's level of immersion is higher than the target range (step S18:Y), the process proceeds to step S19. If the driver's level of immersion is not higher than the target range (step S18:N), the process proceeds to step S20. In step S19, the display control unit 34 displays an instruction on the display device 21 to instruct the driver to concentrate on driving the vehicle. The process then ends. In step S20, the display control unit 34 displays an instruction on the display device 21 to increase the driver's alertness. The process then ends.

[0061] (modified version) Figure 5B is a flowchart of an example of a modified version of the information display method of the first embodiment. In this modified version, when a video (which may be a video with sound) is output from the display device 21, the output format of the video is changed according to the driver's level of immersion. For this reason, the processes in steps S10, S11, S19, and S20 in Figure 5A are replaced by the processes in steps S21, S22, S23, and S24, respectively.

[0062] When the display device 21 is outputting a video with sound, if it is determined that the degree of congestion of dynamic traffic elements is high (step S2:Y) ​​and the degree of immersion is higher than the target range (step S9:Y), the display control unit 34 may determine in step S21 that the driver's workload is too high and stop outputting both the video and the audio included in the video with sound. When the display device 21 is outputting a video with sound, if it is determined that the degree of congestion of dynamic traffic elements is high (step S2:Y) ​​and the level of immersion is lower than the target range (step S9:N), the display control unit 34 may, in step S22, determine that the driver's attention to the dynamic traffic elements is insufficient and stop outputting the video from the video and audio included in the video with sound. The audio may continue to be output.

[0063] Furthermore, when the display device 21 is outputting a video, if it is determined that the degree of congestion of dynamic traffic elements is low (step S2:N) and the degree of immersion is higher than the target range (step S18:Y), the display control unit 34 may determine in step S23 that the driver is thinking too much about the content of the video and increase the frame rate of playback of the video output from the display device 21 (i.e., widen the frame rate of playback). By widening the frame rate of playback of the video in this way, the amount of time the driver has to think about the content of the video is reduced, allowing them to concentrate on driving and improving safety. If the video output from the display device 21 is a video with sound, the sound may continue to be output. When a video is being output from the display device 21, if the degree of congestion of the dynamic traffic elements is low (step S2:N) and the level of immersion is lower than the target range (step S18:N), the display control unit 34 may reduce the frame rate of playback of the video output from the display device 21 in step S24 (i.e., narrow the frame rate). By reducing the frame rate of playback of the video in this way and making the content easier to understand, the level of alertness can be increased. If the video output from the display device 21 is a video with sound, the sound may continue to be output.

[0064] (Second Embodiment) Next, the information display device 20 of the second embodiment will be described. As described above, it is preferable to set the target range of immersion higher when the degree of congestion of dynamic traffic elements is high compared to when the degree of congestion is low. For example, the higher the degree of congestion of dynamic traffic elements, the higher the target range may be set. Therefore, the information display device 20 of the second embodiment sets a target range for immersion using a database of target ranges for immersion, which is the degree of congestion of dynamic traffic elements, i.e., the number of dynamic traffic elements detected by the dynamic traffic element detection unit 30.

[0065] Figure 6 is a block diagram of an example of the functional configuration of the information display device 20 of the second embodiment. The information display device 20 of the second embodiment has a configuration similar to the information display device 20 of the first embodiment shown in Figure 2, and the same or similar components are denoted by the same reference numerals. The information display device 20 of the second embodiment includes a target range database (target range DB) 35.

[0066] The target range database 35 is a database that defines target ranges for driver immersion in relation to the number and congestion levels of dynamic traffic elements. The target range database 35 may be a map that associates the number and congestion levels of dynamic traffic elements with the target range of immersion, or a table that stores the number and congestion levels of dynamic traffic elements with the target range of immersion. The display control unit 34 sets the target range for immersion based on the number and degree of congestion of dynamic traffic elements detected by the dynamic traffic element detection unit 30, and the target range database 35.

[0067] The information display device 20 may have a separate target range database 35 for each of the multiple different users (i.e., drivers) using the vehicle 1, or it may have a target range database 35 that is common to multiple different users. If a separate target range database 35 is provided for each user, the display control unit 34 may adjust the target range database 35 for each individual user according to their needs.

[0068] For example, the display control unit 34 may correct the target range database 35 based on the immersion level calculated by the immersion level calculation unit 33 for each user. For example, the target range database 35 may be corrected based on the immersion history of each user. For example, the display control unit 34 may correct the target range database 35 based on statistical quantities such as the average value, median value, and standard deviation of each user's past immersion levels.

[0069] Furthermore, for example, when the display control unit 34 sets a target range based on the number of dynamic traffic elements detected by the dynamic traffic element detection unit 30 and the target range database 35, it may correct the target range database 35 based on the difference between the set target range and the immersion level calculated by the immersion level calculation unit 33. For example, if the immersion level calculated by the immersion level calculation unit 33 is higher than the target range, the target range set in the map or table of the target range database 35 may be updated to be higher by a predetermined correction amount. Conversely, if the immersion level calculated by the immersion level calculation unit 33 is lower than the target range, the target range set in the map or table of the target range database 35 may be updated to be lower by a predetermined correction amount.

[0070] Figure 7 is a flowchart of an example of the information display method of the second embodiment. In the information display method of the second embodiment, step S30 is added between step S7 and step S8, steps S31 and S32 are added after steps S10 and S11 respectively, step S33 is added between step S16 and step S17, and steps S34 and S35 are added after steps S19 and S20 respectively.

[0071] In steps S30 and S33, the display control unit 34 sets a target range for the driver's level of immersion in dynamic traffic elements based on the number of dynamic traffic elements detected in step S1 and the target range database 35. Furthermore, in steps S31 and S32, the display control unit 34 corrects the target range database 35 based on the immersion level calculated in step S7. In steps S34 and S35, the display control unit 34 corrects the target range database 35 based on the immersion level calculated in step S16.

[0072] (Effects of the embodiment) (1) The information display method detects dynamic traffic elements, which are movable elements among the components of the traffic environment around the vehicle, measures the brain activity of the vehicle's driver, determines whether the driver is visually viewing the detected dynamic traffic elements, calculates the driver's level of immersion in the dynamic traffic elements based on the brain activity measured when the driver is visually viewing the detected dynamic traffic elements, sets a target range for the driver's level of immersion in the dynamic traffic elements according to the degree of congestion of the dynamic traffic elements, and controls the display on a display device visible to the driver so that the driver's level of immersion in the dynamic traffic elements approaches the target range if the calculated level of immersion falls outside the set target range. This allows the driver's level of engagement with dynamic traffic elements to be guided to a level appropriate to the level of congestion of those elements, thereby improving safety.

[0073] (2) The driver's brain activity may be measured based on at least one of the driver's electroencephalogram, electroencephalogram, electroencephalogram, and brain surface activity. This allows for guidance of immersion levels appropriate to the congestion level of dynamic traffic elements with high temporal resolution, thereby improving safety. (3) The driver's level of immersion may be determined based on the magnitude of the driver's brain activity. This allows for the guidance of an appropriate level of immersion based on the degree of immersion determined by the highly accurate measurement of brain activity, thereby further improving safety.

[0074] (4) The driver's line of sight may be detected, and based on the detected line of sight, it may be determined whether or not the driver is visually observing the detected dynamic traffic elements. This allows for the guidance of an appropriate level of immersion based on the degree of immersion determined by quantitative and highly accurate visual detection means, thereby further improving safety. (5) A target range for driver immersion may be set based on a database that defines a target range for driver immersion relative to the number of dynamic traffic elements, and the number of dynamic traffic elements detected. This improves responsiveness by allowing the target range to be set quickly through matching with the database.

[0075] (6) The database may be adjusted for each individual driver according to the driver of the vehicle. This allows for more accurate setting of target ranges using a database that reflects individual characteristics, thus improving safety. (7) The greater the degree of congestion of dynamic traffic elements, the higher the target range for driver immersion may be set. This allows for guidance to an immersive level appropriate to the congestion level of dynamic traffic elements, thereby improving safety. (8) The greater the degree of congestion of the dynamic traffic elements, the less information may be displayed on the display device. This allows for guidance to an immersive level appropriate to the congestion level of dynamic traffic elements, thereby improving safety.

[0076] (9) If the degree of congestion of the dynamic traffic elements is less than a threshold and the calculated level of immersion is lower than the set target range, content that increases the driver's alertness may be displayed on the display device. This improves safety by guiding the driver to an immersion level appropriate to the degree of congestion of the dynamic traffic elements. When a video is being output from the display device, if the congestion level of the dynamic traffic elements is determined to be below the threshold and the level of immersion is below the set target range, the frame rate of the video output from the display device may be reduced. By reducing the frame rate of the video in this way and making the content easier to understand, the level of alertness can be increased. (10) If the degree of congestion of dynamic traffic elements is less than a threshold and the calculated level of immersion is higher than the set target range, an instruction to the driver to concentrate on driving the vehicle may be displayed on the display device. This allows for guidance to an immersive level appropriate to the congestion level of dynamic traffic elements, thereby improving safety. When a video is being output from the display device, if the degree of congestion of dynamic traffic elements is less than a threshold and the calculated level of immersion is higher than the set target range, it may be determined that the driver is thinking too much about the video content, and the frame rate of playback of the video output from the display device may be increased. This reduces the amount of time the driver spends thinking about the video content, allowing them to concentrate on driving and improving safety.

[0077] (11) If the degree of congestion of the dynamic traffic elements is determined to be above a threshold and the calculated level of immersion is lower than the set target range, an instruction to draw the driver's attention to the dynamic traffic elements may be displayed on the display device. If the display device is outputting a video with sound, and it is determined that the congestion level of dynamic traffic elements is above a threshold, and the calculated level of immersion is lower than the set target range, then it may be determined that the driver's attention to the dynamic traffic elements is insufficient, and the output of the video and audio included in the video with sound may be stopped, and the audio output may be continued. This allows for guidance to an immersive level appropriate to the congestion level of dynamic traffic elements, thereby improving safety. (12) If the degree of congestion of the dynamic traffic elements is determined to be above a threshold and the calculated immersion level is higher than the set target range, the display device may show a message prompting the driver to switch the vehicle's driving mode from manual driving mode to automatic driving mode, or the driving mode may be automatically switched from manual driving mode to automatic driving mode. If the display device is outputting video with sound, and the degree of congestion of dynamic traffic elements is above a threshold, and the calculated level of immersion is determined to be higher than the set target range, the driver's workload may be deemed too high, and the output of both the video and audio included in the video with sound may be stopped. This allows the system to switch to an automated driving mode depending on the level of congestion in dynamic traffic elements, thereby improving safety. [Explanation of Symbols]

[0078] 1...Vehicle, 10...Driving assistance system, 11...Positioning device, 12...Map database, 13...Object sensor, 14...Vehicle sensor, 15...Driving assistance controller, 16...Actuator, 20...Information display device, 21...Display device, 22...Display controller, 23...Brain activity sensor, 23a...Brain activity measurement unit, 23b...Blink detection unit, 24...Occupant sensor, 25...Processor, 26...Storage device, 30...Dynamic traffic element detection unit, 31...Gaze detection unit, 32...Visual target determination unit, 33...Immersion level calculation unit, 34...Display control unit, 35...Target range database

Claims

1. Among the components of the traffic environment surrounding the vehicle, the system detects preceding vehicles, following vehicles, oncoming vehicles traveling around the vehicle, and pedestrians and cyclists located around the vehicle as dynamic traffic elements. The brain activity of the driver of the aforementioned vehicle is measured, It is determined whether the driver has visually observed the detected dynamic traffic element. Based on the brain activity measured while the driver is visually observing the detected dynamic traffic elements, the degree of the driver's immersion in the dynamic traffic elements is calculated. Referencing a database that defines a target range for the driver's level of immersion relative to the number of dynamic traffic elements, and setting the target range for the driver's level of immersion based on the number of detected dynamic traffic elements and the database, If the calculated level of immersion falls outside the set target range, the display on the driver-visible display device is controlled so that the driver's level of immersion in the dynamic traffic elements approaches the target range. For each individual driver, the database is corrected based on the difference between the set target range and the calculated level of immersion. A method for displaying information characterized by the following features.

2. The information display method according to claim 1, characterized in that the brain activity of the driver is measured based on at least one of the driver's electroencephalogram, electroencephalogram, electroencephalogram, and brain surface activity state.

3. The information display method according to claim 1 or 2, characterized in that the driver's level of immersion is determined based on the magnitude of the driver's brain activity.

4. The driver's line of sight is detected, The information display method according to claim 1 or 2, characterized in that it determines whether or not the driver is visually viewing the detected dynamic traffic element based on the detected line of sight.

5. The information display method according to claim 1 or 2, characterized in that the target range for the driver's level of immersion is set higher as the degree of congestion of the dynamic traffic elements increases.

6. The information display method according to claim 1 or 2, characterized in that when it is determined that the degree of congestion of the dynamic traffic elements is less than a threshold and the calculated level of immersion is lower than the set target range, content that increases the driver's alertness is displayed on the display device.

7. The information display method according to claim 1 or 2, characterized in that when it is determined that the degree of congestion of the dynamic traffic elements is less than a threshold and the calculated level of immersion is higher than the set target range, an instruction to the driver to concentrate on driving the vehicle is displayed on the display device.

8. The information display method according to claim 1 or 2, characterized in that, when it is determined that the degree of congestion of the dynamic traffic elements is above a threshold and the calculated level of immersion is lower than the set target range, an instruction to draw the driver's attention to the dynamic traffic elements is displayed on the display device.

9. The information display method according to claim 1 or 2, characterized in that when it is determined that the degree of congestion of the dynamic traffic elements is above a threshold and the calculated immersion level is higher than the set target range, a message prompting the user to switch the driving mode of their vehicle from manual driving mode to automatic driving mode is displayed on the display device.

10. A first sensor detects, as dynamic traffic elements, the components of the traffic environment around the vehicle, including preceding vehicles, following vehicles, oncoming vehicles traveling around the vehicle, and pedestrians and cyclists located around the vehicle. A second sensor for measuring the brain activity of the driver of the aforementioned vehicle, The aforementioned display device that can be seen by the driver, It is determined whether the driver has visually observed the detected dynamic traffic element. A controller that calculates the driver's level of immersion in the dynamic traffic elements based on the brain activity measured when the driver is visually observing the detected dynamic traffic elements, refers to a database that defines a target range for the driver's level of immersion relative to the number of dynamic traffic elements, sets a target range for the driver's level of immersion based on the number of detected dynamic traffic elements and the database, controls the display on the display device so that the driver's level of immersion in the dynamic traffic elements approaches the target range if the calculated level of immersion falls outside the set target range, and corrects the database for each individual driver based on the difference between the set target range and the calculated level of immersion. An information display device characterized by having the following features.

Images