Notification method, notification system, and vehicle

Abstract

To notify a driver of a vehicle of appropriate strength.SOLUTION: A notification method provides a notification to a driver sitting in a driver seat of a vehicle. Specifically, a target level of notification strength is set. When the target level is a first level, a visual notification is provided from a visual device of the vehicle without providing an audio notification from a speaker of the vehicle. When the target level is higher than the first level, a stronger visual notification is output from the visual device or the audio notification is output from the speaker than when the target level is the first level.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to techniques for providing notifications to drivers of vehicles. [Background technology]

[0002] Patent Document 1 discloses an alertness maintaining device for maintaining the alertness of a vehicle driver. When the alertness maintaining device detects a decrease in the driver's alertness, it provides the driver with an audio stimulus of an intensity corresponding to the driver's alertness. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-159711 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the technology disclosed in Patent Document 1, an audio stimulus is given to a driver of a vehicle. The audio stimulus is considered to be strong and effective, but due to its strength, the driver may perceive the audio stimulus as excessive. In other words, the driver may perceive the audio stimulus as annoying.

[0005] One object of the present disclosure is to provide a technology that can issue a notification of appropriate strength to a vehicle driver. [Means for solving the problem]

[0006] The first aspect relates to a notification method for issuing a notification to a driver sitting in the driver's seat of a vehicle. The notification method is Setting a target level of notification intensity; If the target level is the first level, issuing a visual notification from a visual device of the vehicle without issuing an audio notification from a speaker of the vehicle; When the target level is higher than the first level, a stronger visual notification is output from the visual device or an audio notification is output from the speaker than when the target level is the first level. Includes.

[0007] The second aspect relates to a notification system that issues a notification to a driver sitting in the driver's seat of a vehicle. The notification system includes one or more processors. The one or more processors Set a target level of notification intensity, If the target level is the first level, a visual notification is issued from a visual device of the vehicle without issuing an audio notification from a speaker of the vehicle; If the target level is higher than the first level, a stronger visual notification is issued from the visual device or an audio notification is issued from the speaker than when the target level is the first level. It is configured as follows.

[0008] The third aspect relates to vehicles. The vehicle is Visual devices and A speaker and a control device that controls notifications to a driver sitting in the driver's seat; Equipped with. The control device Set a target level of notification intensity, If the target level is the first level, a visual notification is issued from the visual device without issuing an audio notification from the speaker; If the target level is higher than the first level, a stronger visual notification is issued from the visual device or an audio notification is issued from the speaker than when the target level is the first level. It is configured as follows. [Effects of the Invention]

[0009] According to the present disclosure, when the target level of the notification intensity is low, the audio notification is not issued, but the visual notification is issued, and the stimulus provided to the driver by the visual notification is weaker than the stimulus provided to the driver by the audio notification, thereby reducing the driver's perceived annoyance by the notification. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a conceptual diagram for explaining an overview of a notification system according to an embodiment. [Figure 2] 1 is a block diagram illustrating a configuration example of a notification system according to an embodiment. [Figure 3] 1 is a diagram illustrating an example of a plurality of visual devices mounted on a vehicle according to an embodiment; [Figure 4] FIG. 2 is a conceptual diagram for explaining the positional relationship between a first visual device and a second visual device according to the embodiment. [Figure 5] FIG. 10 is a conceptual diagram for explaining an overview of a notification control process according to an embodiment. [Figure 6] 10 is a flowchart illustrating a first example of a notification control process according to an embodiment. [Figure 7] 10 is a flowchart illustrating a second example of a notification control process according to an embodiment. [Figure 8] 10A to 10C are conceptual diagrams illustrating various patterns of notification control processing according to an embodiment. [Figure 9] 1 is a block diagram showing an example of the configuration of a vehicle control system according to an embodiment; DETAILED DESCRIPTION OF THE INVENTION

[0011] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0012] 1. Notification System 1 is a conceptual diagram for explaining an overview of a notification system 10 according to this embodiment. The notification system 10 is mounted on a vehicle 1 and issues various notifications N to a driver 3 sitting in a driver's seat 2 of the vehicle 1. The vehicle 1 may be an autonomous vehicle. Note that even during autonomous driving, the operator sitting in the driver's seat 2 is referred to as the driver 3.

[0013] The notification system 10 issues a notification N when a predetermined notification condition is met. For example, the notification condition is that the wakefulness level of the driver 3 falls below a threshold. As another example, the notification condition is that, during the automatic driving of the vehicle 1, it becomes necessary to request the driver 3 to monitor the surrounding situation. As yet another example, the notification condition is that, during the automatic driving of the vehicle 1, it becomes necessary to request the driver 3 to hold the steering wheel. As yet another example, the notification condition is that, during the automatic driving of the vehicle 1, it becomes necessary to request the driver 3 to switch to manual driving. As yet another example, the notification condition is that it becomes necessary to request the driver 3 to apply the brakes. As yet another example, the notification condition is that driving assistance control that assists the driving of the vehicle 1 is activated. Examples of driving assistance control include collision avoidance control and lane departure prevention control.

[0014] The types of notification N to the driver 3 include "visual notification NV" and "audio notification NA." Visual notification NV is a notification using visual information such as light emission or display. Audio notification NA is a notification using audio information such as a voice message or buzzer.

[0015] 2 is a block diagram showing an example of the configuration of a notification system 10 according to this embodiment. The notification system 10 includes a visual device 20, a speaker 30, a driver monitor 40, and a control device 50.

[0016] The visual device 20 issues a visual notification NV to the driver 3. Examples of the visual device 20 include a light-emitting device and a display panel. An example of a light-emitting device is an LED (Light Emitting Diode). A light-emitting device can provide (express) different information by emitting light in different colors. Examples of a display panel include a liquid crystal panel, an organic EL panel, etc. Typically, the visual device 20 is installed in the interior of the vehicle 1.

[0017] The speaker 30 issues an audio notification NA to the driver 3. Typically, the speaker 30 is installed inside the vehicle 1.

[0018] The driver monitor 40 is a device for detecting the state and actions of the driver 3. For example, the driver monitor 40 includes a camera installed in a position where it can capture an image of the driver 3. The driver monitor 40 may include a steering touch sensor installed on a steering wheel operated by the driver 3. The driver monitor 40 may detect driving operations by the driver 3.

[0019] The control device 50 controls the notification system 10. More specifically, the control device 50 controls the visual device 20 and the speaker 30 to control the notification N to the driver 3. The control device 50 also analyzes information detected by the driver monitor 40. For example, the control device 50 detects various states of the driver 3 by analyzing images obtained by a camera. For example, the control device 50 can detect the driver 3's facial direction, line of sight, eye opening degree, opening degree, etc. The control device 50 can also determine whether the driver 3 is gripping the steering wheel or has their hands off the steering wheel based on the detection results from the steering touch sensor.

[0020] The control device 50 includes one or more processors 51 (hereinafter simply referred to as processors 51) and one or more storage devices 52 (hereinafter simply referred to as storage devices 52). The processor 51 executes various processes. The storage device 52 stores various information. Examples of the storage device 52 include a volatile memory, a non-volatile memory, a hard disk drive (HDD), and a solid state drive (SSD). The processor 51 may execute a notification control program, which is a computer program for controlling the notification system 10. The functions of the control device 50 are realized by cooperation between the processor 51 that executes the notification control program and the storage device 52. The notification control program is stored in the storage device 52. Alternatively, the notification control program may be recorded on a computer-readable recording medium.

[0021] The notification system 10 may include multiple visual devices 20. For example, the notification system 10 includes a first visual device 20-1 and a second visual device 20-2. The first visual device 20-1 issues a first visual notification NV1 to the driver 3. The second visual device 20-2 issues a second visual notification NV2 to the driver 3.

[0022] 3 shows an example of multiple visual devices 20 mounted on a vehicle 1. The multiple visual devices 20 include door illuminations 21 installed in the doors, pillar illuminations 22 installed in the window frames, instrument panel illuminations 23 installed in the instrument panel, and steering illuminations 24 installed in the steering wheel. Each illumination is a light-emitting device including multiple LEDs. Each illumination is arranged in a strip shape.

[0023] The forward direction D0 is the direction from the driver's seat 2 toward the front of the vehicle 1. In the example shown in Fig. 3, the door illuminations 21L, pillar illuminations 22L, and instrument panel illuminations 23L are installed to the left of the forward direction D0. On the other hand, the door illuminations 21R, pillar illuminations 22R, and instrument panel illuminations 23R are installed to the right of the forward direction D0. The steering illumination 24 is installed approximately in the forward direction D0.

[0024] FIG. 4 is a conceptual diagram illustrating the positional relationship between the first visual device 20-1 and the second visual device 20-2 according to this embodiment. The forward direction D0 is the direction from the driver's seat 2 toward the front of the vehicle 1. The first direction D1 is the direction from the driver's seat 2 toward the first visual device 20-1. The second direction D2 is the direction from the driver's seat 2 toward the second visual device 20-2. The first angle θ1 is the angle between the first direction D1 and the forward direction D0. The second angle θ2 is the angle between the second direction D2 and the forward direction D0. As shown in FIG. 4, the first angle θ1 is greater than the second angle θ2. Therefore, when the driver 3 sitting in the driver's seat 2 faces forward (forward direction D0), the first visual device 20-1 is far from the line of sight of the driver 3, and the second visual device 20-2 is close to the line of sight of the driver 3.

[0025] The human visual field is divided into the effective visual field (central visual field) and the peripheral visual field. Humans can clearly perceive objects in the effective visual field. On the other hand, humans can vaguely perceive objects in the peripheral visual field. Preferably, when the driver 3 sitting in the driver's seat 2 is facing forward (forward direction D0), the first visual device 20-1 neighborhood The second visual device 20-2 is within the field of view of the driver 3. Effective field of view ( central vision ) It is included in.

[0026] Here, let us consider the average human field of view angle. The average field of view angle is the average value of the field of view angles of a certain number of people and is commonly known. For example, the average human field of view is about 100 degrees left and right (200 degrees in total), and the average human effective field of view is about 30 degrees left and right (60 degrees in total). The reference angle α is half the angle of the average human effective field of view (e.g., 30 degrees). The limit angle β is half the angle of the average human field of view (e.g., 100 degrees). The first angle θ1 between the first direction D1 and the forward direction D0 may be greater than the reference angle α and smaller than the limit angle β. The second angle θ2 between the second direction D2 and the forward direction D0 may be smaller than the reference angle α.

[0027] In the example shown in Figure 3, the door illumination 21L, pillar illumination 22L, instrument panel illumination 23L, and door illumination 21R may belong to the first visual device 20-1, while the steering illumination 24 may belong to the second visual device 20-2.

[0028] 2. Notification control processing The stimulus that the audio notification NA provides to the driver 3 is stronger than the stimulus that the visual notification NV provides to the driver 3. In that sense, the audio notification NA is effective. However, because the stimulus of the audio notification NA is strong, the driver 3 may perceive the audio notification NA as excessive. In other words, the driver 3 may find the audio notification NA annoying. Furthermore, the audio notification NA can be heard not only by the driver 3 but also by passengers. Therefore, the passengers may also find the audio notification NA annoying.

[0029] It is desirable to issue a notification N of appropriate strength to the driver 3 of the vehicle 1. To this end, the notification system 10 (control device 50) according to this embodiment executes a "notification control process" as described below.

[0030] FIG. 5 is a conceptual diagram for explaining an outline of the notification control process according to this embodiment. In the following description, "strength of notification N" means "strength of the stimulus that notification N provides to driver 3." The audio notification NA from the speaker 30 is stronger than the visual notification NV from the visual device 20. Conversely, the visual notification NV from the visual device 20 is weaker than the audio notification NA from the speaker 30. Furthermore, among the visual notifications NV, the second visual notification NV2 from the second visual device 20-2 is stronger than the first visual notification NV1 from the first visual device 20-1. Conversely, the first visual notification NV1 from the first visual device 20-1 is weaker than the second visual notification NV2 from the second visual device 20-2.

[0031] The control device 50 sets (determines) a "target level" for the strength of the notification N. Then, the control device 50 determines the type of notification N to the driver 3 according to the target level. When the target level is relatively low, the control device 50 selects a relatively weak notification N. Specifically, when the target level is relatively low, the control device 50 issues a visual notification NV from the visual device 20 without issuing an audio notification NA from the speaker 30. On the other hand, when the target level is relatively high, the control device 50 selects a relatively strong notification N. Specifically, when the target level is relatively high, the control device 50 issues a stronger visual notification NV from the visual device 20 or issues an audio notification NA from the speaker 30.

[0032] For example, the candidate target levels include three types: a first level LV1, a second level LV2, and a third level LV3. The second level LV2 is higher than the first level LV1, and the third level LV3 is even higher than the second level LV2. When the target level is the first level LV1, a strong notification N is not required, and the first visual notification NV1 is selected. When the target level is the third level LV3, a strong notification N is required, and the audio notification NA is selected. When the target level is the second level LV2, the second visual notification NV2, which has a medium strength, is selected.

[0033] <Effects> As described above, when the target level of the strength of the notification N is low, the audio notification NA is not issued, and the visual notification NV is issued. The stimulus that the visual notification NV provides to the driver 3 is weaker than the stimulus that the audio notification NA provides to the driver 3. Therefore, the driver 3 is prevented from feeling annoyed by the notification N. In addition, since the audio notification NA is not issued, the passenger is prevented from feeling annoyed by the notification N.

[0034] The visual notification NV may include a relatively weak first visual notification NV1 and a relatively strong second visual notification NV2. When the target level of the strength of the notification N is a very low first level LV1, the first visual notification NV1 is selected. When the target level is a second level LV2 higher than the first level LV1, the second visual notification NV2 is selected. This enables more detailed notification control. That is, when the target level is a very low first level LV1, the notification N is effectively prevented from being perceived as an annoyance by the driver 3. Furthermore, when the target level is a second level LV2 higher than the first level LV1, the driver's attention can be attracted to some extent while the annoyance felt by the driver 3 is reduced.

[0035] When the target level of the strength of the notification N is high, the audio notification NA is issued. The audio notification NA provides a strong stimulus to the driver 3, so the effect of the notification N is sufficiently obtained.

[0036] In this way, according to this embodiment, it is possible to issue a notification N of appropriate strength to the driver 3 of the vehicle 1.

[0037] 3. Example of notification control processing An example of the notification control process according to this embodiment will be described below.

[0038] 3-1. First example FIG. 6 is a flowchart showing a first example of the notification control process.

[0039] In step S100, the control device 50 determines whether or not the notification condition is met. In other words, the control device 50 determines whether or not notification to the driver 3 is necessary.

[0040] For example, the notification condition is that the wakefulness level of the driver 3 falls below a threshold. The control device 50 calculates the wakefulness level of the driver 3 based on information detected by the driver monitor 40. Specifically, the control device 50 detects the facial direction, line of sight, eye opening degree, openness degree, etc. of the driver 3 by analyzing images obtained by a camera. For example, the wakefulness level is calculated to be lower as the eye opening degree of the driver 3 decreases. As another example, the wakefulness level is calculated to be lower as the facial direction of the driver 3 moves away from the forward direction D0.

[0041] As another example, the notification condition is that a sign of drowsiness of the driver 3 is detected. A sign of drowsiness is a state in which the driver 3's alertness level is reduced, and refers to a state before and after the driver 3 feels drowsy. Typically, a sign of drowsiness appears as a drowsy behavior specific to the driver 3 before and after the driver 3 feels drowsy. Examples of drowsy behavior include half-closed eyes, heavy eyelids, frequent blinking, long yawns, and head movements. The control device 50 detects the drowsiness sign of the driver 3 based on an image obtained by the camera of the driver monitor 40.

[0042] As yet another example, the notification condition may be that, during automated driving of the vehicle 1, it is necessary to request the driver 3 to monitor the surrounding situation. As yet another example, the notification condition may be that, during automated driving of the vehicle 1, it is necessary to request the driver 3 to hold the steering wheel. As yet another example, the notification condition may be that, during automated driving of the vehicle 1, it is necessary to request the driver 3 to switch to manual driving. As yet another example, the notification condition may be that it is necessary to request the driver 3 to apply the brakes. As yet another example, the notification condition may be that driving assistance control that assists in driving the vehicle 1 is activated. This information is obtained from a vehicle control system that controls the vehicle 1. The vehicle control system will be described later in Section 5.

[0043] If the notification condition is met (step S100; Yes), notification N to the driver 3 is necessary, and the process proceeds to step S110. On the other hand, if the notification condition is not met (step S100; No), notification N to the driver 3 is not necessary, and the process for this cycle ends.

[0044] In step S110, the control device 50 first initially sets the target level to the first level LV1. Then, the control device 50 controls the first visual device 20-1 to issue a first visual notification NV1 from the first visual device 20-1. At this time, the control device 50 does not issue an audio notification NA from the speaker 30.

[0045] In step S115, the control device 150 determines whether the driver 3 has responded to the first visual notification NV1 within a certain period of time. For example, the response of the driver 3 is that the driver's line of sight approaches the first direction D1 toward the first visual device 20-1. The response of the driver 3 may be that the driver's line of sight substantially coincides with the first direction D1 toward the first visual device 20-1. As another example, the response of the driver 3 may be that the driver's alertness level recovers to a threshold or higher. As yet another example, the response of the driver 3 may be that the driver 3 performs a requested operation. Examples of operations requested of the driver 3 include gripping the steering wheel, switching to manual driving, and operating the brakes. In either case, the control device 50 can detect whether the driver 3 has responded based on information obtained by the driver monitor 40. If the driver 3 has responded (step S115; Yes), the processing for this cycle ends. On the other hand, if the driver 3 has not responded (step S115; No), the processing proceeds to step S120.

[0046] In step S120, the control device 50 sets (changes) the target level to the second level LV2. Then, the control device 50 controls the second visual device 20-2 to issue a second visual notification NV2 from the second visual device 20-2. At this time, the control device 50 does not issue an audio notification NA from the speaker 30.

[0047] In step S125, the control device 150 determines whether the driver 3 has responded to the second visual notification NV2 within a certain period of time. For example, the response of the driver 3 is that the driver's line of sight approaches the second direction D2 toward the second visual device 20-2. The response of the driver 3 may be that the driver's line of sight substantially coincides with the second direction D2 toward the second visual device 20-2. As another example, the response of the driver 3 may be that the driver's alertness level recovers to a threshold or higher. As yet another example, the response of the driver 3 may be that the driver 3 performs an operation requested of the driver 3. The control device 50 can detect whether the driver 3 has responded based on information obtained by the driver monitor 40. If the driver 3 has responded (step S125; Yes), the processing for this cycle ends. On the other hand, if the driver 3 has not responded (step S125; No), the processing proceeds to step S130.

[0048] In step S130, the control device 50 sets (changes) the target level to the third level LV3, and then controls the speaker 30 to output the audio notification NA.

[0049] In step S135, the control device 150 determines whether or not the driver 3 has responded to the audio notification NA within a certain period of time. For example, the response of the driver 3 is that the driver's alertness level recovers to a threshold or above. As another example, the response of the driver 3 may be that the driver 3 performs a requested operation. The control device 50 can detect whether or not the driver 3 has responded based on information obtained by the driver monitor 40. If the driver 3 has responded (step S135; Yes), the processing for this cycle ends. On the other hand, if the driver 3 has not responded (step S135; No), the processing proceeds to step S140.

[0050] In step S140, the control device 150 performs emergency control. For example, the control device 150 stops the vehicle 1 in a safe place (for example, on the shoulder of the road).

[0051] 3-2. Second example 7 is a flowchart showing a second example of the notification control process. Step S100 is the same as in the first example described above. If the notification condition is met (step S100; Yes), the process proceeds to step S105.

[0052] In step S105, the control device 50 determines (sets) a target level of the intensity of the notification N based on the result of step S100. For example, the control device 50 compares the wakefulness of the driver 3 with a first threshold TH1, a second threshold TH2, and a third threshold TH3. The second threshold TH2 is lower than the first threshold TH1, and the third threshold TH3 is even lower than the second threshold TH2 (TH1>TH2>TH3). If the wakefulness is less than the first threshold TH1 and equal to or greater than the second threshold TH2, the target level is set to the first level LV1. If the wakefulness is less than the second threshold TH2 and equal to or greater than the third threshold TH3, the target level is set to the second level LV2. If the wakefulness is less than the third threshold TH3, the target level is set to the third level LV3.

[0053] If the target level is the first level LV1, the process proceeds to step S110 shown in Fig. 6. If the target level is the second level LV2, the process proceeds to step S120 shown in Fig. 6. If the target level is the third level LV3, the process proceeds to step S130 shown in Fig. 6.

[0054] 4. Variations FIG. 8 is a conceptual diagram showing various patterns of notification control processing.

[0055] Pattern (A): When the target level is the first level LV1, the first visual notification NV1 is selected. When the target level is the second level LV2, the second visual notification NV2 is selected. When the target level is the third level LV3, the audio notification NA is selected.

[0056] Pattern (B): When the target level is the first level LV1, the first visual notification NV1 is selected. When the target level is the second level LV2 or the third level LV3, the second visual notification NV2 is selected.

[0057] Pattern (C): If the target level is the first level LV1 or the second level LV2, the first visual notification NV1 is selected. If the target level is the third level LV3, the second visual notification NV2 is selected.

[0058] Pattern (D): When the target level is the first level LV1, the visual notification NV is selected. The visual notification NV here may be the first visual notification NV1 or the second visual notification NV2. When the target level is the second level LV2 or the third level LV3, the audio notification NA is selected.

[0059] Pattern (E): When the target level is the first level LV1 or the second level LV2, the visual notification NV is selected. The visual notification NV here may be the first visual notification NV1 or the second visual notification NV2. When the target level is the third level LV3, the audio notification NA is selected.

[0060] The processing flow described in Section 3 above is modified as appropriate for each pattern. The above-mentioned technical effects can be obtained with any pattern.

[0061] 5. Vehicle Control System 5-1.Configuration example 9 is a block diagram showing an example of the configuration of a vehicle control system 100 according to this embodiment. The vehicle control system 100 includes the above-described notification system 10. The vehicle control system 100 includes a sensor group 110, a driving device 120, an HMI (Human Machine Interface) 130, a driver monitor 140, and a control device 150.

[0062] The sensor group 110 includes a recognition sensor, a vehicle state sensor, a position sensor, etc. The recognition sensor recognizes (detects) the situation around the vehicle 1. Examples of the recognition sensor include a camera, a LIDAR (Laser Imaging Detection and Ranging), a radar, etc. The vehicle state sensor detects the state of the vehicle 1. The vehicle state sensor includes a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, etc. The position sensor detects the position and orientation of the vehicle 1. For example, the position sensor includes a GNSS (Global Navigation Satellite System).

[0063] The traveling device 120 includes a steering device, a drive device, and a braking device. The steering device steers the wheels. For example, the steering device includes an electric power steering (EPS) device. The drive device is a power source that generates driving force. Examples of the drive device include an engine, an electric motor, and an in-wheel motor. The braking device generates braking force.

[0064] The HMI 130 presents various types of information to the driver 3 and also receives various types of information from the driver 3. The HMI 130 includes the above-mentioned visual device 20 and speaker 30. The HMI 130 also includes an input device such as a touch panel.

[0065] The driver monitor 140 includes the above-mentioned driver monitor 40. That is, the driver monitor 140 includes a camera for capturing an image of the driver 3, a steering touch sensor, and the like. The driver monitor 40 may detect the driving operation of the driver 3.

[0066] The control device 150 is a computer that controls the vehicle 1. The control device 150 includes one or more processors 160 (hereinafter simply referred to as processors 160) and one or more storage devices 170 (hereinafter simply referred to as storage devices 170). The processor 160 executes various processes. For example, the processor 160 includes a CPU (Central Processing Unit). The storage device 170 stores various information. Examples of the storage device 170 include a volatile memory, a non-volatile memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc. The control device 150 includes the control device 50 of the notification system 10.

[0067] The vehicle control program 180 is a computer program executed by the processor 160. The functions of the control device 150 are realized by cooperation between the processor 160, which executes the vehicle control program 180, and the storage device 170. The vehicle control program 180 is stored in the storage device 170. Alternatively, the vehicle control program 180 may be recorded on a computer-readable recording medium.

[0068] 5-2. Driving environment information The control device 150 uses the sensor group 110 to acquire driving environment information 190 that indicates the driving environment of the vehicle 1. The driving environment information 190 is stored in the storage device 170.

[0069] The driving environment information 190 includes surrounding situation information indicating the recognition results of the recognition sensor. For example, the surrounding situation information includes images captured by a camera. The surrounding situation information may include object information regarding objects around the vehicle 1. Examples of objects around the vehicle 1 include pedestrians, other vehicles (preceding vehicles, parked vehicles, etc.), white lines, traffic lights, signs, roadside structures, etc. The object information indicates the relative position and relative speed of the object with respect to the vehicle 1.

[0070] The driving environment information 190 also includes vehicle state information indicating the vehicle state detected by the vehicle state sensor.

[0071] Furthermore, the driving environment information 190 includes vehicle position information indicating the position and direction of the vehicle 1. The vehicle position information is obtained by a position sensor. Highly accurate vehicle position information may be obtained by a self-position estimation process (localization) using map information and surrounding situation information (object information).

[0072] Furthermore, the driving environment information 190 includes information obtained by the driver monitor 140 .

[0073] 5-3.Vehicle driving control The control device 150 executes vehicle driving control to control the driving of the vehicle 1. The vehicle driving control includes steering control, drive control, and braking control. The control device 150 executes vehicle driving control by controlling the driving device 120 (steering device, drive device, and brake device).

[0074] The control device 150 may perform automatic driving control based on the driving environment information 190. More specifically, the control device 150 generates a driving plan for the vehicle 1 based on the driving environment information 190. Furthermore, the control device 150 generates a target trajectory required for the vehicle 1 to drive according to the driving plan based on the driving environment information 190. The target trajectory includes a target position and a target speed. Then, the control device 150 performs vehicle driving control so that the vehicle 1 follows the target trajectory.

[0075] The control device 150 may perform driving assistance control to assist the driving of the vehicle 1 based on the driving environment information 190. Examples of driving assistance control include collision avoidance control and lane departure prevention control. Collision avoidance control controls at least one of steering and deceleration to assist in avoiding a collision between the vehicle 1 and a surrounding object. Lane departure prevention control controls steering to prevent the vehicle 1 from departing from the driving lane. When the activation conditions for the driving assistance control are met, the control device 150 activates the driving assistance control.

[0076] As an example of driving assistance control, consider collision avoidance control. The control device 150 recognizes an object to be avoided (e.g., a nearby vehicle, a pedestrian) ahead of the vehicle 1 based on surrounding situation information. Furthermore, the control device 150 predicts the future positions of the vehicle 1 and the object to be avoided based on the vehicle state information and surrounding situation information, and calculates the possibility of the vehicle 1 colliding with the object to be avoided. The activation condition for collision avoidance control is that the possibility of the vehicle 1 colliding with the object to be avoided exceeds an activation threshold.

[0077] As another example of driving assistance control, consider lane departure prevention control. For example, when vehicle 1 stumbles within its lane and approaches a lane marking (white line), lane departure prevention control steers vehicle 1 to return it to the center of the lane. To achieve this, control device 150 recognizes the lane marking of the lane in which vehicle 1 is traveling based on surrounding situation information, and monitors the distance between vehicle 1 and the marking. The condition for lane departure prevention control to be activated is that the distance between vehicle 1 and the marking is less than a predetermined distance threshold.

[0078] 5-4. Notification Control The control device 150 controls the HMI 130 (visual device 20, speaker 30) to control notification N to the driver 3 (see Sections 2 to 4). More specifically, the control device 150 issues notification N to the driver 3 when a predetermined notification condition is met.

[0079] For example, the notification condition is that the wakefulness level of the driver 3 falls below a threshold. The control device 150 can calculate the wakefulness level of the driver 3 based on information obtained by the driver monitor 140.

[0080] As another example, the notification condition is the detection of a sign of drowsiness in the driver 3. The control device 50 can detect the sign of drowsiness in the driver 3 based on an image obtained by the camera of the driver monitor 140.

[0081] As yet another example, the notification condition may be that, during automatic driving of the vehicle 1, it is necessary to request the driver 3 to monitor the surrounding situation. As yet another example, the notification condition may be that, during automatic driving of the vehicle 1, it is necessary to request the driver 3 to hold the steering wheel. As yet another example, the notification condition may be that, during automatic driving of the vehicle 1, it is necessary to request the driver 3 to switch to manual driving. As yet another example, the notification condition may be that it is necessary to request the driver 3 to apply the brakes. As yet another example, the notification condition may be that driving assistance control is activated.

[0082] When the notification condition is met, the control device 150 issues a notification N to the driver 3 with an appropriate strength according to the target level. [Explanation of symbols]

[0083] 1...vehicle, 2...driver's seat, 3...driver, 10...notification system, 20...visual device, 20-1...first visual device, 20-2...second visual device, 30...speaker, 40...driver monitor, 50...control device, 100...vehicle control system, D0...forward direction, D1...first direction, D2...second direction, N...notification, NA...audio notification, NV...visual notification, NV1...first visual notification, NV2...second visual notification

Claims

1. A notification method for issuing a notification to a driver sitting in a driver's seat of a vehicle, comprising: setting a target level of the intensity of the notification; if the target level is a first level, issuing a first visual notification from a first visual device of the vehicle without issuing an audio notification from a speaker of the vehicle; if the target level is a second level higher than the first level, outputting a second visual notification from a second visual device of the vehicle, the second visual notification being stronger than the first visual notification, without outputting the audio notification from the speaker; Including, the forward direction is a direction from the driver's seat toward the front of the vehicle, the first direction is a direction from the driver's seat toward the first visual device, the second direction is a direction from the driver's seat toward the second visual device, A first angle between the first direction and the forward direction is greater than a second angle between the second direction and the forward direction. Notification method.

2. 2. The notification method according to claim 1, The reference angle is half the angle of the average human visual field, The limiting angle is half the angle of the average human visual field, the first angle between the first direction and the forward direction is greater than the reference angle and less than the limit angle; The second angle between the second direction and the forward direction is smaller than the reference angle. Notification method.

3. 2. The notification method according to claim 1, If the target level is a third level that is higher than the second level, the audio notification is issued from the speaker. Further includes Notification method.

4. 4. The notification method according to claim 1, further comprising: determining whether the notification to the driver is necessary; If the notification to the driver is required, initially setting the target level to the first level; determining whether the driver responded to the first visual notification when the target level is the first level; If the driver does not respond to the first visual notification, setting the target level higher than the first level. Further includes Notification method.

5. The notification method according to claim 3, determining whether the notification to the driver is necessary; If the notification to the driver is required, initially setting the target level to the first level; determining whether the driver responded to the first visual notification; If the driver does not respond to the first visual notification, setting the target level to the second level; determining whether the driver responded to the second visual notification; If the driver does not respond to the second visual notification, setting the target level to the third level. Further includes Notification method.

6. A notification system that issues a notification to a driver sitting in a driver's seat of a vehicle, one or more processors; the one or more processors: setting a target level of the intensity of the notification; if the target level is a first level, issuing a first visual notification from a first visual device of the vehicle without issuing an audio notification from a speaker of the vehicle; If the target level is a second level higher than the first level, the audio notification is not issued from the speaker, and a second visual notification stronger than the first visual notification is issued from a second visual device of the vehicle. It is configured as follows: the forward direction is a direction from the driver's seat toward the front of the vehicle, the first direction is a direction from the driver's seat toward the first visual device, the second direction is a direction from the driver's seat toward the second visual device, A first angle between the first direction and the forward direction is greater than a second angle between the second direction and the forward direction. Notification system.

7. A visual device including a first visual device and a second visual device; A speaker and a control device that controls notifications to a driver sitting in the driver's seat; Equipped with The control device setting a target level of the intensity of the notification; if the target level is a first level, issuing a first visual notification from the first visual device without issuing an audio notification from a speaker of the vehicle; If the target level is a second level higher than the first level, the audio notification is not issued from the speaker, and a second visual notification stronger than the first visual notification is issued from the second visual device. It is configured as follows: the forward direction is a direction from the driver's seat toward the front of the vehicle, the first direction is a direction from the driver's seat toward the first visual device, the second direction is a direction from the driver's seat toward the second visual device, A first angle between the first direction and the forward direction is greater than a second angle between the second direction and the forward direction. vehicle.

Images