Attention reminding device and attention reminding method

Abstract

The invention provides an attention reminding device and an attention reminding method, which enable a driver of a vehicle to intuitively recognize a change in risk level in a scene in which sounds of a plurality of languages are output for a short period of time. A warning device (100) is provided with: a risk index value acquisition unit (132) that acquires a risk index value indicating the risk of collision between a vehicle (1) and an object present in front of the vehicle (1) on the basis of outputs from a plurality of sensors mounted on the vehicle (1); and a notification control unit (135) that, when an object whose risk index value is equal to or greater than a preset first threshold value is detected, outputs a voice to notify the driver of the vehicle of the object, and, in a process of notifying the driver of the vehicle of the object by the output of the voice of the voice, notifies the driver of the object of the object by the output of the voice of the voice of the voice of the voice of the voice of the voice of the voice of the voice of the voice. When the risk indicator value changes at or above a preset second threshold value, a driver of the vehicle is notified that the risk indicator value has changed by a method other than voice-based notification.

Description

Technical Field

[0001] This invention relates to a attention reminder device and a attention reminder method. Background Technology

[0002] In recent years, systems have been known to notify vehicle drivers of risks through multiple HMI (Human Machine Interface) devices.

[0003] For example, the driving assistance device disclosed in Patent Document 1 includes a notification unit that acquires object information including the position, direction of travel, and speed of other vehicles, detects driving information including the position, direction of travel, and speed of the vehicle itself, and issues a notification based on the detected information. Furthermore, the driving assistance device includes a display device that shows the positional relationship between the vehicle and other vehicles on a map, and a speaker that outputs sound corresponding to the risk level as the notification unit.

[0004] Furthermore, the information providing device disclosed in Patent Document 2 includes an output control unit that outputs sound information and displays display output information generated based on at least a portion of the sound output information before the sound output information is about to be output. The display output information is displayed for a predetermined time before the sound output information corresponding to the display output information is about to be output.

[0005] [Existing technical documents]

[0006] [Patent Literature]

[0007] [Patent Document 1]

[0008] Japanese Patent Application Publication No. 2002-342899

[0009] [Patent Document 2]

[0010] Japanese Patent Application Publication No. 2005-352645 Summary of the Invention

[0011] [The problem the invention aims to solve]

[0012] However, when multiple messages are communicated to a driver via sound within a short period, the ease of intuitive understanding is a challenge. This application aims to address this challenge by improving recognizability. Furthermore, it contributes to the development of sustainable transportation systems that further enhance traffic safety.

[0013] [Methods used to solve problems]

[0014] One aspect of the present invention is a attention reminder device comprising: an index value acquisition unit that acquires a risk index value representing the collision risk between an object in front of the vehicle and the vehicle based on the outputs of multiple sensors mounted on the vehicle; and a notification control unit that, when detecting an object whose risk index value is above a preset first threshold, outputs a spoken voice to notify the driver of the object, and, during the notification of the object to the driver via the spoken voice, if the risk index value changes above a preset second threshold, notifies the driver of the change in the risk index value via a method other than the spoken voice notification.

[0015] In other aspects of the invention, when the risk indicator value is lower than the first threshold, the notification control unit terminates the voice-based notification and notifies the driver of the vehicle of the change in the risk indicator value by a method other than the voice-based notification.

[0016] In other embodiments of the invention, the notification control unit notifies the driver of the vehicle of a change in the risk indicator value via a display made by a visual HMI device or a vibration made by a tactile HMI device.

[0017] Another aspect of the present invention is a method for alerting attention, in which a processor of an attention alert device mounted in a vehicle performs the following steps: based on the outputs of multiple sensors mounted in the vehicle, obtaining a risk index value representing the collision risk between an object in front of the vehicle and the vehicle; when an object with a risk index value of more than a preset first threshold is detected, outputting a spoken voice to notify the driver of the object; and during the process of notifying the driver of the object through the output of the spoken voice, when the risk index value changes by more than a preset second threshold, notifying the driver of the change in the risk index value through a method other than the spoken voice notification.

[0018] [Invention Effects]

[0019] According to one aspect of the present invention, in scenarios where multiple languages ​​are output within a short period of time, the driver of a vehicle can intuitively identify situations where the level of risk has changed. Attached Figure Description

[0020] Figure 1 This is a diagram showing the structure of the vehicle.

[0021] Figure 2This is an example of a light-up or flashing display.

[0022] Figure 3 This is a flowchart illustrating the operation of the attention reminder device. Detailed Implementation

[0023] [1. Vehicle Structure]

[0024] Figure 1 This is a diagram showing the structure of vehicle 1.

[0025] The vehicle 1 is equipped with a vehicle sensor 10, an HMI device 30, a vehicle control device 50, and a attention reminder device 100.

[0026] The vehicle sensor 10 includes at least one of one or more cameras, radars, lidars, and sonars distributed around the vehicle body of the vehicle 1, and detects objects present in the vicinity of the vehicle 1. The vehicle sensor 10 outputs the data measured by the camera, radar, lidar, or sonar as sensor data to the attention alert device 100.

[0027] HMI device 30 includes a visual HMI device 310 that outputs visual information, an auditory HMI device 330 that outputs sound as auditory stimulation to the driver, and a tactile HMI device 350 that provides tactile stimulation to the driver.

[0028] The visual HMI device 310 is a head-up display installed on the instrument panel 4 of the vehicle 1. The head-up display notifies the presence of a potential collision object by displaying a warning display on the windshield 3, which serves as the projection surface. The driver, seated in the driver's seat, can see the scene in front of the vehicle 1 and the warning display through the windshield 3.

[0029] In this embodiment, the auditory HMI device 330 is a single loudspeaker. Alternatively, the auditory HMI device 330 may also be a loudspeaker system consisting of multiple loudspeakers.

[0030] The haptic HMI device 350 provides tactile stimulation to the driver by vibrating the driver's seat in vehicle 1. The haptic HMI device 350 includes a vibration motor that generates vibrations within the driver's seat. The vibration motor is driven based on a vibration control signal input from the attention alert device 100 to apply vibration to the seat (not shown). The haptic HMI device 350 can also be an electrically operated seatbelt that provides tactile stimulation to the driver by changing the tension of the seatbelt.

[0031] The vehicle control device 50 detects the operating status of the accelerator pedal, brake pedal, turn indicator lights, steering wheel and other control devices of the vehicle 1, and detects the vehicle speed, acceleration, yaw rate and other motion status of the vehicle 1.

[0032] [2. Note the structure of the reminder device]

[0033] The attention reminder device 100 consists of an ECU and a computer, which include a storage unit 110, a processor 130, and input / output interfaces (not shown).

[0034] Storage unit 110 may include, for example, non-volatile RAM (Read Only Memory), or ROM and RAM (Random Access Memory). Alternatively, storage unit 110 may also have an auxiliary storage device such as SSD (Solid State Drive).

[0035] The storage unit 110 stores the control program 111 executed by the processor 130, the voice data 113 of speech output from the auditory HMI device 330, etc.

[0036] Processor 130 is an arithmetic processing device that includes a CPU (Central Processing Unit) and an MPU (Micro-Processing Unit). Processor 130 can consist of a single processor or multiple processors. Alternatively, processor 130 can be a System-on-a-Chip (SoC) integrated with part or all of the storage unit 110 and other circuitry. Furthermore, processor 130 can be a combination of a CPU that executes programs and a DSP (Digital Signal Processor) that performs specified arithmetic operations. Moreover, all functions of processor 130 can be implemented in hardware or configured using programmable devices.

[0037] The attention reminder device 100 includes an object detection unit 131, a risk indicator value acquisition unit 133, and a notification control unit 135 as functional units. These functional units are implemented by the processor 130 executing the control program 111 stored in the storage unit 110.

[0038] The object detection unit 131 detects objects present around the vehicle 1 based on sensor data input from the vehicle sensor 10. Objects refer to objects that may collide with or come into contact with the vehicle 1, and also to objects whose presence needs to be communicated to the driver of the vehicle 1. Objects detected by the object detection unit 131 include, for example, pedestrians, other vehicles, walls, and utility poles, which are stationary objects. In this embodiment, the object detection unit 131 detects objects present in front of the vehicle 1 in its direction of travel that may come into contact with or collide with the vehicle 1. That is, when the vehicle 1 is moving forward, objects in front of the vehicle 1 are detected; when the vehicle 1 is moving backward, objects behind the vehicle 1 are detected. The object detection unit 131 detects the position, speed, and direction of movement of the detected objects. The object detection unit 131 outputs information indicating the position, speed, and direction of movement of the detected objects to the risk index value acquisition unit 133.

[0039] In addition to using sensors such as the front-facing camera included with the vehicle sensor 10, the object detection unit 131 can also use vehicle-to-vehicle communication devices, GNSS (Global Navigation Satellite) units, V2X (Vehicle-to-Everything) communication devices to detect objects or their relative positions. Furthermore, the object detection unit 131 can also use judgments from a virtual environment provided by a server to detect objects appearing in captured images and their relative positions. The communication targets for VX2 communication include other vehicles, pedestrians, networks, and infrastructure.

[0040] The risk indicator value acquisition unit 133 receives information such as the position, speed, and direction of movement of the object detected by the object detection unit 131. Additionally, it receives images captured by a camera as sensor data. The risk indicator value acquisition unit 133 corresponds to the indicator value acquisition unit.

[0041] For example, the risk index value acquisition unit 133 predicts the movement routes of the object and vehicle 1 based on the position, direction of movement, and speed of the object, the position, direction of movement, and speed of vehicle 1, and the status of traffic lights detected from images captured by the camera. Furthermore, the risk index value acquisition unit 133 calculates the risk index value based on the closest position between vehicle 1 and the object and the distance between vehicle 1 and the object at that position. The risk index value represents the probability of vehicle 1 contacting or colliding with the object. For example, the risk index value is obtained by fractionally dividing the position, direction of movement, speed of movement, and distance between vehicle 1 and the object, and then summing the values ​​obtained by multiplying the fractionalized values ​​by weighting coefficients.

[0042] When predicting the movement route of an object, the risk index value acquisition unit 133 can consider the illumination status of the vehicle's turn signals and brake lights if the object is a vehicle, and the direction the pedestrian's face is facing if the object is a pedestrian. Similarly, when predicting the movement route of vehicle 1, the risk index value acquisition unit 133 can consider the illumination status of vehicle 1's turn signals and brake lights obtained from vehicle sensor 10, and can also consider the guided path if the navigation device (not shown) is providing path guidance.

[0043] In addition to detecting the contact risk between the vehicle 1 and objects actually detected in the surrounding environment, the risk indicator value acquisition unit 133 can also estimate the contact risk with objects that have not yet been detected but may appear (hypothetical vehicles, pedestrians, etc.) based on factors such as visibility at intersections and the number of accidents. The risk indicator value acquisition unit 133 outputs the acquired risk indicator values ​​to the notification control unit 135.

[0044] When the notification control unit 135 detects an object whose risk indicator value exceeds a preset first threshold, it plays speech data 113 and outputs the speech from the auditory HMI device 330. For example, the notification control unit 135 outputs speech such as "There's a wall in front" or "There's someone behind the wall" through the auditory HMI device 330. The notification control unit 135 continues to output speech through the auditory HMI device 330 when the risk indicator value exceeds the preset first threshold. That is, speech is output from the auditory HMI device at regular intervals.

[0045] Next, the control unit 135 is notified to monitor changes in the risk indicator values ​​input from the risk indicator value acquisition unit 133, and to determine whether the risk indicator values ​​have changed above a preset second threshold. The second threshold is a threshold value that is smaller than the first threshold.

[0046] If the risk indicator value changes by more than a pre-set second threshold, the notification control unit 135 will notify the driver of the change in the risk indicator value by a method other than voice notification based on language.

[0047] For example, if the risk indicator value rises or falls above a second threshold, the notification control unit 135 notifies the driver of the change in the risk indicator value via the visual HMI device 310 or the tactile HMI device 350.

[0048] Figure 2 This diagram shows a light display or flashing display 200 displayed on the windshield 3 via a visual HMI device 310.

[0049] For example, when the notification control unit 135 notifies the driver of a change in a risk indicator value via the visual HMI device 310, it displays a bright or flashing indicator 200, or a colored indicator, within a predetermined range near the instrument panel 4 on the passenger side. These displays are referred to as attention alert displays. The predetermined range for displaying attention alert displays is preferably within a range that does not obstruct the driver's field of vision.

[0050] For example, whenever the risk indicator value rises above the second threshold, the control unit 135 is notified to increase the brightness of the light display or the flashing display, or to speed up the flashing interval, or to change the color of the colored display to a bright color or red.

[0051] In addition, whenever the risk indicator value drops below the second threshold, the control unit 135 is notified to reduce the brightness of the light display or the flashing display, or to slow down the flashing interval, or to change the color of the colored display to dark or blue.

[0052] Preferably, the timing of the visual HMI device 310 displaying an attention alert or the tactile HMI device 350 outputting vibration does not overlap with the timing of the auditory HMI device 330 outputting spoken language. This is because if the two notifications overlap, the driver will have difficulty recognizing either output. For example, a microphone can be installed in the vehicle 1, and the timing of the visual HMI device 310 displaying an attention alert or the tactile HMI device 350 outputting vibration can be delayed when spoken language is input from the microphone.

[0053] Furthermore, the timing of displaying a attention alert on the visual HMI device 310 or outputting vibration on the tactile HMI device 350 is more preferably before the auditory HMI device 330 outputs spoken language. Spoken language is output after the driver has recognized a change in the risk indicator value through the attention alert display and vibration, allowing the driver to easily understand the content of the spoken language.

[0054] In addition, when the notification control unit 135 notifies the driver of changes in risk indicator values ​​via the tactile HMI device 350, it causes a change in the vibration of the driver's seat.

[0055] For example, whenever the risk indicator value rises above the second threshold, the control unit 135 is notified to control the haptic HMI device 350 by increasing the vibration of the seat or shortening the vibration interval.

[0056] In addition, whenever the risk indicator value drops below the second threshold, the control unit 135 is notified to control the tactile HMI device 350 by reducing the vibration level of the seat or extending the vibration interval.

[0057] Additionally, when the risk indicator value falls below a first threshold, the notification control unit 135 causes the auditory HMI device 330 to terminate voice-based notifications and notifies the driver of the vehicle of the decrease in the risk indicator value using a method other than voice-based notifications. The notification control unit 135 also notifies the driver of the change in the risk indicator value via the visual HMI device 310 or the tactile HMI device 350.

[0058] For example, when the notification control unit 135 notifies the driver of a change in a risk indicator value via the tactile HMI device 350, it gradually reduces the brightness of the illuminated or flashing indicator and eventually turns it off. Furthermore, the notification control unit 135 changes the color of the tinted display to a darker color and eventually removes the tinted display altogether.

[0059] [3. Note the operation of the reminder device]

[0060] Figure 3 This is a flowchart illustrating the operation of the attention reminder device 100.

[0061] Reference Figure 3 The flowchart shown illustrates the operation of the attention reminder device 100.

[0062] First, when the vehicle's ignition switch is turned on (step S1), the attention reminder device 100 is activated and obtains sensor data from the vehicle sensor 10 (step S2).

[0063] Next, the attention reminder device 100 detects objects around the vehicle 1 based on sensor data (step S3), and detects the position, speed and direction of movement of the detected objects.

[0064] Next, the attention reminder device 100 obtains the risk index value of the detected object (step S4). For example, the attention reminder device 100 fractionates the object's position, direction of travel, speed of movement, and distance between vehicle 1 and the object, and multiplies each fractionated value by a weighting coefficient, and calculates the sum of these values ​​as the risk index value.

[0065] Next, the attention reminder device 100 determines whether there is an object whose obtained risk index value is higher than a preset first threshold (step S5).

[0066] If there is an object with a risk indicator value that is above a pre-set first threshold (step S5 / Yes), the attention reminder device 100 determines whether a voice-based notification action is being performed (step S6).

[0067] If a voice-based notification action is not being performed (step S6 / No), the attention reminder device 100 begins a voice-based notification action (step S7). The attention reminder device 100 plays the sound data stored in the storage unit 110 and outputs the voice through the auditory HMI device 330. Afterward, the attention reminder device 100 determines whether the ignition switch has been turned off (step S10).

[0068] Additionally, when the reminder device 100 determines that it is performing a voice-based notification action (step S6 / Yes), it compares the previous risk indicator value of the same object with the current risk indicator value and determines whether the risk indicator value has changed by more than a second threshold (step S8).

[0069] If the risk indicator value does not change above the second threshold (step S8 / No), the attention reminder device 100 moves to the determination of step S10.

[0070] Additionally, when the risk indicator value changes above the second threshold (step S8 / Yes), the attention reminder device 100 notifies the driver of the change in the risk indicator value via the visual HMI device 310 or the tactile HMI device 350 (step S9). For example, the visual HMI device 310 may illuminate or flash the windshield 3, or display a colored indicator. Alternatively, the tactile HMI device 350 may vibrate the driver's seat. Afterward, it is determined whether the ignition switch is turned off (step S10).

[0071] When the reminder device 100 determines that there is no object with a risk index value above the first threshold (step S5 / No), it determines whether a voice-based notification action is being performed (step S11). That is, it determines whether the following state exists: although there is an object with a risk index value above the first threshold, the risk index value of the object has been reduced to less than the first threshold.

[0072] For example, the attention reminder device 100 may also set a flag indicating the presence of an object performing a voice-based notification action to be on, and determine whether a voice-based notification action is being performed based on whether the flag is on or off. Alternatively, if the attention reminder device 100 determines in step S5 that there is no object with a risk indicator value above a first threshold, it will change the flag to be off.

[0073] While the attention reminder device 100 is performing a voice-based notification action (step S11 / Yes), it causes the auditory HMI device 330 to stop the voice-based notification (step S12). Afterwards, the attention reminder device 100 notifies the risk indicator value of a decrease via a light or flashing display on the visual HMI device 310, or by vibration from the tactile HMI device 350 (step S13).

[0074] When the processing of step S7, step S9, or step S13 is completed, the attention reminder device 100 determines whether the ignition switch has been turned off (step S10). If the ignition switch is not turned off (step S10 / No), the attention reminder device 100 returns to the determination of step S2. Alternatively, if the ignition switch is turned off (step S10 / Yes), the attention reminder device 100 ends the processing flow.

[0075] [4. Other Implementation Methods]

[0076] The above-described implementation is merely one approach, which can be arbitrarily modified and applied.

[0077] In the described embodiment, the case where the visual HMI device 310 is a HUD has been described, but the visual HMI device 310 may also be a display unit such as a light-emitting diode (LED).

[0078] in addition, Figure 1 The structure of each part of the vehicle 1 shown is an example, and the specific installation method is not particularly limited. That is, it is not necessarily necessary to install hardware corresponding to each part separately; of course, it is also possible to configure the vehicle to implement the functions of each part by executing a program through a processor. In addition, in the above embodiment, a part of the function implemented by software can be made into hardware, or a part of the function implemented by hardware can be implemented by software.

[0079] in addition, Figure 3 The steps shown are divided according to the main processing content, and this invention is not limited to the method or name of the processing unit division. Depending on the processing content, it can also be divided into more step units. Alternatively, a single step unit may contain more processes. Furthermore, the order of these steps can be appropriately replaced without affecting the spirit of this invention.

[0080] Furthermore, when implementing the attention reminder method based on the attention reminder device 100 described above using the processor 130, the program executed by the processor 130 can also be configured as a recording medium or a transmission medium for transmitting the program. That is, the control program 111 can also be implemented with the control program 111 recorded on a removable information recording medium. Examples of information recording media include magnetic recording media such as hard disks, optical recording media such as CDs, USB (Universal Serial Bus) memory, SSD (Solid State Drive) and other semiconductor storage devices, but other recording media can also be used.

[0081] [Structure supported by the above embodiments]

[0082] The above implementation supports the following structure.

[0083] (Structure 1)

[0084] A attention reminder device includes: an index value acquisition unit that acquires a risk index value, representing the collision risk between an object in front of the vehicle and the vehicle, based on the outputs of multiple sensors mounted on the vehicle; and a notification control unit that, when detecting an object whose risk index value is higher than or equal to a preset first threshold, outputs a spoken voice to notify the driver of the object; and, during the notification of the object to the driver via the spoken voice, if the risk index value changes by a preset second threshold or higher, notifies the driver of the change in the risk index value via a method other than the spoken voice notification.

[0085] According to the attention reminder device in Structure 1, when an object with a risk indicator value exceeding a first threshold is detected, the presence of the object can be communicated to the driver via voice. Furthermore, when the risk indicator value changes by a second threshold or more, the driver is notified of the change in the risk indicator value using a method other than voice notification. Therefore, in scenarios where multiple voice prompts are output within a short period, the driver can intuitively understand when the risk indicator value has changed.

[0086] (Structure 2)

[0087] According to the attention reminder device of structure 1, when the risk index value is lower than the first threshold, the notification control unit terminates the notification based on the language voice and notifies the driver of the vehicle that the risk index value has changed by a method other than the notification based on the language voice.

[0088] According to the attention reminder device in structure 2, when the risk indicator value falls below a first threshold, the voice-based notification ends, and the driver is notified of the change in the risk indicator value through a method other than voice-based notification. Therefore, the driver can intuitively recognize that the risk indicator value has changed.

[0089] (Structure 3)

[0090] According to the attention reminder device of structure 1 or 2, the notification control unit notifies the driver of the vehicle that the risk indicator value has changed through a display made by a visual HMI device or a vibration made by a tactile HMI device.

[0091] According to the attention reminder device in structure 3, a change in risk indicator value is notified through a display by a visual HMI device or a vibration by a tactile HMI device. Therefore, the driver of the vehicle can intuitively recognize that a change in risk indicator value has occurred.

[0092] (Structure 4)

[0093] A method for alerting attention includes having a processor of an attention alert device mounted in a vehicle perform the following steps: acquiring a risk index value representing the collision risk between an object in front of the vehicle and the vehicle based on the outputs of multiple sensors mounted in the vehicle; when an object with a risk index value of more than or equal to a preset first threshold is detected, notifying the driver of the object by outputting a spoken voice; and when, during the notification of the object to the driver by outputting the spoken voice, the risk index value changes by more than or equal to a preset second threshold, notifying the driver of the change in the risk index value by a method other than the notification based on the spoken voice.

[0094] According to the attention reminder method in Structure 4, when an object with a risk indicator value exceeding a first threshold is detected, the presence of the object can be notified to the driver via voice. Furthermore, when the risk indicator value changes by a second threshold or more, the driver is notified of the change in the risk indicator value using a method other than voice notification. Therefore, in scenarios where multiple voice prompts are output within a short period, the driver can intuitively understand when the risk indicator value has changed.

[0095] Explanation of reference numerals in the attached figures

[0096] 1…Vehicle, 3…Windshield, 4…Instrument panel, 10…Vehicle sensors, 30…HMI device, 40…Vibration unit, 50…Vehicle control unit, 100…Attention reminder device, 110…Storage unit, 111…Control program, 130…Processor, 131…Object detection unit, 133…Risk indicator value acquisition unit, 135…Notification control unit, 200…Light display or flashing display, 310…Visual HMI device, 330…Audiovisual HMI device, 350…Haptic HMI device.

Claims

1. A attention reminder device, the attention reminder device comprising: The indicator value acquisition unit acquires risk indicator values, representing the collision risk between the vehicle and an object located in front of it, based on the outputs of multiple sensors mounted on the vehicle; and The notification control unit, upon detecting an object whose risk indicator value exceeds a preset first threshold, outputs a voice message to notify the driver of the object. During the process of notifying the driver of the object via the voice message, if the risk indicator value changes by more than a preset second threshold, the driver is notified of the change in the risk indicator value via a method other than the voice message notification.

2. The attention reminder device according to claim 1, wherein, When the risk indicator value is lower than the first threshold, the notification control unit terminates the voice-based notification and notifies the driver of the vehicle of the change in the risk indicator value through a method other than the voice-based notification.

3. The attention reminder device according to claim 1 or 2, wherein, The notification control unit notifies the driver of the vehicle that the risk indicator value has changed through a display made by a visual human-machine interface device or through vibration made by a tactile human-machine interface device.

4. A method for reminding attention, wherein a processor of a attention reminder device mounted in a vehicle performs the following steps: Based on the outputs of multiple sensors mounted on the vehicle, a risk index value representing the collision risk between the vehicle and an object in front of it is obtained; If an object is detected whose risk index value is above a preset first threshold, the driver of the vehicle is notified of the object by outputting a spoken voice. as well as During the process of notifying the driver of the vehicle about the object through the output of the language, when the risk indicator value changes above a preset second threshold, the driver of the vehicle is notified of the change in the risk indicator value by a method other than notification based on the language.

Images