Warning device and warning method

Abstract

When multiple objects that are the subject of notification are detected, the presence of these multiple objects is notified to the driver, and the driver is made aware of the object that requires the most attention. [Solution] The warning device (100) includes a notification determination unit (135) that determines the type of notification to be notified to the driver of the vehicle (1) by the HMI device and the notification level for each type of notification based on the risk index value when an object with a risk index value within a set range is detected, and a notification execution unit (137) that causes the driver of the vehicle (1) to make a notification. The notification determination unit (135) determines the notification level for an object with a low risk index value such that, when multiple objects with risk index values ​​within the set range are detected, the notification level for an object with a low risk index value is lower than the notification level for at least one type of notification used to notify an object with a high risk index value.

Description

Technical Field

[0001] The present invention relates to an alert device and an alert method.

Background Art

[0002] In recent years, a system that notifies a vehicle driver of risks by a plurality of HMI (Human Machine Interface) devices has been known.

[0003] For example, Patent Document 1 discloses a vehicle peripheral monitoring system including a first device that provides information on the direction of an approaching object at least by voice output, and a second device that provides information on the direction of an approaching object at least by display.

[0004] Further, Patent Document 2 discloses a display and instruction method that displays a danger warning and outputs a voice warning of danger.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] By the way, when there are a plurality of objects for which it is desired to notify the vehicle driver of their presence, there is a problem that it is difficult to notify the driver which object the notification is for. The present application is for solving the above problems and aims to improve visual recognition. And, by extension, it contributes to the development of a sustainable transportation system by further improving traffic safety.

Means for Solving the Problems

[0007] One aspect of the present invention is a warning device comprising: an index value acquisition unit that acquires a risk index value indicating the risk of collision with an object in front of the vehicle based on the output of a plurality of sensors mounted on the vehicle; a determination unit that determines, based on the risk index value, the type of notification to be notified to the driver of the vehicle by an HMI device and the level of notification for each type of notification when an object within a preset range of the acquired risk index value is detected; and a notification execution unit that causes the HMI device corresponding to the type of notification determined by the determination unit to notify the driver of the vehicle at the level of notification determined by the determination unit, wherein the determination unit determines the level of notification for an object with a low risk index value such that, when a plurality of objects with risk index values ​​within the preset range are detected, the level of notification corresponding to at least one type of notification used for notifying an object with a low risk index value is lower than the level of notification corresponding to at least one type of notification used for notifying an object with a high risk index value. In another aspect of the present invention, the determination unit detects, among the plurality of objects, a first object whose risk index value is within the set range, and a second object whose risk index value is within the set range and is smaller than that of the first object, and when the risk index value of the first object falls below the set range, it changes the level of any one of the notification types for the first object to a higher or lower level, and then stops the notification of the first object by the HMI device. In another aspect of the present invention, the determination unit detects, among the plurality of objects, a first object whose risk index value is within the set range, and a second object whose risk index value is within the set range and is smaller than that of the first object, and when the risk index value of the first object falls below the set range, it changes the level of the first type of notification used for notifying the first object to an increased level, changes the level of the second type of notification which is different from the first type to a lower level, and then stops the notification of the first object by the HMI device. Another aspect of the present invention includes a visual HMI device, wherein the determination unit determines values ​​of visual parameters that change the image displayed by the visual HMI device from the level of notification determined based on the risk index value, and the visual parameters include parameters that change at least one of the enlargement and reduction of the image, the display area of ​​the image, the display brightness of the image, the display color of the image, the shape of the image, and the flashing period of the image. Another aspect of the present invention includes an auditory HMI device, wherein the determination unit determines the value of auditory parameters that modify the sound output by the auditory HMI device from the level of notification determined based on the risk index value, and the auditory parameters include parameters that modify at least one of the playback speed or playback cycle, volume, and pitch of the audio data that is the source of the sound. Another aspect of the present invention is that the HMI device includes a haptic visual HMI device, the determination unit determines a value of a tactile parameter that modifies the vibration level of the vibration output by the haptic HMI device from the level of notification determined based on the risk index value, the tactile parameter includes a parameter that modifies at least one of the period of the vibration, the amplitude of the vibration, and the waveform of the vibration. Another aspect of the present invention is a warning method comprising: an acquisition step of a processor of a vehicle-mounted warning device that acquires a risk index value indicating the risk of collision with an object in front of the vehicle based on the output of a plurality of sensors mounted on the vehicle; a determination step of determining, based on the risk index value, the type of notification to be notified to the driver of the vehicle by an HMI device and the level of notification for each type of notification when an object within a preset range of the acquired risk index value is detected; and a notification step of causing the HMI device corresponding to the type of notification determined in the determination step to notify the driver of the vehicle at the level of notification determined in the determination step, wherein the determination step determines the level of notification for an object with a low risk index value such that, when a plurality of objects with a risk index value within the preset range are detected, the level of notification corresponding to at least one type of notification used for notifying an object with a low risk index value is lower than the level of notification corresponding to at least one type of notification used for notifying an object with a high risk index value. [Effects of the Invention]

[0008] According to one aspect of the present invention, when multiple objects that are the subject of notification are detected, the presence of these multiple objects is notified to the driver, and by differentiating the level of notification, the driver can be made to recognize the object that requires the most attention. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 shows the configuration of the vehicle's control system. [Figure 2] Figure 2 shows the layout of the vehicle's interior. [Figure 3] Figure 3 shows an example of the first setting table. [Figure 4] Figure 4 shows an example of a second configuration table. [Figure 5] Figure 5 shows another example of the second configuration table. [Figure 6]FIG. 6 is a diagram showing Output Example 1 of the HMI device. [Figure 7] FIG. 7 is a diagram showing Output Example 2 of the HMI device. [Figure 8] FIG. 8 is a diagram showing Output Example 3 of the HMI device. [Figure 9] FIG. 9 is a diagram showing Output Example 3 of the HMI device. [Figure 10] FIG. 10 is a flowchart showing the operation of the warning device.

Embodiments for Carrying Out the Invention

[0010] [1. Configuration of Vehicle] FIG. 1 is a diagram showing the configuration of Vehicle 1. Vehicle 1 is equipped with a vehicle sensor 10, an HMI device 30, a vehicle control device 50, and a warning device 100.

[0011] The vehicle sensor 10 includes at least one of one or more cameras, radars, lidars, and sonars distributed on the vehicle body of Vehicle 1, and detects objects existing around Vehicle 1. The vehicle sensor 10 outputs data measured by the camera, radar, lidar, or sonar as sensor data to the warning device 100.

[0012] The HMI device 30 includes a visual HMI device 310 that outputs visual information, an auditory HMI device 330 that outputs sound, which is an auditory stimulus to the driver, and a tactile HMI device 350 that gives a stimulus to the driver's touch.

[0013] FIG. 2 is a diagram showing the configuration inside the passenger compartment of Vehicle 1. The X-axis, Y-axis, and Z-axis shown in FIG. 2, FIG. 5, and FIG. 6 described later are orthogonal to each other. The Z-axis indicates the vertical direction. The X-axis and Y-axis are parallel to the horizontal direction in the traveling state of Vehicle 1. The X-axis indicates the left-right direction as the vehicle width direction. The Y-axis indicates the front-back direction. The positive direction of the X-axis indicates the right direction. The positive direction of the Y-axis indicates the front direction. The positive direction of the Z-axis indicates the upward direction.

[0014] In this embodiment, the visual HMI device 310 is a head-up display provided on the instrument panel 4 of the vehicle 1. The head-up display notifies the driver of the presence of an object that may collide by displaying an image on the front glass 3 as a projection surface. A driver sitting in the driver's seat can view the image together with the scene in front of the vehicle 1 through the front glass 3.

[0015] In this embodiment, the auditory HMI device 330 is a speaker. FIG. 2 shows a speaker 330A provided on the driver's seat side of the vehicle 1 and a speaker 330B provided on the passenger seat side as the auditory HMI device 330.

[0016] In this embodiment, the tactile HMI device 350 is a device that vibrates the seat of the driver's seat in the vehicle 1. The tactile HMI device 350 includes, for example, a vibration motor that generates vibration within the seat on which the driver is seated, and drives the vibration motor based on a vibration control signal input from the attention - calling device 100. Thereby, a tactile stimulus is given to the driver. The tactile HMI device 350 may be an electric seat belt that changes the tension of the seat belt to give a tactile stimulus to the driver.

[0017] [2. Configuration of the Attention - calling Device] The attention - calling device 100 is composed of an ECU or a computer including a storage unit 110, a processor 130, an input / output interface (not shown), etc.

[0018] The storage unit 110 includes, for example, a non - volatile RAM (Read Only Memory), or a configuration including a ROM and a RAM (Random Access Memory). The storage unit 110 may also be configured to include an auxiliary storage device such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive).

[0019] The storage unit 110 stores a control program 111 executed by the processor 130, a first setting table 113, a second setting table 115, and calibration data 117.

[0020] The first setting table 113 and the second setting table 115 are used by the warning device 100 to determine the type of HMI device 30 to be used for notification and the level of notification to be given by the HMI device 30 when notifying the driver of the presence of an object. Details of the first setting table 113 and the second setting table 115 will be described later.

[0021] Calibration data 117 is conversion data that converts the position of an object obtained by image analysis of the camera image captured by the vehicle sensor 10 into the display position when it is displayed on the windshield 3 by the head-up display. Calibration data 117 is registered in the storage unit 110 in advance when the camera is installed.

[0022] The processor 130 is an arithmetic processing unit equipped with a processor such as a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit). The processor 130 may be composed of a single processor or multiple processors. Furthermore, the processor 130 may be composed of part or all of the memory unit 110 or an SoC integrated with other circuits. Alternatively, the processor 130 may be composed of a combination of a CPU that executes programs and a DSP (Digital Signal Processor) that performs predetermined arithmetic processing. In addition, the processor 130 may be configured with all its functions implemented in hardware, or it may be configured using programmable devices.

[0023] The warning device 100 comprises, as functional units, an object detection unit 131, a risk indicator value acquisition unit 133, a notification decision unit 135, and a notification execution unit 137. These functional units are realized by the processor 130 executing a control program 111 stored in the storage unit 110. The risk indicator value acquisition unit 133 corresponds to the indicator value acquisition unit, and the notification decision unit 135 corresponds to the decision unit.

[0024] The object detection unit 131 detects objects present around the vehicle 1 based on sensor data input from the vehicle sensor 10. An object is an object that may collide with or come into contact with the vehicle 1, and an object whose presence needs to be notified to the driver of the vehicle 1. Objects detected by the object detection unit 131 include, for example, pedestrians, other vehicles, walls, utility poles, and other stationary objects. In this embodiment, the object detection unit 131 detects objects that are located in front of the vehicle 1 in the direction of travel and that may come into contact with or collide with the vehicle 1. That is, when the vehicle 1 is traveling forward, it detects objects located in front of the vehicle 1, and when the vehicle 1 is traveling backward, it detects objects located behind the vehicle 1. The object detection unit 131 detects the position, speed, and direction of movement of the detected object. The object detection unit 131 outputs information indicating the position, speed, and direction of movement of the detected object to the risk index value acquisition unit 133. The object detection unit 131 may detect objects and their relative positions using sensors such as the front camera provided by the vehicle sensor 10, as well as vehicle-to-vehicle communication devices, GNSS (Global Navigation Satellite Systems) units, V2X (Vehicle-to-Everything) communication, etc. Furthermore, the object detection unit 131 may detect objects in the captured image and their relative positions using methods such as judgment in a virtual environment via a server. Communication partners for VX2 communication include other vehicles, pedestrians, networks, infrastructure, etc.

[0025] The risk indicator value acquisition unit 133 receives information such as the position, speed, and direction of movement of an object detected by the object detection unit 131. The risk indicator value acquisition unit 133 also receives sensor data, such as images captured by a camera.

[0026] For example, the risk index acquisition unit 133 predicts the movement path between the object and vehicle 1 based on the object's position, direction of movement, and speed, as well as the position, direction of movement, and speed of vehicle 1, and the status of traffic lights detected from the camera's captured image. The risk index acquisition unit 133 then calculates a risk index value based on the position where vehicle 1 and the object are closest, and the distance between vehicle 1 and the object at that position. The risk index value is a value that indicates the possibility of contact or collision between vehicle 1 and the object. The risk index value is calculated, for example, by assigning points to the object's position, direction of movement, speed, and distance between vehicle 1 and the object, and then summing the values ​​obtained by multiplying these points by a weighting coefficient.

[0027] When predicting the movement of an object, the risk indicator value acquisition unit 133 may consider the illumination status of the vehicle's turn signals and brake lights if the object is a vehicle, and may consider the direction of the pedestrian's face if the object is a pedestrian. Similarly, when predicting the movement of vehicle 1, the risk indicator value acquisition unit 133 may consider the illumination status of vehicle 1's turn signals and brake lights obtained from the vehicle sensor 10, and may consider the route being guided if a navigation device (not shown) is providing route guidance.

[0028] Furthermore, the risk indicator value acquisition unit 133 may detect the risk of contact between the vehicle 1 and objects actually detected in the surrounding environment of the vehicle 1, as well as estimate and detect the risk of contact with objects that have not yet been detected but may appear (such as virtual vehicles or pedestrians) based on factors such as the visibility at intersections and the number of accidents. The risk indicator value acquisition unit 133 outputs the risk indicator value for each object to the notification decision unit 135.

[0029] The notification decision unit 135 receives risk index values ​​for each object from the risk index value acquisition unit 133. Based on the input risk index values, the notification decision unit 135 determines whether or not there are objects that require notification. If there are objects that require notification, the notification decision unit 135 determines the type of notification to the driver and the level of the notification. The type of notification indicates the type of HMI device 30 used to notify the driver. The level of the notification is, for example, the size and brightness of the figures displayed by the visual HMI device 310, the volume of the sound output by the auditory HMI device 330, or the level of vibration output by the tactile HMI device 350.

[0030] First, the notification decision unit 135 compares the risk indicator value input from the risk indicator value acquisition unit 133 with a preset first threshold value to detect objects whose risk indicator value is equal to or greater than the first threshold value. Objects whose risk indicator value is equal to or greater than the first threshold value correspond to objects whose risk indicator value is within the preset range. Furthermore, when the notification decision unit 135 detects an object whose risk indicator value is equal to or greater than the first threshold, it determines whether or not there is only one object detected.

[0031] [3. When one object is detected] First, let's explain the case where there is only one object to be notified. The notification determination unit 135 determines the type of notification and the level of notification based on the risk indicator value of the detected object if there is only one object whose risk indicator value is equal to or greater than a preset first threshold. For example, the range of risk indicator values ​​equal to or greater than the first threshold is divided into multiple ranges such as the first range, second range, third range, fourth range, and fifth range. The notification determination unit 135 may determine the type of notification and the level of notification by determining which range the detected risk indicator value equal to or greater than the first threshold falls into. The first range is the range between the first threshold and the second threshold. The second range is the range between the second threshold and the third threshold. The third range is the range between the third threshold and the fourth threshold. The fourth range is the range between the fourth threshold and the fifth threshold. The fifth range is the range that matches the fifth threshold.

[0032] [4. When multiple objects are detected] Next, we will explain the case where there are multiple objects subject to notification. If the notification determination unit 135 detects multiple objects whose risk indicator value is equal to or greater than the first threshold, it determines the type of notification and the level of notification for each of these multiple objects. First, the notification decision unit 135 identifies the object with the highest risk indicator value among the detected objects. This object is called the first object. The notification decision unit 135 then determines the type of notification and the level of notification for the identified first object. The method for determining the type of notification and the level of notification for the first object is the same as when a single object is detected, as described above.

[0033] Next, the notification decision unit 135 identifies the object with the second highest risk index value among the detected objects, after the first object. This object is called the second object. The notification decision unit 135 determines the type of notification and the level of notification for the identified second object. At this time, the notification decision unit 135 determines the type of notification and the level of notification for the presence of the second object so that the level of notification for the presence of the second object is lower than the level of notification for the presence of the first object.

[0034] For example, the notification determination unit 135 determines the types and number of notifications for the second object so that the number of types of notifications used to notify the presence of the second object is less than the number of types used to notify the presence of the first object. For example, the visual HMI device 310 and the auditory HMI device 330 may be used to notify the presence of the first object, and the auditory HMI device 330 may be used to notify the presence of the second object. In this case, the driver may be made to recognize the first object by synchronizing the HMI devices used to notify the first object. For example, the flashing of a figure displayed on the windshield 3 by the visual HMI device 310 may be synchronized with the output timing of an electronic sound output by the auditory HMI device 330, or the flashing period of the figure displayed by the visual HMI device 310 may be synchronized with the period of vibration output by the tactile HMI device 350. More specifically, the timing of the flashing of the shapes displayed by the visual HMI device 310 and the timing of the vibration of the seat by the tactile HMI device 350 may be synchronized.

[0035] Furthermore, the notification determination unit 135 determines the notification level of the second object so that the notification level of the second object is lower than the notification level of the first object. The notification determination unit 135 determines the values ​​of the parameters that define the notification level of the HMI device 30. At this time, the notification determination unit 135 sets the parameters such that the parameter defining the notification level of the first object is higher than the parameter defining the notification level of the second object.

[0036] Next, with reference to Figures 3 and 4, the operation of the notification determination unit 135 in determining the type of notification and the level of notification will be explained. First, let's explain Figures 3 and 4. Figure 3 shows an example of the first setting table 113. The first setting table 113 registers the parameters included in each of the visual, auditory, and tactile HMI devices 30, along with the settings for each notification level of each parameter.

[0037] Parameters for changing the state of the graphics displayed by the visual HMI device 310 include, for example, display size, size change range, display color, graphic shape, and blinking cycle. These parameters correspond to visual parameters. The first setting table 113 registers the settings for each of these parameters for each notification level. The notification levels include high notification level, medium notification level, and low notification level. Hereinafter, the high notification level will be referred to as high level, the medium notification level as medium level, and the low notification level as low level.

[0038] If the parameter is display size, the high-level setting will register a display size of "large," the medium-level setting will register a display size of "medium," and the low-level setting will register a display size of "small." When the parameter is the width of the change size, "Large" is registered for high-level settings, "Medium" for medium-level settings, and "Small" for low-level settings. The width of the change size is the range by which the display size of the figures displayed by the visual HMI device 310 is enlarged and reduced. For example, if the width of the change size is "Large," the size of the figures displayed by the visual HMI device 310 will be significantly changed from "Large" to "Small," and from "Small" to "Large." By significantly changing the display size of the figures displayed by the visual HMI device 310 in this way, the driver's attention is attracted. If the parameter is luminance, the high-level setting will register "bright," the medium-level setting will register "medium," and the low-level setting will register "dark." If the parameter is a display color, then "red" will be registered as the display color for high-level settings, "yellow" for medium-level settings, and "blue" for low-level settings. When the parameter is a geometric shape, "sharp" is registered as the geometric shape for high-level settings, "angular" is registered as the geometric shape for medium-level settings, and "rounded" is registered as the geometric shape for low-level settings. The sharper the geometric shape displayed by the visual HMI device 310, the higher the perceived risk, and the rounder the geometric shape, the lower the perceived risk. If the parameter is the blinking period, "fast" will be registered as the blinking period for high-level settings, "medium" for medium-level settings, and "slow" for low-level settings.

[0039] Parameters that change the state of the sound output by the auditory HMI device 330 include, for example, electronic sound or spoken voice, playback speed, volume, and pitch. These parameters correspond to auditory parameters. The first setting table 113 registers the settings for each of these parameters at different notification levels.

[0040] If the parameter is either an electronic sound or a spoken voice, "Spoken voice" will be registered for high-level settings, "Spoken voice" for medium-level settings, and "Electronic sound" for low-level settings. If the parameter is playback speed, "fast" will be registered as the playback speed for high-level settings, "medium" for medium-level settings, and "slow" for low-level settings. If the parameter is volume, "High" will be registered as the volume for high-level settings, "Medium" for medium-level settings, and "Low" for low-level settings. If the parameter is voice pitch (tone), a high level setting will register "high" as the voice pitch, and a medium level setting will register "medium" as the voice pitch. In the case of a low level setting, an electronic sound is output, so no voice pitch is registered.

[0041] Parameters that change the state of vibration output by the tactile HMI device 350 include, for example, frequency and period, amplitude, and waveform. These parameters correspond to tactile parameters. The first setting table 113 registers the settings for each of these parameters at different notification levels.

[0042] When the parameters are frequency and period, "fast" is registered for high-level settings, "medium" for medium-level settings, and "slow" for low-level settings. If the parameter is amplitude, "Large" is registered for high-level settings, "Medium" for medium-level settings, and "Small" for low-level settings. If the parameter is a waveform, "Rectangle" is registered for high-level settings, "Sawtooth" is registered for medium-level settings, and "Steady" is registered for low-level settings.

[0043] Figure 4 shows an example of the second setting table 115A. The second setting table 115A is a table that associates the ranges of the categorized risk indicator values ​​with the types of notifications and notification levels within each range. In Figure 4, the risk indicator value is denoted as the risk indicator value R. The columns in the second setting table 115A register the range of the risk indicator value R. Figure 4 shows an example where the risk indicator value R is divided into six ranges: "R=10", "8≦R<10", "6≦R<8", "4≦R<6", "2≦R<4", and "0≦R<2".

[0044] The rows in the second setting table 115A register the visual, auditory, and tactile output settings. The output settings include the type and level of notifications used to notify the driver. The notification type settings include visual output, auditory output, and tactile output. Visual output is a notification using graphics displayed by the visual HMI device 310, auditory output is a notification using sound output from the auditory HMI device 330, and tactile output is a notification using seat vibrations generated by the tactile HMI device 350. The notification level settings include three levels: "high," "medium," and "low." The "high" notification level corresponds to the "high level" shown in Figure 3, the "medium" notification level corresponds to the "medium level" shown in Figure 3, and the "low" notification level corresponds to the "low level" shown in Figure 3.

[0045] In the example of the second setting table 115A shown in Figure 4, when the risk index value R is "10", visual output, auditory output, and tactile output are selected as the outputs used for notification, and the notification level is set to "High" for all of them. If the risk index value R is "8 ≤ R < 10", visual, auditory, and tactile outputs are selected as the outputs used for notification, the notification level for visual output is set to "high", and the notification levels for auditory and tactile output are set to "medium". If the risk index value R is "6 ≤ R < 8", visual, auditory, and tactile outputs are selected as the outputs used for notification, the notification level for visual output is set to "high", the notification level for auditory output is set to "medium", and the notification level for visual output is set to "low". If the risk index value R is "4 ≤ R < 6", visual and auditory output are selected as the output used for notification, the notification level for visual output is set to "medium", and the notification level for auditory output is set to "low". Haptic output is not selected as the output used for notification. If the risk index value R is "2 ≤ R < 4", visual output is selected as the output used for notification, and the level of visual output notification is set to "low". Auditory and tactile output are not selected as outputs used for notification.

[0046] For example, suppose two objects, a first object and a second object, are detected as objects to be notified. In this case, the notification determination unit 135 refers to the second setting table 115A shown in Figure 4 and determines the notification level for the second object such that the notification level of at least one type of notification used for notifying the second object, which has a low risk index value R, is lower than the notification level of at least one type of notification used for notifying the first object, which has a high risk index value R. For example, suppose the risk index value R for the first object is "10" and the risk index value R for the second object is "8 ≤ R < 10". In this case, the level of auditory output notification for the first object is set to "high", and the level of auditory output notification for the second object is set to "medium". Furthermore, assume that the risk index value R for the first object is "6 ≤ R < 8" and the risk index value R for the second object is "4 ≤ R < 6". In this case, the level of visual output notification for the first object is set to "high", and the level of visual output notification for the second object is set to "medium". Also, the level of auditory output notification for the first object is set to "medium", and the level of auditory output notification for the second object is set to "low".

[0047] The notification determination unit 135 determines the type of notification and the notification level by referring to the second setting table 115A shown in Figure 4. Next, it refers to the first setting table 113 shown in Figure 3 to obtain the parameter values ​​corresponding to the determined type of notification and the notification level. The notification determination unit 135 outputs the determined type of notification and the obtained parameter values ​​to the notification execution unit 137. For example, if the risk index value R is "8 ≤ R < 10", then based on the second setting table 115A shown in Figure 4, the types of notifications are determined to be a visual HMI device 310 and an auditory HMI device 330, and the notification levels are determined to be "high" for the visual HMI device 310 and "medium" for the auditory HMI device 330. Next, since the visual HMI device 310 is at the "high" level, the display size "large", resizing range "large", brightness "bright", display color "red", graphic shape "sharp", and blinking cycle "fast" are obtained from the first setting table 113 shown in Figure 3. The notification determination unit 135 outputs the type of notification and the acquired parameter values ​​to the notification execution unit 137.

[0048] The notification execution unit 137 receives the notification type and parameters from the notification determination unit 135. The notification execution unit 137 operates the HMI device 30 corresponding to the input notification type according to the parameters. At this time, if the notification type includes a visual output, the notification execution unit 137 uses the calibration data 117 to convert the coordinates of the object detected by the object detection unit 131 in the captured image to coordinates on the windshield 3. The notification execution unit 137 displays a figure corresponding to the notification level on the converted coordinates of the windshield 3.

[0049] Figure 5 shows another example of the second setting table 115. When setting the notification type and notification level using the first setting table 113 shown in Figure 3 and the second setting table 115A shown in Figure 4, for example, if the notification level for visual output is set to "High", then all parameters of the visual HMI device 310, such as "Display Size", "Size Change Width", "Brightness", "Display Color", "Graphic Shape", and "Blinking Period", will be set to the "High" level.

[0050] Another second setting table 115B shown in Figure 5 is an example in which the values ​​of the parameters of the visual HMI device 310, namely "display size," "size change range," "brightness," "display color," "graphic shape," and "flashing cycle," are changed to different values ​​depending on the range in which the risk index value R is included. In the example of another settings table shown in Figure 5, when the risk index value R is "10", all parameters of "Display Size", "Resize Range", "Brightness", "Display Color", "Shape", and "Blinking Period" are set to "High". Furthermore, if the range containing the risk index value R is "8 ≤ R < 10", the parameters for "Brightness" and "Shape" are set to "High", and the parameters for "Display Size", "Resizing Range", "Display Color", and "Blinking Period" are set to "Medium". Furthermore, if the range containing the risk index value R is "6 ≤ R < 8", the parameters for "Display Size", "Brightness", "Display Color", and "Shape" are set to "Medium", and the parameters for "Resize Range" and "Blinking Period" are set to "Low". Furthermore, if the risk index value R is "4 ≤ R < 6", the parameters for "Brightness" and "Shape" are set to "Medium", and the parameters for "Display Size" and "Display Color" are set to "Low". In addition, "Size Change Range" and "Blinking Period" are set to not used. In other words, if the risk index value R is "4 ≤ R < 6", the size of the shapes displayed by the visual HMI device 310 remains constant and does not blink. If the risk index value R is "2 ≤ R < 4", the parameters for "Display Size", "Brightness", "Display Color", and "Shape" are set to "Low". In addition, "Resize Range" and "Blinking Period" are set to "Disabled".

[0051] Although not shown in the diagram, the parameters of the auditory HMI device 330, such as "electronic sound or spoken voice," "playback speed," "volume," and "voice pitch," may be changed to different values ​​depending on the range in which the risk index value R is included, similar to the case of the visual HMI device 310. Similarly, the tactile HMI device 350 may also change the parameters of the tactile HMI device 350, namely "frequency and period," "amplitude," and "waveform," to different values ​​depending on the range in which the risk index value R is included.

[0052] Furthermore, if the risk index value R is a high value, such as "10" or "8 ≤ R < 10", the level of notification for the parameters included in the visual output may be matched with the level of notification for the parameters included in the auditory or tactile output. For example, by matching the flashing cycle of the figures displayed by the visual HMI device 310 and the playback speed of the audio output by the auditory HMI device 330 to a "high" level, the driver's attention is drawn to the objects notified by the visual HMI device 310 and the auditory HMI device 330. Furthermore, by matching the flashing cycle of the figures displayed by the visual HMI device 310 and the amplitude of the seat vibration output by the tactile HMI device 350 to a "high" level, the driver's attention is drawn to the objects notified by the visual HMI device 310 and the tactile HMI device 350.

[0053] The notification determination unit 135, after obtaining the notification level for each parameter of "display size," "size change width," "brightness," "display color," "graphic shape," and "blinking cycle" by referring to the other setting tables shown in Figure 5, then obtains the parameter values ​​corresponding to the notification level by referring to the first setting table 113 shown in Figure 3.

[0054] Subsequently, the notification determination unit 135 outputs the determined type of notification and the acquired parameter values ​​to the notification execution unit 137.

[0055] [5. Notification Example 1] Figure 6 shows a first example of operation of the HMI device 30. This first example of operation shows a case where two people, person P1 and person P2, are detected as objects. Person P1 is closer to vehicle 1 than person P2, but person P1 is on the sidewalk, not the roadway, and is stationary. Person P2 is further away from vehicle 1 than person P1, but person P2 is on the roadway and is crossing the roadway. Based on these circumstances, the notification decision unit 135 sets person P2 as the first object and person P1 as the second object.

[0056] The warning device 100 displays figures 200A and 200B around the first target object, person P2. Figure 200A is a figure displayed at the feet of person P2, and figure 200B is a figure surrounding person P2. The warning device 100 also displays figure 200C at the feet of person P1. When the warning device 100 detects people P1 and person P2 from the camera's captured image, it uses calibration data 117 to convert the positions of people P1 and person P2 in the captured image to their positions on the windshield 3. The warning device 100 displays figures 200A, 200B, and 200C at the converted positions of people P1 and person P2 on the windshield 3.

[0057] The shape 200A displayed at the feet of the first target object, person P2, is made sharp (high notification level), while the shape 200C displayed at the feet of the second target object, person P1, is made rounded (low notification level). In addition, the shape 200A displayed at the feet of the first target object, person P2, flashes rapidly (high notification level), while the shape 200C displayed at the feet of the second target object, person P1, does not flash. This makes the driver focus more on person P2 than on person P1. Furthermore, the auditory HMI device 330 outputs a voice message notifying the driver that there is a person on the road. For example, it outputs a voice message such as, "It's dangerous. There is a person on the road." In addition, by matching the notification level of the playback speed of this voice message (notification level: high) with the notification level of the flashing speed of the figure 200A (notification level: high), the driver can be made to focus on person P2.

[0058] [6. Example of Notification 2] Figure 7 shows a second example of operation of the HMI device 30. This example 2 shows the case where person P3 is detected as the target object. The warning device 100 determines, based on the direction of movement of person P3, that there is a possibility that person P3 may run into the roadway. Therefore, the warning device 100 displays a figure 200D at the feet of person P3 and outputs a warning sound from the auditory HMI device 330. For example, the warning device 100 sets the notification level to "high" and outputs a "beep, beep, beep, beep, beep" electronic sound as a warning notification sound to the auditory HMI device 330. In addition, the warning device 100 sets the notification level to "high" and makes the figure 200D flash in sync with the timing of this electronic sound output. By setting the notification level of the electronic sound output by the auditory HMI device 330 (notification level: high) and the notification level of the flashing speed of the figure 200A displayed by the visual HMI device 310 (notification level: high) to the same notification level, the driver can be made to pay attention to person P3.

[0059] [7. Notification Example 3] Figures 8 and 9 show a third example of operation of the HMI device 30. Figure 8 shows the state in which persons P4 and P5 walking near the roadway are detected as objects. The distance between person P4 and the roadway is shorter than the distance between person P5 and the roadway, person P4 is walking towards the roadway, and person P5 is walking away from the roadway. Therefore, the warning device 100 sets person P4 as the first object and person P5 as the second object and starts issuing a warning.

[0060] Figure 9 shows the changes in the warning output by the warning device 100. Let's assume that time T1 is when the alert device 100 sets people P4 and P5 as its target objects. At time T1, the alert device 100 displays a flashing figure 200E at the feet of person P4 and outputs an electronic sound from the auditory HMI device 330 that is synchronized with the flashing of this figure 200E. That is, the flashing of the figure 200E and the timing of the output of the electronic sound "beep" are synchronized. At this time, the notification level of the electronic sound output by the auditory HMI device 330 and the notification level of the flashing speed of the figure 200E displayed by the visual HMI device 310 may be set to the same notification level to draw the driver's attention to person P4. Furthermore, at time T1, the warning device 100 displays figure 200F at the feet of person P4. Figure 200F does not flash like figure 200E.

[0061] Next, let's assume that at time T2, person P4 makes a U-turn and begins walking away from the road. As person P4 begins walking away from the road, the risk index value R for person P4 decreases, and the risk index value for person P4 becomes lower than the risk index value for person P5. At this point, let's assume that the risk index value R for person P4 falls below the first threshold and is no longer subject to notification.

[0062] When the risk index value R of person P4 is removed from the list of people to be notified, the alert device 100 changes the level of one of the notification types for person P4 to a higher or lower level, and then stops the notifications for person P4 by the HMI devices 30. For example, it increases the display size of the figure 200E displayed by the visual HMI device 310, which is one of the notification types for person P4, the first target object, and then stops all notifications for person P4 by the HMI devices 30. That is, it erases the figure 200E displayed by the visual HMI device 310 and stops the electronic sound output by the auditory HMI device 330. Here, the case of increasing the display size of figure 200E is shown, but the display size of figure 200E may also be decreased.

[0063] Furthermore, if the risk index value R of person P4 is removed from the list of people to be notified, the alert device 100 may change the level of the first type of notification for person P4 to a higher level, and the level of the second type of notification (which is different from the first type) to a lower level, before stopping the notifications for person P4 by the HMI device 30. For example, the electronic sound output by the auditory HMI device 330 may be increased in volume, the display size of the figure 200E displayed by the visual HMI device 310 may be decreased in volume, before the figure 200E displayed by the visual HMI device 310 is erased, and the electronic sound output by the auditory HMI device 330 is stopped.

[0064] Next, let's assume that at time T3, person P5 makes a U-turn and begins walking toward the roadway. As person P5 begins walking toward the roadway, the risk index value R for person P5 increases, and the risk index value for person P5 becomes higher than that of person P4. At this time, the alert device 100 changes the notification level for person P5 to a higher level than the notification level at time T2. As a result, for example, the display size of the figure 200F displayed at person P5's feet becomes larger than it was at time T2. Then, as person P5 approaches the roadway and the risk index value for person P5 increases, the alert device 100 also starts outputting an electronic sound in addition to displaying the figure 200F.

[0065] [8. Operation of the warning device] Figure 10 is a flowchart showing the operation of the warning device 100. The operation of the warning device 100 will be explained with reference to the flowchart shown in Figure 10. First, the warning device 100 is activated when the vehicle's IG switch is turned on (step S1) and acquires sensor data from the vehicle sensor 10 (step S2).

[0066] Next, the warning device 100 detects objects present around the vehicle 1 based on sensor data (step S3), and detects the position, speed, and direction of movement of the detected objects.

[0067] Next, the warning device 100 acquires the risk index value of the detected object (step S4). The warning device 100 scores the object's position, direction of travel, speed, and distance between vehicle 1 and the object, multiplies each of these scores by a weighting coefficient, and calculates the sum of these values ​​as the risk index value.

[0068] Next, the alert device 100 determines whether the acquired risk indicator value is equal to or greater than a preset first threshold (step S5). If the risk indicator value is not equal to or greater than the first threshold (step S5 / NO), the alert device 100 returns to the process in step S2.

[0069] Next, if the risk indicator value is equal to or greater than the first threshold (step S5 / YES), the warning device 100 determines whether it has detected multiple objects with a risk indicator value equal to or greater than the first threshold (step S6).

[0070] If the number of detected objects is not multiple (step S6 / NO), the alert device 100 determines the type and level of notification based on the risk indicator value of the detected objects (step S7).

[0071] Furthermore, if multiple objects are detected (step S6 / YES), the alerting device 100 first determines the type and level of notification for the first object (step S8). The first object is the object with the highest risk indicator value among the detected objects.

[0072] Next, the alerting device 100 determines the type and level of notification for the second object (step S9). The second object is the object with the next highest risk indicator value after the first object. The alerting device 100 determines the type and level of notification for the second object so that the notification level for the second object is lower than the notification level for the first object set in step S7 (step S9).

[0073] Next, the alerting device 100 determines whether or not there are other objects besides the first object and the second object in the object detected in step 5 (step S10).

[0074] If there are other objects besides the first and second objects (step S10 / YES), the alerting device 100 determines the type and level of notification for the other objects so that the notification level for the other objects is lower than the notification levels for the first and second objects set in steps S7 and S8 (step S11). Furthermore, if there are multiple objects as other objects, the alerting device 100 determines the type and level of notification for the other objects so that they are lower than the notification levels for the objects already determined, similar to steps S7 and S8.

[0075] If there are no other objects besides the first and second objects (step S10 / NO), and if the type and level of notification for other objects are set in step S11, the alert device 100 instructs the HMI device 30 corresponding to the type of notification to set the level of notification (step S12).

[0076] Next, the alert device 100 activates the HMI device 30 at the determined notification level to notify the driver of the presence of the object (step S13).

[0077] Next, the warning device 100 determines whether the ignition key of vehicle 1 has been turned off (step S14). If the ignition key of vehicle 1 is not turned off (step S14 / NO), the warning device 100 returns to the process in step S2. Also, if the ignition key of vehicle 1 is turned off (step S14 / YES), the warning device 100 terminates this process flow.

[0078] [9. Other Embodiments] The embodiments described above are merely examples and can be modified and applied as needed.

[0079] In the embodiment described above, the case where the visual HMI device 310 is a HUD was explained, but the visual HMI device 310 may also be a display means such as an LED (Light Emitting Diode).

[0080] Furthermore, the configuration of each part of Vehicle 1 shown in Figure 1 is merely an example, and the specific implementation is not particularly limited. In other words, it is not necessarily required that hardware corresponding to each part be implemented individually; it is certainly possible to configure the system so that a single processor executes a program to realize the functions of each part. Also, in the embodiments described above, some of the functions realized by software may be implemented as hardware, or conversely, some of the functions realized by hardware may be implemented as software.

[0081] Furthermore, the operation steps shown in Figure 10 are divided according to the main processing content, and the present invention is not limited by the way the processing units are divided or their names. Depending on the processing content, it may be further divided into more steps. Alternatively, it may be divided so that one step unit includes even more processing. Also, the order of the steps may be changed as appropriate, as long as it does not impede the spirit of the present invention.

[0082] Furthermore, when the warning method by the warning device 100 described above is implemented using the processor 130, the program to be executed by the processor 130 can also be configured as a recording medium or a transmission medium for transmitting this program. In other words, the control program 111 can also be implemented by recording it on a portable information recording medium. Examples of information recording media include magnetic recording media such as hard disks, optical recording media such as CDs, and semiconductor storage devices such as USB (Universal Serial Bus) memory and SSDs (Solid State Drives), but other recording media can also be used.

[0083] [10. Configurations supported by the above embodiment] The above embodiment supports the following configuration:

[0084] (Composition 1) An alert device comprising: an index value acquisition unit that acquires a risk index value indicating the risk of collision with an object in front of the vehicle based on the output of a plurality of sensors mounted on the vehicle; a determination unit that determines, based on the risk index value, the type of notification to be notified to the driver of the vehicle by an HMI device and the level of notification for each type of notification when an object is detected within a preset range for the acquired risk index value; and a notification execution unit that causes the HMI device corresponding to the type of notification determined by the determination unit to notify the driver of the vehicle at the level of notification determined by the determination unit, wherein the determination unit determines the level of notification for an object with a low risk index value such that, when a plurality of objects with a risk index value within the preset range are detected, the level of notification corresponding to at least one type of notification used for notifying an object with a low risk index value is lower than the level of notification corresponding to at least one type of notification used for notifying an object with a high risk index value.

[0085] According to the warning device of Configuration 1, when multiple objects with risk indicator values ​​within a set range are detected, the notification level for objects with low risk indicator values ​​is determined such that the notification level corresponding to at least one type of notification used for notifying objects with low risk indicator values ​​is lower than the notification level corresponding to at least one type of notification used for notifying objects with high risk indicator values. Therefore, when multiple objects that require notification are detected, the presence of these multiple objects can be notified to the driver, as well as the first object that requires the most attention.

[0086] (Configuration 2) The determination unit detects, among the plurality of objects, a first object whose risk index value is within the set range, and a second object whose risk index value is within the set range and whose risk index value is smaller than that of the first object, and when the risk index value of the first object falls below the set range, it changes the level of any one of the notification types for the first object to a higher or lower level, and then stops the notification of the first object by the HMI device, as described in Configuration 1.

[0087] According to the alert device of configuration 2, when the risk indicator value of the first object falls below the set range, the level of one of the notification types for the first object is changed to a higher or lower level, and then the notification of the first object by the HMI device is stopped. This allows the driver to pay attention to the first object and recognize that the first object has become an object with a low risk indicator value.

[0088] (Composition 3) The determination unit detects, among the plurality of objects, a first object whose risk index value is within the set range, and a second object whose risk index value is within the set range and is smaller than that of the first object, and when the risk index value of the first object falls below the set range, it changes the level of the first type of notification used for notifying the first object to a higher level, changes the level of the second type of notification which is different from the first type to a lower level, and then stops the notification of the first object by the HMI device, as described in Configuration 1.

[0089] According to the alert device of configuration 3, when the risk indicator value of the first object falls below the set range, the level of the first type of notification used to notify about the first object is changed to a higher level, and the level of the second type of notification, which is different from the first type, is changed to a lower level, after which the notification of the first object by the HMI device is stopped. This allows the driver to pay attention to the first object and recognize that the first object has become an object with a low risk indicator value.

[0090] (Composition 4) The warning device according to Configuration 1, wherein the HMI device includes a visual HMI device, and the determination unit determines the value of a visual parameter that changes the image displayed by the visual HMI device from the level of notification determined based on the risk index value, and the visual parameter includes a parameter that changes at least one of the enlargement and reduction of the image, the display area of ​​the image, the display brightness of the image, the color of the image, the shape of the image, and the flashing period of the image.

[0091] According to the alert device of configuration 4, the value of the visual parameter is determined by the level of notification, which is determined based on the risk indicator value. Therefore, at least one of the following can be changed to correspond to the level of notification: the enlargement and reduction of the image displayed by the visual HMI device, the display area of ​​the image, the display brightness of the image, the color of the image, the shape of the image, and the flashing cycle of the image.

[0092] (Composition 5) The warning device according to Configuration 1, wherein the HMI device includes an auditory HMI device, the determination unit determines the value of an auditory parameter that modifies the sound output by the auditory HMI device from the level of notification determined based on the risk index value, and the auditory parameter includes a parameter that modifies at least one of the playback speed or playback cycle, volume, and pitch of the audio data that is the source of the sound.

[0093] According to the alert device of configuration 5, the auditory parameter value is determined by the notification level, which is determined based on the risk indicator value. Therefore, at least one of the playback speed or playback cycle, volume, and pitch of the audio data that forms the basis of the sound output by the auditory HMI device can be changed to correspond to the notification level.

[0094] (Composition 6) The HDI device includes a haptic visual HMI device, the determination unit determines the value of a tactile parameter that changes the vibration level of the vibration output by the haptic HMI device from the level of notification determined based on the risk index value, and the tactile parameter includes a parameter that changes at least one of the period of the vibration, the amplitude of the vibration, and the waveform of the vibration, according to Configuration 1.

[0095] According to the alert device of configuration 6, the value of the tactile parameter is determined by the level of notification, which is determined based on the risk indicator value. Therefore, at least one of the period, amplitude, and waveform of the vibration output by the tactile HMI device can be changed to correspond to the level of notification.

[0096] (Composition 7) A warning method comprising: an acquisition step of a processor in a vehicle that acquires a risk index value indicating the risk of collision with an object in front of the vehicle based on the output of a plurality of sensors mounted on the vehicle; a determination step of determining, based on the risk index value, the type of notification to be notified to the driver of the vehicle by the HMI device and the level of notification for each type of notification when an object within a preset range is detected with the acquired risk index value; and a notification step of causing the HMI device corresponding to the type of notification determined in the determination step to notify the driver of the vehicle at the level of notification determined in the determination step, wherein the determination step determines the level of notification for an object with a low risk index value such that, when a plurality of objects with a risk index value within the preset range are detected, the level of notification corresponding to at least one type of notification used for notifying an object with a low risk index value is lower than the level of notification corresponding to at least one type of notification used for notifying an object with a high risk index value.

[0097] According to the warning device of configuration 7, when multiple objects with risk indicator values ​​within a set range are detected, the notification level for objects with low risk indicator values ​​is determined such that the notification level corresponding to at least one type of notification used for notifying objects with low risk indicator values ​​is lower than the notification level corresponding to at least one type of notification used for notifying objects with high risk indicator values. Therefore, when multiple objects that require notification are detected, the presence of these multiple objects can be notified to the driver, as well as the first object that requires the most attention. [Explanation of Symbols]

[0098] 1...Vehicle, 3...Windshield, 4...Instrument panel, 10...Vehicle sensor, 30...HMI device, 40...Vibration unit, 50...Vehicle control device, 100...Warning device, 110...Memory unit, 111...Control program, 113...First setting table, 115A, 115B...Second setting table, 117...Calibration data, 130...Processor, 131...Object detection unit, 133...Risk indicator value acquisition unit, 135...Notification decision unit, 137...Notification execution unit, 200C...Human figure, 310...Visual HMI device, 330A...Speaker, 330B...Speaker, 330...Auditory HMI device, 350...Tactile HMI device.

Claims

1. An index value acquisition unit that acquires a risk index value indicating the risk of collision with an object located in front of the vehicle based on the output of multiple sensors mounted on the vehicle, When the acquired risk indicator value detects an object within a preset range, a determination unit determines, based on the risk indicator value, the type of notification to be sent to the vehicle driver via the HMI device, and the level of notification for each type of notification. A notification execution unit causes the HMI device corresponding to the type of notification determined by the determination unit to notify the driver of the vehicle at the level of notification determined by the determination unit, Equipped with, The determination unit determines the notification level for objects with low risk indicator values ​​when multiple objects with risk indicator values ​​within the set range are detected, such that the notification level corresponding to at least one type of notification used for notifying objects with low risk indicator values ​​is lower than the notification level corresponding to at least one type of notification used for notifying objects with high risk indicator values.

2. The aforementioned determination unit, Among the multiple objects, a first object whose risk indicator value is within the set range and a second object whose risk indicator value is within the set range and whose risk indicator value is smaller than that of the first object are detected. The warning device according to claim 1, wherein when the risk indicator value of the first object falls below the set range, the level of any one of the notification types for the first object is changed to a higher or lower level, and then the notification of the first object by the HMI device is stopped.

3. The aforementioned determination unit, Among the multiple objects, a first object whose risk indicator value is within the set range and a second object whose risk indicator value is within the set range and whose risk indicator value is smaller than that of the first object are detected. The warning device according to claim 1, wherein when the risk index value of the first object falls below the set range, the level of the first type of notification used to notify the first object is changed to a higher level, the level of the second type of notification which is different from the first type is changed to a lower level, and then the notification of the first object by the HMI device is stopped.

4. The HMI device includes a visual HMI device. The determination unit determines the value of the visual parameter that changes the image displayed by the visual HMI device from the level of notification determined based on the risk indicator value. The warning device according to claim 1, wherein the visual parameters include parameters for changing at least one of the following: scaling the image, the display size of the image, the display brightness of the image, the display color of the image, the shape of the image, and the flashing period of the image.

5. The HMI device includes an auditory HMI device. The determination unit determines the value of the auditory parameter that changes the sound output by the auditory HMI device from the level of notification determined based on the risk indicator value. The attention-raising device according to claim 1, wherein the auditory parameters include a parameter that changes at least one of the playback speed or playback cycle, volume, and pitch of the audio data that is the source of the sound.

6. The HMI device includes a tactile HMI device. The determination unit determines the value of a tactile parameter that changes the vibration level of the vibration output by the tactile HMI device from the level of notification determined based on the risk index value. The attention-raising device according to claim 1, wherein the tactile parameters include a parameter that modifies at least one of the period of the vibration, the amplitude of the vibration, and the waveform of the vibration.

7. The processor of the warning device installed in the vehicle, An acquisition step of obtaining a risk index value indicating the risk of collision with an object located in front of the vehicle, based on the output of multiple sensors mounted on the vehicle, When the acquired risk indicator value detects an object within a predetermined range, a decision step is made to determine, based on the risk indicator value, the type of notification to be sent to the vehicle driver via the HMI device, and the level of notification for each type of notification. A notification step in which the HMI device corresponding to the type of notification determined in the decision step causes the vehicle driver to receive a notification at the level of the notification determined in the decision step, Make it run, The aforementioned decision step is, A warning method that, when multiple objects with risk indicator values ​​within the set range are detected, determines the level of notification for objects with low risk indicator values ​​such that the level of notification corresponding to at least one type of notification used for notifying objects with low risk indicator values ​​is lower than the level of notification corresponding to at least one type of notification used for notifying objects with high risk indicator values.

Images