Alarm device, alarm method, and program

Abstract

To appropriately alert a driver to a pedestrian.SOLUTION: A warning device 20 comprises: a first determination unit 232 that determines whether or not a pedestrian exists around a vehicle 1 on the basis of a result of detecting an object around the vehicle 1; a second determination unit 233 that determines whether or not a visibility indicating the visibility of a pedestrian for a driver of the vehicle 1 is equal to a threshold value or greater on the basis of a result of detecting a surrounding environment of the vehicle 1; a third determination unit 234 that determines whether or not the driver is visually aware of the pedestrian on the basis of a driver monitor camera image containing the driver's face captured by a driver monitor camera 7; and a warning unit 235. The warning unit 235 executes warning processing in a case where a pedestrian exists around the vehicle 1, the visibility is equal to the threshold value or greater, and the driver is not visually aware of the pedestrian, and executes the warning processing regardless of whether the driver is visually aware if the pedestrian in a case where a pedestrian exists around the vehicle 1 and the visibility is less than the threshold value.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to an alarm device, an alarm method, and a program.

Background Art

[0002] Patent Document 1 describes a technique for imaging the surroundings of a host vehicle, outputting imaging information, and detecting an object existing in the surroundings of the host vehicle based on the imaging information. In the technique described in Patent Document 1, it is determined whether or not an object existing in the surroundings of the host vehicle is a person, and when the object existing in the surroundings of the host vehicle is a person, a driver is alerted or the like.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, when a pedestrian existing around a vehicle is detected by a driving support system or the like, the surrounding environment of the vehicle includes a case where the surrounding environment is such that the driver can easily visually recognize the pedestrian, such as during the day, and a case where the surrounding environment of the vehicle is such that the driver has difficulty visually recognizing the pedestrian, such as at night. When the driver does not visually recognize the pedestrian, in order to avoid a collision between the vehicle and the pedestrian, for example, it is preferable that the driver be alerted about the pedestrian by a driving support system or the like. On the other hand, if an alert about the pedestrian is given to the driver by a driving support system or the like even though the driver has already visually recognized the pedestrian, there is a risk that the driver may find the alert annoying. A technique that can appropriately alert the driver about the pedestrian is desired.

[0005] In view of the above points, an object of the present disclosure is to provide an alarm device, an alarm method, and a program that can appropriately alert a driver about pedestrians.

Means for Solving the Problems

[0006] (1) One aspect of the present disclosure includes a first determination unit that determines whether a pedestrian exists around the vehicle based on a detection result of an object around the vehicle, a second determination unit that determines whether a visibility indicating the ease of visual recognition of the pedestrian by the driver of the vehicle is equal to or greater than a threshold value based on a detection result of the surrounding environment of the vehicle, a third determination unit that determines whether the driver visually recognizes the pedestrian based on a driver monitor camera image including the face of the driver captured by a driver monitor camera, and an alarm unit that executes an alarm process for outputting an alarm. The alarm unit executes the alarm process when the pedestrian exists around the vehicle, the visibility is equal to or greater than the threshold value, and the driver does not visually recognize the pedestrian, and executes the alarm process regardless of whether the driver visually recognizes the pedestrian when the pedestrian exists around the vehicle and the visibility is less than the threshold value. This is an alarm device.

[0007] (2) In the alarm device of (1), a fourth determination unit that determines whether a pedestrian group, which is a gathering of a plurality of pedestrians, exists around the vehicle based on a detection result of an object around the vehicle, and a fifth determination unit that determines whether a single pedestrian, who is a pedestrian not belonging to the pedestrian group, exists around the vehicle based on a detection result of an object around the vehicle are provided. The alarm unit may execute the alarm process regardless of whether the driver visually recognizes the single pedestrian when the pedestrian group exists around the vehicle and the single pedestrian exists around the vehicle.

[0008] (3) One aspect of the present disclosure includes a first determination step in which an alarm device determines whether a pedestrian is present around the vehicle based on a detection result of an object around the vehicle, a second determination step in which the alarm device determines whether a visibility level indicating the ease of visual recognition of the pedestrian by the driver of the vehicle is equal to or higher than a threshold value based on a detection result of the surrounding environment of the vehicle, a third determination step in which the alarm device determines whether the driver is visually recognizing the pedestrian based on a driver monitor camera image including the face of the driver captured by a driver monitor camera, and an alarm step in which the alarm device outputs an alarm. The alarm device executes the alarm step when the pedestrian is present around the vehicle, the visibility level is equal to or higher than the threshold value, and the driver is not visually recognizing the pedestrian, and executes the alarm step regardless of whether the driver is visually recognizing the pedestrian when the pedestrian is present around the vehicle and the visibility level is less than the threshold value. This is an alarm method.

[0009] (4) One aspect of the present disclosure is a program for causing a processor to execute a first determination step of determining whether a pedestrian is present around the vehicle based on a detection result of an object around the vehicle, a second determination step of determining whether a visibility level indicating the ease of visual recognition of the pedestrian by the driver of the vehicle is equal to or higher than a threshold value based on a detection result of the surrounding environment of the vehicle, a third determination step of determining whether the driver is visually recognizing the pedestrian based on a driver monitor camera image including the face of the driver captured by a driver monitor camera, and an alarm step of outputting an alarm. The alarm step is executed when the pedestrian is present around the vehicle, the visibility level is equal to or higher than the threshold value, and the driver is not visually recognizing the pedestrian, and the alarm step is executed regardless of whether the driver is visually recognizing the pedestrian when the pedestrian is present around the vehicle and the visibility level is less than the threshold value.

Advantages of the Invention

[0010] According to the present disclosure, it is possible to appropriately alert a driver about a pedestrian.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Modes for Carrying Out the Invention

[0012] Hereinafter, embodiments of an alarm device, an alarm method, and a program of the present disclosure will be described with reference to the drawings.

[0013] <First Embodiment> FIG. 1 is a diagram showing an example of a schematic configuration of a vehicle 1 to which an alarm device 20 according to the first embodiment is applied. FIG. 2 is a diagram showing an example of a specific configuration of the alarm device 20 according to the first embodiment. In the examples shown in FIGS. 1 and 2, vehicle 1 is equipped with an external camera 2, a radar 3, a LiDAR (Light Detection And Ranging) 4, a driving support ECU (Electronic Control Unit) 12, and an alarm device 20. The external camera 2 captures an image showing objects around vehicle 1 (such as pedestrians, preceding vehicles, etc.) and the road environment around vehicle 1 (such as road structure, rules, etc.), generates data of the image as detection results of the objects around vehicle 1 and the road environment around vehicle 1, and transmits it to the driving support ECU 12 and the alarm device 20. The radar 3 is, for example, a millimeter-wave radar, etc., detects the relative position and relative speed of the objects around vehicle 1 and the road structure around vehicle 1 with respect to vehicle 1, and transmits the detection results to the driving support ECU 12 and the alarm device 20. The LiDAR 4 detects the relative position and relative speed of the objects around vehicle 1 and the road structure around vehicle 1 with respect to vehicle 1, and transmits the detection results to the driving support ECU 12 and the alarm device 20.

[0014] Also, vehicle 1 is equipped with a GPS (Global Positioning System) unit 5 and a map information unit 6. The GPS unit 5 acquires position information indicating the current position of vehicle 1 based on GPS signals and transmits the position information of vehicle 1 to the driving support ECU 12. The map information unit 6 is formed, for example, in a storage such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) mounted on vehicle 1. The map information possessed by the map information unit 6 includes various types of information such as road structure (position of the road, shape of the road, etc.) and rules. Furthermore, vehicle 1 is equipped with a driver monitor camera 7, an HMI (Human Machine Interface) 8, and an illuminance sensor 9. The driver monitor camera 7 captures a driver monitor camera image including the face of the driver of vehicle 1 and transmits the data of the image to the alarm device 20. The driver monitor camera 7 is arranged at the upper part of the steering column (not shown) of vehicle 1 so as to be able to capture the face and a part of the upper body of the driver of vehicle 1. In other examples, the driver monitor camera 7 may be disposed in the center cluster of the vehicle 1, or the driver monitor camera 7 may be disposed on the rearview mirror, meter panel, meter hood, etc. of the vehicle 1. Also in these examples, the driver monitor camera 7 can image the face and a part of the upper body of the driver of the vehicle 1.

[0015] In the examples shown in FIGS. 1 and 2, the HMI 8 is an interface for inputting and outputting information between the driver assistance ECU 12, the warning device 20, etc. and the driver. The HMI 8 is an information providing device for providing various information to the driver, specifically, includes a display for displaying characters, images, etc., a speaker for outputting sound, etc. Further, the HMI 8 includes an operation button, a touch panel, etc. for receiving an input operation of the driver. The illuminance sensor 9 detects the illuminance around the vehicle 1 as the surrounding environment of the vehicle 1 and transmits the detection result to the warning device 20. The external camera 2, the radar 3, the LiDAR 4, the GPS unit 5, the map information unit 6, the driver monitor camera 7, the HMI 8, the illuminance sensor 9, the driver assistance ECU 12, and the warning device 20 are connected via the in-vehicle network 13.

[0016] Further, the vehicle 1 includes a steering actuator 14, a braking actuator 15, and a driving actuator 16. The steering actuator 14 includes, for example, a power steering system, a steer-by-wire steering system, a rear-wheel steering system, etc. The braking actuator 15 has a function of decelerating the vehicle 1. The braking actuator 15 includes, for example, a hydraulic brake, an electric regenerative brake, etc. The driving actuator 16 has a function of accelerating the vehicle 1. The driving actuator 16 includes, for example, an engine, an EV (electric vehicle) system, a hybrid system, a fuel cell system, etc. The driving support ECU 12 supports the driving of the vehicle 1 by the driver of the vehicle 1 based on data of an image showing objects around the vehicle 1 (e.g., pedestrians, preceding vehicles, etc.) and the road environment around the vehicle 1 (e.g., road structure, rules, etc.) captured by the vehicle exterior camera 2, detection results of the relative positions and relative speeds of the objects around the vehicle 1 and the road structure around the vehicle 1 with respect to the vehicle 1 by the radar 3, detection results of the relative positions and relative speeds of the objects around the vehicle 1 and the road structure around the vehicle 1 with respect to the vehicle 1 by the LiDAR 4, etc. The driving support ECU 12 transmits a control signal for operating the steering actuator 14 to the steering actuator 14, transmits a control signal for operating the braking actuator 15 to the braking actuator 15, and transmits a control signal for operating the driving actuator 16 to the driving actuator 16.

[0017] The warning device 20 has a function of executing warning processing or the like for causing the HMI 8 to output a warning to the driver of the vehicle 1. The warning device 20 is constituted by a microcomputer including a communication interface (I / F) 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 are connected via a signal line 24. The communication interface 21 has an interface circuit for connecting the warning device 20 to the in-vehicle network 13. The memory 22 has, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 22 stores programs and various data used in processing executed by the processor 23. The processor 23 has a function of determining whether it is necessary to output a warning to the HMI 8 or the like. In the examples shown in FIGS. 1 and 2, the warning device 20 includes one processor 23, but in other examples, the warning device 20 may include a plurality of processors. Also, in the examples shown in FIGS. 1 and 2, the warning device 20 is constituted by one ECU (e.g., warning ECU), but in other examples, the warning device 20 may be constituted by a plurality of ECUs.

[0018] In the examples shown in FIGS. 1 and 2, the processor 23 has functions as an acquisition unit 231, a first determination unit 232, a second determination unit 233, a third determination unit 234, and an alarm unit 235. The acquisition unit 231 acquires a detection result of an object (such as a pedestrian) around the vehicle 1 from data of an image showing the object around the vehicle 1 (such as a pedestrian) and the road environment around the vehicle 1 captured by the outside vehicle camera 2. Further, the acquisition unit 231 acquires a detection result of an object around the vehicle 1 from information indicating the relative position and relative speed of an object (such as a pedestrian) around the vehicle 1 and the road environment around the vehicle 1 with respect to the vehicle 1 detected by the radar 3. Furthermore, the acquisition unit 231 acquires a detection result of an object around the vehicle 1 from information indicating the relative position and relative speed of an object (such as a pedestrian) around the vehicle 1 and the road environment around the vehicle 1 with respect to the vehicle 1 detected by the LiDAR 4. Also, the acquisition unit 231 acquires data of a driver monitor camera image including the face of the driver of the vehicle 1 captured by the driver monitor camera 7. Furthermore, the acquisition unit 231 acquires information indicating the illuminance around the vehicle 1 detected by the illuminance sensor 9 as a detection result of the surrounding environment of the vehicle 1. Also, the acquisition unit 231 acquires information indicating the positional relationship between the outside vehicle camera 2 and the driver monitor camera 7 from, for example, the memory 22.

[0019] The first determination unit 232 determines whether or not there is a pedestrian around the vehicle 1 based on the detection result of the object around the vehicle 1 acquired by the acquisition unit 231. Specifically, the first determination unit 232 determines whether or not there is a pedestrian around the vehicle 1 based on the data of the image captured by the outside vehicle camera 2. Also, the first determination unit 232 determines whether or not there is a pedestrian around the vehicle 1 based on the detection result of the radar 3. Furthermore, the first determination unit 232 determines whether or not there is a pedestrian around the vehicle 1 based on the detection result of the LiDAR 4.

[0020] The second determination unit 233 determines whether or not the visibility indicating the ease of the driver of the vehicle 1 to visually recognize a pedestrian is equal to or higher than a threshold value based on the detection result of the surrounding environment of the vehicle 1 acquired by the acquisition unit 231. Specifically, the second determination unit 233 determines whether or not the visibility indicating the ease of the driver of the vehicle 1 to visually recognize a pedestrian is equal to or higher than a threshold value based on the illuminance around the vehicle 1 detected by the illuminance sensor 9. That is, in the examples shown in FIGS. 1 and 2, the illuminance around the vehicle 1 detected by the illuminance sensor 9 is used as the visibility indicating the ease of the driver of the vehicle 1 to visually recognize a pedestrian. The second determination unit 233 determines that the visibility indicating the ease of the driver of the vehicle 1 to visually recognize a pedestrian is equal to or higher than the threshold value, for example, during the daytime hours, and determines that the visibility indicating the ease of the driver of the vehicle 1 to visually recognize a pedestrian is less than the threshold value, for example, during the nighttime hours. In another example, based on an image showing the objects around the vehicle 1 captured by the out-of-vehicle camera 2 and the road environment around the vehicle 1, for example, a target in front of the vehicle 1 is detected, and the distance from the vehicle 1 to the farthest detected target may be used as the visibility. In still another example, the rainfall state detected by the rain sensor may be reflected in the visibility. In these examples, at night and in inclement weather (rain, snow, fog, etc.), the second determination unit 233 determines that the visibility indicating the ease of the driver of the vehicle 1 to visually recognize a pedestrian is less than the threshold value. In still another example, for example, in an environment where there is backlight for the driver, the second determination unit 233 may determine that the visibility indicating the ease of the driver of the vehicle 1 to visually recognize a pedestrian is less than the threshold value based on the image captured by the out-of-vehicle camera 2.

[0021] In the example shown in FIGS. 1 and 2, the third determination unit 234 determines whether the driver of the vehicle 1 can visually recognize a pedestrian based on a driver monitor camera image including the face of the driver of the vehicle 1 captured by the driver monitor camera 7. Specifically, the third determination unit 234 determines whether the driver of the vehicle 1 can visually recognize a pedestrian based on a driver monitor camera image including the face of the driver of the vehicle 1 captured by the driver monitor camera 7, an image including the pedestrian captured by the outside vehicle camera 2, and the positional relationship between the outside vehicle camera 2 and the driver monitor camera 7 indicated by the information acquired by the acquisition unit 231. The third determination unit 234 specifies the driver's gaze position on the outside vehicle camera image by using, for example, the techniques described in paragraphs 0018, 0019, etc. of Japanese Patent No. 6459856. Further, when a pedestrian exists at the driver's gaze position on the outside vehicle camera image, the third determination unit 234 determines that the driver of the vehicle 1 can visually recognize the pedestrian, and when no pedestrian exists at the driver's gaze position on the outside vehicle camera image, the third determination unit 234 determines that the driver of the vehicle 1 cannot visually recognize the pedestrian. In another example, the third determination unit 234 may determine whether the driver of the vehicle 1 can visually recognize a pedestrian by using a technique different from the techniques described in paragraphs 0018, 0019, etc. of Japanese Patent No. 6459856, such as the technique described in Japanese Patent No. 4879189.

[0022] In the example shown in FIGS. 1 and 2, the warning unit 235 executes a warning process for causing the HMI 8 to output a warning. Specifically, the warning unit 235 executes a warning process for causing the HMI 8 to output a warning when it is determined by the first determination unit 232 that there is a pedestrian around the vehicle 1, when it is determined by the second determination unit 233 that the visibility indicating the ease of visually recognizing the pedestrian by the driver of the vehicle 1 is equal to or greater than a threshold value, and when it is determined by the third determination unit 234 that the driver cannot visually recognize the pedestrian. That is, in the examples shown in FIGS. 1 and 2, when the surrounding environment of the vehicle 1 is a surrounding environment where the driver can easily visually recognize a pedestrian, such as during the daytime, etc., the warning unit 235 executes a warning process only when the driver does not visually recognize the pedestrian. Therefore, in the examples shown in FIGS. 1 and 2, when the surrounding environment of the vehicle 1 is a surrounding environment where the driver can easily visually recognize a pedestrian, such as during the daytime, etc., even if the driver has already visually recognized the pedestrian, it is possible to suppress the possibility that the driver feels bothered by the warning process (attention call regarding the pedestrian) by the warning unit 235.

[0023] Furthermore, in the examples shown in FIGS. 1 and 2, the warning unit 235 executes a warning process to cause the HMI 8 to output a warning regardless of whether the driver visually recognizes the pedestrian, when it is determined by the first determination unit 232 that there is a pedestrian around the vehicle 1 and it is determined by the second determination unit 233 that the visibility indicating the ease of the driver of the vehicle 1 to visually recognize the pedestrian is less than a threshold value. That is, when it is determined by the first determination unit 232 that there is a pedestrian around the vehicle 1 and it is determined by the second determination unit 233 that the visibility indicating the ease of the driver of the vehicle 1 to visually recognize the pedestrian is less than a threshold value, the warning unit 235 executes a warning process to cause the HMI 8 to output a warning without being determined by the third determination unit 234 whether the driver of the vehicle 1 visually recognizes the pedestrian. That is, in the examples shown in FIGS. 1 and 2, when the surrounding environment of the vehicle 1 is a surrounding environment where the driver has difficulty visually recognizing a pedestrian, such as at night, etc., in order to avoid a collision between the vehicle 1 and the pedestrian, etc., the warning unit 235 executes a warning process (attention call regarding the pedestrian to the driver) regardless of whether the driver visually recognizes the pedestrian. Therefore, in the examples shown in FIGS. 1 and 2, it is possible to improve the driving safety of the vehicle 1 when the surrounding environment of the vehicle 1 is a surrounding environment where the driver has difficulty visually recognizing a pedestrian, such as at night, etc.

[0024] Figure 3 is a flowchart for explaining an example of the processing executed by the processor 23 of the warning device 20 according to the first embodiment. In the example shown in FIG. 3, in step S10, the acquisition unit 231 acquires the detection result of the objects around the vehicle 1, the detection result of the surrounding environment of the vehicle 1, and the like. In step S11, the first determination unit 232 determines whether there is a pedestrian around the vehicle 1 based on the detection result of the objects around the vehicle 1 acquired in step S10. If YES, the process proceeds to step S12. If NO, the routine shown in FIG. 3 ends. In step S12, the second determination unit 233 determines whether the visibility indicating the ease of the driver of the vehicle 1 to visually recognize the pedestrian is equal to or greater than a threshold value based on the detection result of the surrounding environment of the vehicle 1 acquired in step S10. If YES, the process proceeds to step S13. If NO, the process proceeds to step S14.

[0025] In step S13, the third determination unit 234 determines whether the driver of the vehicle 1 visually recognizes the pedestrian based on the driver monitor camera image including the face of the driver captured by the driver monitor camera 7. If YES, the routine shown in FIG. 3 ends. If NO, the process proceeds to step S14. In step S14, the warning unit 235 executes a warning process for causing the HMI 8 to output a warning.

[0026] <Second Embodiment> The vehicle 1 to which the warning device 20 according to the second embodiment is applied is configured in the same manner as the vehicle 1 to which the warning device 20 according to the first embodiment shown in FIG. 1 is applied, except for the points described later. FIG. 4 is a diagram showing an example of a specific configuration of the warning device 20 according to the second embodiment.

[0027] As described above, in the examples shown in FIGS. 1 and 2, the processor 23 has functions as the acquisition unit 231, the first determination unit 232, the second determination unit 233, the third determination unit 234, and the warning unit 235. On the other hand, in the example shown in FIGS. 1 and 4, the processor 23 has functions as an acquisition unit 231, a first determination unit 232, a second determination unit 233, a third determination unit 234, a fourth determination unit 236, a fifth determination unit 237, and an alarm unit 235.

[0028] Based on the detection result of the objects around the vehicle 1 acquired by the acquisition unit 231, the fourth determination unit 236 determines whether or not there is a pedestrian group, which is a gathering of a plurality of pedestrians, around the vehicle 1. Specifically, the fourth determination unit 236 uses a known technique such as the techniques described in the following Documents 1 to 3, for example, to recognize a plurality of pedestrians who appear to overlap as one pedestrian group, and determines whether or not the pedestrian group exists around the vehicle 1. Document 1: "Association of Multiple Pedestrians Moving in Non-overlapping Camera Views" by Hisaya Akatsuka et al. (https: / / www.cs.is.i.nagoya-u.ac.jp / users / ide / res / paper / J06-symposium-akatsuka-2.pdf) Document 2: "Real-time Recognition of Multiple Pedestrians Using an On-vehicle Camera" by Hidekazu Suzuki et al., Transactions of the Institute of Electronics, Information and Communication Engineers A, Vol. J88-A, No. 2, pp. 131-141 Document 3: "Proposal of a Pedestrian Flow Estimation Method Using Deep Learning with an On-vehicle Camera" by Yusuke Hara et al., Research Report of the Information Processing Society of Japan, Vol. 2018-ITS-72, No. 3

[0029] Based on the detection result of the objects around the vehicle 1 acquired by the acquisition unit 231, the fifth determination unit 237 determines whether or not there is a single pedestrian, who is a pedestrian not belonging to the pedestrian group, around the vehicle 1.

[0030] Since a single pedestrian makes a different movement from each of the plurality of pedestrians included in the pedestrian group, the single pedestrian does not belong to the pedestrian group. When the driver of the vehicle 1 is distracted by the pedestrian group and does not notice the single pedestrian while the vehicle 1 is running, there is a risk of an increased risk of collision between the vehicle 1 and the single pedestrian.

[0031] Therefore, in the examples shown in FIGS. 1 and 4, when the fourth determination unit 236 determines that a pedestrian group exists around the vehicle 1 and the fifth determination unit 237 determines that a single pedestrian exists around the vehicle 1, regardless of whether the driver of the vehicle 1 visually recognizes the single pedestrian, the warning unit 235 executes a warning process for causing the HMI 8 to output a warning. As a result, in the examples shown in FIGS. 1 and 4, it is possible to suppress the possibility that the driver of the vehicle 1 is distracted by the pedestrian group and fails to notice the single pedestrian.

[0032] FIG. 5 is a flowchart for explaining an example of the process executed by the processor 23 of the warning device 20 according to the second embodiment. In the example shown in FIG. 5, in step S10, similar to the example shown in FIG. 3, the acquisition unit 231 acquires the detection result of the object around the vehicle 1, the detection result of the surrounding environment of the vehicle 1, and the like. In step S11, similar to the example shown in FIG. 3, the first determination unit 232 determines whether a pedestrian exists around the vehicle 1 based on the detection result of the object around the vehicle 1 acquired in step S10. If YES, the process proceeds to step S20; if NO, the routine shown in FIG. 5 ends.

[0033] In step S20, the fourth determination unit 236 determines whether a pedestrian group, which is a gathering of a plurality of pedestrians, exists around the vehicle 1 based on the detection result of the surrounding environment of the vehicle 1 acquired in step S10. If YES, the process proceeds to step S21; if NO, the process proceeds to step S12. In step S21, the fifth determination unit 237 determines whether a single pedestrian, who does not belong to the pedestrian group, exists around the vehicle 1 based on the detection result of the surrounding environment of the vehicle 1 acquired in step S10. If YES, the process proceeds to step S14; if NO, the routine shown in FIG. 5 ends.

[0034] In step S12, similar to the example shown in FIG. 3, the second determination unit 233 determines whether the visibility indicating the ease of the driver of the vehicle 1 to visually recognize a pedestrian is equal to or greater than a threshold value based on the detection result of the surrounding environment of the vehicle 1 acquired in step S10. If YES, the process proceeds to step S13, and if NO, the process proceeds to step S14. In step S13, the third determination unit 234 determines whether the driver of the vehicle 1 is visually recognizing a pedestrian based on the driver monitor camera image including the face of the driver imaged by the driver monitor camera 7. If YES, the routine shown in FIG. 5 ends, and if NO, the process proceeds to step S14. In step S14, similar to the example shown in FIG. 3, the warning unit 235 executes a warning process for outputting a warning to the HMI 8.

[0035] As described above, the embodiments of the warning device, warning method, and program of the present disclosure have been described with reference to the drawings. However, the warning device, warning method, and program of the present disclosure are not limited to the above-described embodiments, and appropriate changes can be made without departing from the spirit of the present disclosure. The configurations of the respective examples of the above-described embodiments may be combined as appropriate. In each of the above-described embodiments, the processing performed in the warning device 20 has been described as software processing performed by executing a program. However, the processing performed in the warning device 20 may be processing performed by hardware. Alternatively, the processing performed in the warning device 20 may be processing that combines both software and hardware. Further, a program (a program that realizes the functions of the processor 23 of the warning device 20) stored in the memory 22 of the warning device 20 may be recorded and provided, distributed, etc. on a computer-readable storage medium such as a semiconductor memory, a magnetic recording medium, an optical recording medium, or the like.

Description of Reference Numerals

[0036] 1 Vehicle 2 Outdoor camera 3 Radar 4 LiDAR 5 GPS unit 6 Map Information Unit 7 Driver Monitoring Camera 8 HMI 9 Illuminance Sensor 12 Driving Support ECU 14 Steering Actuator 15 Brake Actuator 16 Drive Actuator 20 Alarm Device 21 Communication Interface 22 Memory 23 Processor 231 Acquisition Unit 232 First Judgment Unit 233 Second Judgment Unit 234 Third Judgment Unit 235 Alarm Unit 236 Fourth Judgment Unit 237 Fifth Judgment Unit

Claims

1. A first determination unit that determines whether a pedestrian exists around the vehicle based on a detection result of an object around the vehicle; A second determination unit that determines whether a visibility level indicating the ease of visual recognition of the pedestrian by the driver of the vehicle is equal to or higher than a threshold value based on a detection result of the surrounding environment of the vehicle; A third determination unit that determines whether the driver visually recognizes the pedestrian based on a driver monitor camera image including the face of the driver captured by a driver monitor camera; A fourth determination unit that determines whether a pedestrian group, which is a gathering of a plurality of pedestrians, exists around the vehicle based on a detection result of an object around the vehicle; A fifth determination unit that determines whether a single pedestrian, who is a pedestrian not belonging to the pedestrian group, exists around the vehicle based on a detection result of an object around the vehicle; An alarm unit that executes an alarm process for outputting an alarm, wherein the alarm unit executes the alarm process when the pedestrian exists around the vehicle, the visibility level is equal to or higher than the threshold value, and the driver does not visually recognize the pedestrian; executes the alarm process when the pedestrian exists around the vehicle and the visibility level is less than the threshold value, regardless of whether the driver visually recognizes the pedestrian; wherein the fifth determination unit determines whether the single pedestrian, who makes a movement different from each of the plurality of pedestrians included in the pedestrian group, exists around the vehicle based on data of an image captured by an external camera, a detection result of a radar, and a detection result of a LiDAR; wherein the alarm unit executes the alarm process regardless of whether the driver visually recognizes the single pedestrian when the pedestrian group exists around the vehicle and the single pedestrian, who makes a movement different from each of the plurality of pedestrians included in the pedestrian group, exists around the vehicle. An alarm device.

2. An alarm device includes a first determination step of determining whether a pedestrian exists around the vehicle based on a detection result of an object around the vehicle; A second determination step of determining whether a visibility level indicating the ease of visual recognition of the pedestrian by the driver of the vehicle is equal to or higher than a threshold value based on a detection result of the surrounding environment of the vehicle; A third determination step in which the warning device determines whether or not the driver visually recognizes the pedestrian based on a driver monitor camera image including the face of the driver imaged by the driver monitor camera; A fourth determination step in which the warning device determines whether or not a pedestrian group, which is a gathering of a plurality of pedestrians, exists around the vehicle based on a detection result of an object around the vehicle; A fifth determination step in which the warning device determines whether or not a single pedestrian who is not a member of the pedestrian group exists around the vehicle based on a detection result of an object around the vehicle; The warning device includes a warning step of outputting a warning; The warning device executes the warning step when the pedestrian exists around the vehicle, the visibility is equal to or higher than the threshold value, and the driver does not visually recognize the pedestrian; when the pedestrian exists around the vehicle and the visibility is less than the threshold value, the warning step is executed regardless of whether or not the driver visually recognizes the pedestrian; In the fifth determination step, based on the data of the image captured by the out-of-vehicle camera, the detection result of the radar, and the detection result of the LiDAR, it is determined whether or not the single pedestrian who makes a movement different from each of the plurality of pedestrians included in the pedestrian group exists around the vehicle; The warning step is executed regardless of whether or not the driver visually recognizes the single pedestrian when the pedestrian group exists around the vehicle and the single pedestrian who makes a movement different from each of the plurality of pedestrians included in the pedestrian group exists around the vehicle. A warning method.

3. A processor A first determination step of determining whether or not a pedestrian exists around the vehicle based on a detection result of an object around the vehicle; A second determination step of determining whether or not a visibility indicating the ease of visual recognition of the pedestrian by the driver of the vehicle is equal to or higher than a threshold value based on a detection result of the surrounding environment of the vehicle; A third determination step of determining whether or not the driver visually recognizes the pedestrian based on a driver monitor camera image including the face of the driver imaged by the driver monitor camera; A fourth determination step of determining whether or not a pedestrian group, which is a gathering of a plurality of pedestrians, exists around the vehicle based on a detection result of an object around the vehicle; A fifth determination step of determining whether or not a single pedestrian, who is a pedestrian not belonging to the pedestrian group, exists around the vehicle based on a detection result of an object around the vehicle; A program for causing an alarm step of outputting an alarm to be executed, wherein the alarm step is executed when the pedestrian exists around the vehicle, the visibility is equal to or higher than the threshold value, and the driver does not visually recognize the pedestrian; the alarm step is executed regardless of whether or not the driver visually recognizes the pedestrian when the pedestrian exists around the vehicle and the visibility is less than the threshold value; in the fifth determination step, based on data of an image captured by an outside vehicle camera, a detection result of a radar, and a detection result of a LiDAR, it is determined whether or not the single pedestrian, who makes a movement different from each of the plurality of pedestrians included in the pedestrian group, exists around the vehicle; the alarm step is, a program that is executed regardless of whether or not the driver visually recognizes the single pedestrian when the pedestrian group exists around the vehicle and the single pedestrian, who makes a movement different from each of the plurality of pedestrians included in the pedestrian group, exists around the vehicle.

Images