Drive assistance device, drive assistance method, and program

Abstract

To provide a drive assistance device, a drive assistance method and a program capable of making an occupant intuitively recognize a position or a direction of a risk object present around a mobile body.SOLUTION: A drive assistance device comprises: a recognition unit that recognizes a risk object present around a mobile body on the basis of an output of an object detection device; and a notification control unit that makes at least one speaker output a notification sound on the basis of a position of the risk object. The notification control unit makes the at least one speaker output a first notification sound when the risk object is present in a first region including a front region of the mobile body, and makes the at least one speaker output a second notification sound when the risk object is present in a second region including a rear region of the mobile body. The first notification sound is a sound having a pitch different from that of the second notification sound.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a driving assistance device, a driving assistance method, and a program. [Background technology]

[0002] Conventionally, there are known techniques for using sound to alert a driver (passenger) to a target around a moving vehicle. Among these techniques, there is one that uses sound image localization to generate sound in a position and direction to alert the driver (passenger) to the target target (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6587776 Summary of the Invention [Problem to be solved by the invention]

[0004] However, sound image localization in the interior of a moving body may not be as effective as desired. For example, in the case of a vehicle, sound generated near the rear seat may be reflected by the windshield, causing the driver to feel as if the sound is coming from the front. Thus, with conventional technology, there are cases where the passengers cannot intuitively recognize the position or direction of risk objects present around the moving body.

[0005] The present invention has been made taking such circumstances into consideration, and one of its objectives is to provide a driving assistance device, a driving assistance method, and a program that can enable an occupant to intuitively recognize the position or direction of a risk object present in the vicinity of a moving vehicle. [Means for solving the problem]

[0006] A driving assistance device, a driving assistance method, and a program according to the present invention employ the following configuration.

[0007] (1): A driving assistance device according to one embodiment of the present invention includes a recognition unit that recognizes a risk object present in the vicinity of a moving body based on the output of an object detection device, and a notification control unit that causes at least one speaker to output a notification sound based on the position of the risk object, wherein the notification control unit causes the at least one speaker to output a first notification sound when the risk object is present in a first area including a forward area of ​​the moving body, and causes the at least one speaker to output a second notification sound when the risk object is present in a second area including a rear area of ​​the moving body, and the first notification sound is a sound having a different pitch from the second notification sound.

[0008] (2) In the above aspect (1), the first notification sound is a sound with a higher pitch than the second notification sound.

[0009] (3): In the above aspect (1) or (2), the first notification sound includes a first reference sound and a first additional sound that is an integer harmonic overtone of the first reference sound.

[0010] (4): In any of the above aspects (1) to (3), the second notification sound includes a second reference sound and the second additional sound having a frequency different from 1 / n times the frequency of the second reference sound (n is a natural number).

[0011] (5): A driving assistance device according to another aspect of the present invention includes a recognition unit that recognizes a risk object present in the vicinity of a moving body based on the output of an object detection device, and a notification control unit that causes at least one speaker to output a notification sound based on the position of the risk object, wherein the notification control unit causes the at least one speaker to output a first notification sound when the risk object is present in a first area including a forward area of ​​the moving body, and causes the at least one speaker to output a second notification sound when the risk object is present in a second area including a rear area of ​​the moving body, one of the first notification sound and the second notification sound being a sound including a reference sound A and an integer harmonic of the reference sound A, and the other of the first notification sound and the second notification sound being a sound including a reference sound B and an additional sound having a frequency different from 1 / n times the frequency of the reference sound B (n is a natural number).

[0012] (6): In the above aspect (5), the first notification sound includes a first reference sound and a first additional sound that is an integer harmonic of the first reference sound.

[0013] (7): In the above aspect (3) or (6), the notification control unit, when the risk object is present in the first area, changes the manner of the first additional sound based on the position of the risk object within the first area.

[0014] (8): In the above aspect (7), when the risk object is present in the first area, the notification control unit increases the number of frequency types of the first additional sound, the closer the direction of the risk object as seen from the moving body is to the direction of travel of the moving body.

[0015] (9): In the above aspect (7) or (8), when the risk object is present in a first central area of ​​the first area, which is located in the center of the moving body in the width direction, the notification control unit increases the number of frequency types of the first additional sound compared to when the risk object is present in a first left area or a first right area of ​​the first area, which are located on either the left or right side of the first central area.

[0016] (10): In any of the above aspects (5) to (9), the second notification sound includes a second reference sound and a second additional sound having a frequency different from 1 / n times the frequency of the second reference sound.

[0017] (11): In the above aspect (4) or (10), when the risk object is present in the second area, the notification control unit changes the manner of the second additional sound based on the position of the risk object within the second area.

[0018] (12): In the above aspect (11), when the risk object is present in the second area, the notification control unit increases the volume of the second additional sound the closer the direction of the risk object as seen from the moving body is to the opposite direction to the traveling direction of the moving body.

[0019] (13): In the above aspect (11) or (12), the notification control unit increases the volume of the second notification sound when the risk object is present in a second central area of ​​the second area, which is located in the center of the moving body in the width direction, compared to when the risk object is present in a second left area or a second right area of ​​the second area, which are located on either the left or right side of the second central area.

[0020] (14): In any of the above aspects (1) to (13), when the risk object is present in the first area, the notification control unit slows down the tempo of the first notification sound the closer the direction of the risk object as seen from the moving body is to the direction of travel of the moving body.

[0021] (15): In any of the above aspects (1) to (14), when the risk object is present in the second area, the notification control unit increases the tempo of the second notification sound the closer the direction of the risk object as seen from the moving body is to the opposite direction to the traveling direction of the moving body.

[0022] (16): In any of the above aspects (1) to (15), the notification control unit increases the tempo of the first notification sound or the second notification sound as the distance between the moving body and the risk object decreases.

[0023] (17): In any of the above aspects (1) to (16), the at least one speaker includes a plurality of speakers, and the notification control unit determines a position for localizing a sound image of the first notification sound based on the position of the risk object in the first area when the risk object is present in the first area, and determines a position for localizing a sound image of the second notification sound based on the position of the risk object in the second area when the risk object is present in the second area.

[0024] (18): In any of the above aspects (1) to (17), the risk object is a traffic participant.

[0025] (19): A driving assistance method according to another aspect of the present invention includes a computer mounted on a moving body that recognizes a risk object present in the vicinity of the moving body based on the output of an object detection device, and causes at least one speaker to output a notification sound based on the position of the risk object, and when outputting the notification sound, if the risk object is present in a first area including a forward area of ​​the moving body, the computer causes the at least one speaker to output a first notification sound, and if the risk object is present in a second area including a rear area of ​​the moving body, the computer causes the at least one speaker to output a second notification sound, and the first notification sound is a sound having a different pitch from the second notification sound.

[0026] (20): A driving assistance method according to another aspect of the present invention includes a computer mounted on a moving body that recognizes a risk object present in the vicinity of the moving body based on the output of an object detection device, and causes at least one speaker to output a notification sound based on the position of the risk object, and when outputting the notification sound, if the risk object is present in a first area including a forward area of ​​the moving body, the computer causes the at least one speaker to output a first notification sound, and if the risk object is present in a second area including a rear area of ​​the moving body, the computer causes the at least one speaker to output a second notification sound, wherein one of the first notification sound and the second notification sound is a sound including a reference sound A and an integer harmonic of the reference sound A, and the other of the first notification sound and the second notification sound is a sound including a reference sound B and an additional sound having a frequency different from a frequency that is 1 / n times the frequency of the reference sound B (n is a natural number).

[0027] (21): A program according to another aspect of the present invention causes a computer mounted on a moving body to recognize a risk object present in the vicinity of the moving body based on the output of an object detection device, and to output a notification sound from at least one speaker based on the position of the risk object, and when outputting the notification sound, the computer causes the at least one speaker to output a first notification sound if the risk object is present in a first area including a front area of ​​the moving body, and causes the at least one speaker to output a second notification sound if the risk object is present in a second area including a rear area of ​​the moving body, and the first notification sound is a sound having a different pitch from the second notification sound.

[0028] (22): A program according to another aspect of the present invention causes a computer mounted on a moving body to recognize a risk object present in the vicinity of the moving body based on the output of an object detection device, and to output a notification sound from at least one speaker based on the position of the risk object, and when outputting the notification sound, the computer causes the at least one speaker to output a first notification sound if the risk object is present in a first area including a front area of ​​the moving body, and causes the at least one speaker to output a second notification sound if the risk object is present in a second area including a rear area of ​​the moving body, wherein one of the first notification sound and the second notification sound is a sound including a reference sound A and an integer harmonic of the reference sound A, and the other of the first notification sound and the second notification sound is a sound including a reference sound B and an additional sound having a frequency different from 1 / n times the frequency of the reference sound B (n is a natural number). Effect of the Invention

[0029] According to the above aspects (1) to (22), it is possible to allow a passenger to intuitively recognize the position or direction of a risk object existing around the moving body. [Brief description of the drawings]

[0030] [Figure 1] 1 is a diagram showing devices mounted on a vehicle M, focusing on a driving assistance device 100 according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating the configuration of a speaker unit 60. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a speaker unit 60. [Figure 4] FIG. 13 is a diagram illustrating another example of the hardware configuration of the speaker unit 60. [Diagram 5] FIG. 2 is a diagram for explaining a first region A1 and a second region A2. [Figure 6] 11 is a diagram illustrating an example of what is meant when the first notification sound NS1 is higher in pitch than the second notification sound NS2. FIG. [Figure 7]11 is a diagram for explaining the configuration of a first notification sound NS1 and a second notification sound NS2. FIG. [Figure 8] FIG. 13 is a diagram for explaining an example of increasing the number of frequency varieties of the first additional sound NS1-p. [Figure 9] 13 is a diagram illustrating an example of increasing the volume of the second additional sound NS2-q. FIG. [Figure 10] FIG. 4 is a diagram illustrating definitions of parameters in the embodiment. [Figure 11] 11A and 11B are diagrams illustrating an example of varying the tempo of a notification sound. [Figure 12] FIG. 13 is a diagram illustrating an example of volume control according to a distance Drisk. [Figure 13] 4 is a flowchart showing an example of a flow of processing executed by the driving assistance device 100. [Figure 14] 11 is a diagram showing devices mounted on a vehicle M, focusing on a driving assistance device 100A according to a third embodiment. FIG. [Figure 15] FIG. 13 is a diagram showing devices mounted on a two-wheeled vehicle MB, focusing on a driving support device 100B according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Hereinafter, with reference to the drawings, embodiments of the driving assistance device, driving assistance method, and program of the present invention will be described. The driving assistance device is a device that assists driving of a moving object. The moving object may include any moving object on which a person (driver) rides, including three-wheeled or four-wheeled vehicles, two-wheeled vehicles, micromobiles, etc. In the first to third embodiments described below, the moving object is a four-wheeled vehicle, and a vehicle equipped with a driving assistance device is referred to as vehicle M. In the fourth embodiment, the moving object is a two-wheeled vehicle.

[0032] [composition] 1 is a diagram showing devices mounted on a vehicle M, mainly a driving assistance device 100 according to the first embodiment. The vehicle M may be any of an automobile powered by an internal combustion engine such as a diesel engine or a gasoline engine, an electric automobile powered by an electric motor, and a hybrid automobile equipped with both an internal combustion engine and an electric motor.

[0033] The vehicle M is equipped with, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a vehicle sensor 40, a speaker unit 60, a display device 70, and a driving support device 100. In this embodiment, the configuration for driving the vehicle M, for example, a driving operator, a drive device such as an engine or a motor, a steering device, a brake device, etc. are not shown or described, but these configurations may be installed in the vehicle M.

[0034] The camera 10 is, for example, a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to any location of the vehicle M in which the vehicle system 1 is mounted. When capturing an image of the front, the camera 10 is attached to the top of the front windshield, the back of the rearview mirror, or the like. The camera 10, for example, periodically and repeatedly captures images of the periphery of the vehicle M. The camera 10 may be a stereo camera. A plurality of cameras 10 having different detection directions may be mounted on the vehicle M.

[0035] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by an object (reflected waves) to detect at least the position (distance and direction) of the object. The radar device 12 is attached to an arbitrary location of the vehicle M. The radar device 12 may detect the position and speed of an object by an FM-CW (Frequency Modulated Continuous Wave) method. A plurality of radar devices 12 having different detection directions may be mounted on the vehicle M.

[0036] The LIDAR 14 irradiates light (or electromagnetic waves with a wavelength close to that of light) around the vehicle M and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between light emission and light reception. The irradiated light is, for example, a pulsed laser light. The LIDAR 14 is attached to an arbitrary location of the vehicle M.

[0037] The object recognition device 16 recognizes objects by analyzing images captured by the camera 10. Then, the object recognition device 16 performs sensor fusion processing on some or all of the detection results from the camera 10, the radar device 12, and the LIDAR 14 to recognize the position (distance and angle), type, speed, etc. of the object. The object recognition device 16 outputs the recognition result to the driving assistance device 100. The object recognition device 16 may output the detection results from the camera 10, the radar device 12, and the LIDAR 14 to the driving assistance device 100 as they are.

[0038] The group of devices including the camera 10, the radar device 12, the LIDAR 14, and the object recognition device 16 is an example of an "object detection device." A simpler configuration, such as the camera 10 and the object recognition device 16, or the camera 10 and the radar device 12, may be mounted on the vehicle M as the "object detection device." When the object recognition device 16 is omitted, a function of analyzing the image captured by the camera 10 may be mounted on the driving assistance device 100. The "object detection device" may also include an ultrasonic sensor.

[0039] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, a direction sensor that detects the direction of the host vehicle M, and the like.

[0040] 2 is a configuration diagram of the speaker unit 60. The speaker unit 60 includes, for example, four speakers, 60FR, 60FL, 60RR, and 60RL. For example, the speaker 60FR is provided between the right end of the instrument panel and the right door in the vehicle interior of the vehicle M, the speaker 60FL is provided between the left end of the instrument panel and the right door in the vehicle interior of the vehicle M, the speaker 60RR is provided under the seat of the right rear seat in the vehicle interior of the vehicle M, and the speaker 60RL is provided under the seat of the left rear seat in the vehicle interior of the vehicle M. The speaker unit 60 may include a pair of left and right speakers. Also, one speaker may be mounted in the vehicle M instead of the speaker unit 60.

[0041] 3 is a diagram showing an example of a hardware configuration of the speaker unit 60. For example, control processors 62FR, 62FL, 62RR, and 62RL are attached to the speakers 60FR, 60FL, 60RR, and 60RL, respectively. The speakers and the control processors are connected by a Universal Serial Bus (USB), and the control processors are connected to the driving assistance device 100 by a Controller Area Network (CAN) bus or the like.

[0042] 4 is a diagram showing another example of the hardware configuration of the speaker unit 60. For example, the speakers 60FR and 60FL are connected by analog, and the speaker 60FL and the control processor 64FL are connected by USB. The speakers 60RR and 60RL are connected by analog, and the speaker 60RL and the control processor 64RL are connected by USB. The control processors 64FL and RL are connected to the driving assistance device 100 by a CAN bus or the like.

[0043] In this way, a control processor may be attached to each speaker, or a control processor may be attached to each of a plurality of speakers. The control processor causes the corresponding speaker to output a notification sound (including a first notification sound and a second notification sound) having a frequency instructed by the driving support device 100 described later.

[0044] The display device 70 is, for example, a touch panel, and is attached to an arbitrary location within the cabin of the vehicle M. The display device 70 accepts various operations for the driving assistance device 100, and displays images instructed by the driving assistance device 100 or the like.

[0045] Returning to FIG. 1, the driving assistance device 100 includes, for example, a recognition unit 110 and a notification control unit 120. These components are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in a storage device (storage device having a non-transient storage medium) such as an HDD or flash memory in advance, or may be stored in a removable storage medium (non-transient storage medium) such as a DVD or CD-ROM, and installed in the storage device by mounting the storage medium in a drive device.

[0046] [Risk object] The recognition unit 110 recognizes risk objects present around the vehicle M based on at least some of the outputs of the camera 10, the radar device 12, the LIDAR 14, and the object recognition device 16. The risk object is, for example, an external mobile object, in other words, a traffic participant or another mobile object. Specifically, the risk object is a pedestrian, a bicycle with a person on it, or a vehicle. In addition, the risk object may include a static obstacle, for example, an abandoned object, a parked vehicle, or a utility pole. "Recognizing a risk object" means recognizing the presence of a risk object classified as above and the relative position of the risk object with respect to the vehicle M. When information on the type of object is input from the object recognition device 16 together with information on the position of the object, the recognition unit 110 may recognize the risk object by referring to the type of the object, or may recognize the risk object by analyzing the image captured by the camera 10 itself. The recognition unit 110 may also recognize the risk object based on the speed of the object, the reflectivity of electromagnetic waves, or the like.

[0047] [Notification target area] The notification control unit 120 causes the speaker unit 60 (via the control processor) to output a notification sound based on the position of the risk object. When the risk object is present in a first area including the area in front of the vehicle M, the notification control unit 120 causes the speaker unit 60 to output a first notification sound, and when the risk object is present in a second area including the area behind the vehicle M, the notification control unit 120 causes the speaker unit 60 to output a second notification sound. FIG. 5 is a diagram for explaining the first area A1 and the second area A2. In the figure, D M is the direction of travel of vehicle M (forward direction, normal driving direction). Mmay be the direction of the central axis of the vehicle M, the direction of an instantaneous movement vector, or the direction of the lane center line of the road on which the vehicle M exists. The area AF is a forward area. The forward area AF is, for example, an area from the front end of the vehicle M to a predetermined distance, with the left and right outer edge lines being lines extending forward from the left and right side ends of the vehicle M. The area AR is a rear area. The rear area AR is, for example, an area from the rear end of the vehicle M to a predetermined distance (which may be the same as or different from the predetermined distance in the forward area AF) with the left and right outer edge lines being lines extending rearward from the left and right side ends of the vehicle M. The first area A1 is an area that includes at least the forward area AF and extends from the side to the front of the vehicle M. The second area A2 is an area that includes at least the rear area AR and extends from the side to the rear of the vehicle M. The first area A1 and the second area A2 are set so as not to overlap each other. The boundary between the first area A1 and the second area A2 is the reference point R of the vehicle M. M The boundary line passes through the vehicle M and extends to the left and right sides of the vehicle M. This boundary line does not need to exactly match the width direction of the vehicle M, and may be inclined forward or backward with the reference point of the vehicle M as the base point. In the figure, the reference point R of the vehicle M is M The line segments Lr and Ll extending diagonally forward from the first area A1 are the boundary between the first area A1 and the second area A2. Also, there may be a certain amount of gap between the first area A1 and the second area A2.

[0048] [Notification sound] In the first embodiment, the first notification sound is a sound with a different pitch from the second notification sound, so that the driver can intuitively grasp the location of the risk object if he / she knows which of the front and rear the notification sound is output with a higher pitch and which of the rear the notification sound is output with a lower pitch.

[0049] As an example, the first notification sound is a sound higher in frequency than the second notification sound. "A sound higher in frequency than the second notification sound" means, for example, that all of the frequency components of the sound constituting the first notification sound are higher in frequency than the frequency components of the sound constituting the second notification sound. Alternatively, it may mean that some of the frequency components of the sound constituting the first notification sound overlap with the frequency components of the sound constituting the second notification sound, but the overall frequency is high (for example, the weighted sum of the frequencies of the sounds constituting the first notification sound weighted by the volume is higher than that of the second notification sound). FIG. 6 is a diagram illustrating the meaning of the first notification sound NS1 being higher in frequency than the second notification sound NS2. The upper diagram of FIG. 6 shows a state in which all of the frequency components of the sound constituting the first notification sound NS1 are higher in frequency than the frequency components of the sound constituting the second notification sound NS2, and the lower diagram of FIG. 6 shows a state in which some of the frequency components of the sound constituting the first notification sound NS1 overlap with the frequency components of the sound constituting the second notification sound NS2, but the overall frequency is high. In the following description, it is assumed that all of the frequency components of the sound that constitutes the first notification sound NS1 are higher in frequency than the frequency components of the sound that constitutes the second notification sound NS2.

[0050] Generally, high-pitched sounds attract people's attention more than low-pitched sounds, so the driver's attention can be more quickly drawn by issuing a high-pitched notification sound for risk objects in the first area A1, i.e., risk objects in front of the vehicle M. In addition, by issuing a low-pitched notification sound for risk objects in the second area A2, i.e., risk objects in the rear of the vehicle M, the driver can be prevented from being annoyed by excessive notifications.

[0051] Conversely, the first notification sound may be lower in pitch than the second notification sound, because as described above, if the driver knows which of the front and rear the high-pitched notification sound is output for and which of the rear the low-pitched notification sound is output for, the driver can intuitively grasp the location of the risk object.

[0052] [Integer harmonics, etc.] The notification control unit 120 may cause the speaker unit 60 to output, as the first notification sound NS1, a sound including the first reference sound NS1-b and a first additional sound NS1-p (p=1, 2, ...) that is an integer harmonic of the first reference sound NS1-b. The notification control unit 120 may also cause the speaker unit 60 to output, as the second notification sound NS2, a sound including the second reference sound NS2-b and a second additional sound NS2-q (q=1, 2, ...) that has a frequency different from the 1 / n times frequency of the second reference sound NS2-b (n is a natural number). The second additional sound NS2-q may further be a sound that does not correspond to an integer harmonic of the second reference sound NS2-b. In this case, the first reference sound NS1-b is an example of the "reference sound A", and the second reference sound NS2-b is an example of the "reference sound B".

[0053] FIG. 7 is a diagram for explaining the configuration of the first notification sound NS1 and the second notification sound NS2. The first reference sound NS1-b is, for example, a sound higher than the second reference sound NS2-b, but may be a sound of the same pitch (same frequency). In the illustrated example, the first notification sound NS1 includes a first additional sound NS1-1 and a first additional sound NS1-2. The first additional sound NS1-1 and the first additional sound NS1-2 are integer harmonics of the first reference sound NS1-b, and therefore are higher than the first reference sound NS1-b. In addition, the second notification sound NS2 includes a second additional sound NS2-1 and a second additional sound NS2-2. The sound of "a frequency different from 1 / n times the frequency of the second reference sound NS2-b" can be either a sound higher than the second reference sound NS2-b or a sound lower than the second reference sound NS2-b, but the notification control unit 120, for example, causes the speaker unit 60 to output only a sound lower than the second reference sound NS2-b as the second additional sound NS2-q.

[0054] Since a sound composed of a reference sound and integer harmonics sounds clear to the human ear, and conversely, a sound whose constituent sounds do not have an integer harmonic relationship sounds cluttered to the human ear, by configuring the first notification sound NS1 and the second notification sound in this way, the driver's attention can be more quickly drawn by issuing a notification sound including the reference sound and its integer harmonics for risk objects present in the first area A1, i.e., risk objects present in front of the vehicle M. In addition, by issuing a notification sound whose constituent sounds do not have an integer harmonic relationship for risk objects present in the second area A2, i.e., risk objects present in the rear of the vehicle M, the driver can be prevented from being annoyed by excessive notifications.

[0055] Conversely, the notification control unit 120 may cause the speaker unit 60 to output, as the first notification sound, a sound including the first reference sound and a first additional sound having a frequency different from the frequency 1 / n times the frequency of the first reference sound. The notification control unit 120 may also cause the speaker unit 60 to output, as the second notification sound, a sound including the second reference sound and a second additional sound that is an integer harmonic of the second reference sound. In this case, the first reference sound is an example of the "reference sound B", and the second reference sound is an example of the "reference sound A". In both of the above cases, if the driver recognizes which of the front and rear is output with a sound including an integer harmonic and which is output with a sound including an additional sound having a frequency different from the frequency 1 / n times the frequency of the reference sound, the driver can intuitively grasp the position of the risk object.

[0056] The notification control unit 120 may further perform the following processes based on the above. For example, when a risk object exists in the first area A1, the notification control unit 120 of the first embodiment may change the mode of the first additional sound NS1-p based on the position of the risk object in the first area A1. More specifically, when a risk object exists in the first area A1, the notification control unit 120 may increase the number of types of frequencies of the first additional sound NS1-p as the direction of the risk object as seen from the vehicle M is closer to the traveling direction of the moving body. FIG. 8 is a diagram for explaining an example of increasing the number of types of frequencies of the first additional sound NS1-p. For example, when the risk object RO is present in the first area A1, and the risk object RO is present in the first central area A1-c that is in the center of the first area A1 in the width direction of the vehicle M, the notification control unit 120 may increase the number of types of frequencies of the first additional sound NS1-p compared to when the risk object RO is present in the first left area A1-l or the first right area A1-r that are on both the left and right sides of the first central area A1-c in the first area A1. The figure shows an example in which the speaker unit 60 outputs the first additional sounds NS1-1, NS1-2, and NS1-3 when the risk object RO is present in the first central area A1-c, and the speaker unit 60 outputs the first additional sound NS1-1 when the risk object RO is present in the first right area A1-r. The notification control unit 120 may classify the position of the risk object RO in finer increments and change the frequency type of the first additional sound NS1-p, rather than being limited to divisions such as the first central area A1-c, the first left side area A1-l, and the first right side area A1-r.

[0057] Furthermore, the notification control unit 120 may further perform the following processing. For example, when a risk object exists in the second area A2, the notification control unit 120 of the first embodiment may change the manner of the second additional sound NS2-q based on the position of the risk object in the second area A2. More specifically, when a risk object exists in the second area A2, the notification control unit 120 may increase the volume of the second additional sound NS2-q as the direction of the risk object as seen from the vehicle M approaches the opposite direction to the traveling direction of the vehicle M. FIG. 9 is a diagram for explaining an example of increasing the volume of the second additional sound NS2-q. For example, when the risk object RO is in the second area A2, the notification control unit 120 may increase the volume of the second additional sound NS2-q when the risk object RO is in the second central area A2-c in the center of the second area A2 in the width direction of the moving body, compared to when the risk object RO is in the second left area A2-l or the second right area A2-r on both the left and right sides of the second central area A2-c in the second area A2. The notification control unit 120 may classify the position of the risk object RO in finer increments and change the volume of the second additional sound NS2-q, without being limited to the divisions such as the second central area A2-c, the second left area A2-l, and the second right area A2-r.

[0058] Alternatively, the notification control unit 120 may change the volume of the first additional sound NS1-p based on the position of the risk object RO in the first area A1 when the risk object RO is present in the first area A1, and may change the frequency type of the second additional sound NS2-q based on the position of the risk object RO in the second area A2 when the risk object RO is present in the second area A2.

[0059] [Sound image localization, distance-dependent control] The notification control unit 120 may control the output state of one or both of the first notification sound NS1 and the second notification sound NS2 based on one or both of the distance between the vehicle M and the risk object and the direction (angle) of the risk object as seen from the vehicle M. FIG. 10 is a diagram showing the definition of parameters in the embodiment. In the following description, the traveling direction D of the vehicle M is MThe direction that coincides with the above is defined as 0 degrees, and the angle is defined counterclockwise. The direction of the boundary between the first central region A1-c and the first left region A1-l is defined as θ1, the direction of the boundary between the first left region A1-l and the second left region A2-l is defined as θ2, the direction of the boundary between the second left region A2-l and the second central region A2-c is defined as θ3, the direction of the boundary between the second central region A2-c and the second right region A2-r is defined as -θ3, the direction of the boundary between the second right region A2-r and the first right region A1-r is defined as -θ2, and the direction of the boundary between the first right region A1-r and the first central region A1-c is defined as -θ1. In this example, each region is set symmetrically, but each region may be set asymmetrically. In addition, the distance between the vehicle M and the risk object RO is defined as Drisk, and the direction of the risk object RO as seen from the vehicle M (presuming the above definition) is defined as θrisk.

[0060] When a risk object is present in the first area A1, the notification control unit 120 may determine a position for localizing the sound image of the first notification sound NS1 based on the position of the risk object within the first area A1, and when a risk object is present in the second area A2, the notification control unit 120 may determine a position for localizing the sound image of the second notification sound NS2 based on the position of the risk object within the second area A2.

[0061] Specifically, the notification control unit 120 localizes the sound image by determining the volume of the sound output from each speaker based on the following formula. In the following description, SL-FR indicates the volume of the sound output from the speaker 60FR, SL-FL indicates the volume of the sound output from the speaker 60FL, SL-RR indicates the volume of the sound output from the speaker 60RR, and SL-RL indicates the volume of the sound output from the speaker 60RL. Furthermore, Lbs is the reference volume. (a) When the risk object is present in the first central area A1-c SL-FR=Lbs×(θ1-θrisk) / 2θ1 SL-FL=Lbs×(θrisk+θ1) / 2θ1 SL-RR=SL-RL=0 (b) When the risk object is in the first left area A1-l SL-FL=Lbs×(θ2-θrisk) / (θ2-θ1) SL-RL=Lbs×(θrisk-θ1) / (θ2-θ1) SL-FR=SL-RR=0 (c) When the risk object is present in the second left area A2-l SL-FL=Lbs×(θ3-θrisk) / (θ3-θ2) SL-RL=Lbs×(θrisk-θ2) / (θ3-θ2) SL-FR=SL-RR=0 (d) When the risk object is present in the second central area A2-c SL-RR=Lbs×{θrisk+θ3+2×(180-θ3)} / {2×(180-θ3)} SL-RL=Lbs×(-θrisk+θ3) / {2×(180-θ3)} SL-FR=SL-FL=0 (e) When the risk object is present in the second right area A2-r SL-FR=Lbs×(θ3+θrisk) / (θ3-θ2) SL-RR=Lbs×(-θrisk-θ2) / (θ3-θ2) SL-FL=SL-RL=0 (f) When the risk object is present in the first right area A1-r SL-FR=Lbs×(θ2+θrisk) / (θ2-θ1) SL-RR=Lbs×(-θrisk-θ1) / (θ2-θ1) SL-FL=SL-RL=0

[0062] By localizing the sound image in this manner, the position of the risk object RO, particularly in the lateral direction, can be intuitively conveyed to the driver.

[0063] [Notification sound control according to direction and proximity] The notification control unit 120 may control the tempo and volume of the first notification sound NS1 or the second notification sound NS2 based on the direction θrisk and distance Drisk of the risk object RO as follows.

[0064] For example, when a risk object is present in the first area A1, the notification control unit 120 determines whether the direction of the risk object RO as seen from the vehicle M is the traveling direction D of the vehicle M. M When the risk object RO is in the second area A2, the direction of the risk object RO as seen from the vehicle M is closer to the traveling direction D of the vehicle M. M The tempo of the second notification sound NS2 is made faster as the risk object RO is closer to the opposite direction. FIG. 11 is a diagram for explaining an example of changing the tempo of the notification sound. As shown in the figure, when the risk object RO is in the first area A1, the notification control unit 120 may slow down the tempo of the first notification sound NS1 when the risk object RO is in the first center area A1-c compared to when the risk object RO is in the first left area A1-l or the first right area A1-r. This control is intended to avoid excessive notification because the driver often pays attention to the front side of the vehicle M. In addition, it is based on the consideration that the driver can be effectively alerted by notifying the front side with a high sound including integer harmonics. This makes it possible to reduce the stress on the driver by outputting the first notification sound NS1.

[0065] When the risk object RO is in the second area A2, the notification control unit 120 may set the tempo of the second notification sound NS2 faster when the risk object RO is in the second central area A2-c than when the risk object RO is in the second left area A2-l or the second right area A2-r. Furthermore, the notification control unit 120 may set the tempo of the slowest second notification sound NS2 when the risk object RO is in the second area A2 to be faster than the tempo of the fastest first notification sound NS1 when the risk object RO is in the first area A1, assuming that the distance Drisk is the same.

[0066] The notification control unit 120 may classify the position of the risk object RO in finer increments and change the tempo of the first notification sound NS1 or the second notification sound NS2, without being limited to divisions such as the first central area A1-c, the first left area A1-l, the first right area A1-r, the second central area A2-c, the second left area A2-l, and the second right area A2-r.

[0067] The notification control unit 120 may increase the tempo of the first notification sound NS1 or the second notification sound NS2 as the distance Drisk decreases, regardless of whether the risk object RO is present in any area. The notification control unit 120 may also increase the volume of the first notification sound NS1 or the second notification sound NS2 as the distance Drisk decreases. FIG. 12 is a diagram showing an example of volume control according to the distance Drisk. In the diagram, Dth is the distance at which output control of the notification sound is started, and may coincide with the distance from the vehicle M to the outer edge of the area A1 or A2.

[0068] 13 is a flowchart showing an example of the flow of processing executed by the driving support device 100. First, the recognition unit 110 recognizes a risk object (step S100). Next, the recognition unit 110 identifies an area in which the risk object exists, a distance Drisk to the risk object, and a direction θrisk of the risk object (step S102).

[0069] Next, notification control unit 120 determines the configuration, tempo, and volume of the notification sound (step S104), and causes speaker unit 60 to output the notification sound (step S106).

[0070] Next, the recognition unit 110 judges whether the position of the risk object is outside the notification range (including not being recognized) (step S108). Outside the notification range refers to a state in which the distance from the vehicle M is farther than the above-mentioned Dth. If the position of the risk object is outside the notification range, the processing of this flowchart ends. If the position of the risk object is not outside the notification range, the processing returns to step S102.

[0071] According to the first embodiment described above, it is possible to allow a passenger to intuitively recognize the position or direction of a risk object existing around a moving body (vehicle M).

[0072] <Second embodiment> The second embodiment will be described below. The driving support device of the second embodiment does not perform the configuration change control of the first notification sound NS1 in the first area A1 and the configuration change control of the second notification sound NS2 in the second area A2 described in the first embodiment using Fig. 8 and Fig. 9, but changes the configuration of the notification sound depending on whether the risk object RO is present in the first area A1 or the second area A2. That is, the driving support device of the second embodiment outputs the first notification sound NS1 including the first reference sound NS1-b and the first additional sound NS1-p (p=1, 2, ...) which is an integer harmonic of the first reference sound NS1-b from the speaker unit 60 when the risk object RO exists in the first area A1, and outputs the second notification sound NS2 including the second reference sound NS2-b and the second additional sound NS2-q (q=1, 2, ...) whose frequency is different from the frequency 1 / n times the frequency of the second reference sound NS2-b from the speaker unit 60 when the risk object RO exists in the second area A2. Other controls may be the same as those of the first embodiment. This makes it possible to achieve the same effects as those of the first embodiment with simpler control.

[0073] <Third embodiment> The third embodiment will be described below. The driving support device of the third embodiment is mounted on an autonomous vehicle. FIG. 14 is a diagram showing devices mounted on a vehicle M, centering on a driving support device 100A according to the third embodiment. The driving support device 100A is mounted on the vehicle MA together with an autonomous driving control device 150. Output data from the camera 10, the radar device 12, the LIDAR 14, the object recognition device 16, and the vehicle sensor 40 are also input to the autonomous driving control device 150. In addition to these, output data from a navigation device and an MPU (Micro Processing Unit) may also be input to the autonomous driving control device 150. The autonomous driving control device 150 recognizes the surrounding situation of the vehicle MA, recognizes a travelable area while avoiding contact with obstacles, generates a target trajectory (with a speed element) of the vehicle MA, and controls a driving force output device 200, a brake device 210, a steering device 220, and the like so that the vehicle MA can travel along the target trajectory. The driving force output device 200 includes an engine, a driving motor, and the like. Details of autonomous driving technology are publicly known from various documents, so further explanation will be omitted.

[0074] The automatic driving control device 150 is further connected to a steering grip sensor 160 and an in-vehicle camera 170. The steering grip sensor 160 is for detecting whether or not the driver is gripping the steering wheel, which is a steering operator, and the in-vehicle camera 170 is for capturing an image of the driver's head from the front. The automatic driving control device 150 judges whether or not the driver is gripping the steering wheel (hands on) by referring to the output of the steering grip sensor 160, and judges whether or not the driver is visually checking the traveling direction of the vehicle MA (eyes on) by analyzing the captured image of the in-vehicle camera 170. The automatic driving control device 150 allows either or both of releasing the steering wheel (hands off) and directing the gaze to a direction other than the traveling direction of the vehicle MA (eyes off) based on the surrounding environment of the vehicle MA (type of road on which the vehicle is traveling) and the speed of the vehicle MA. This permissible state is sometimes referred to as an automatic driving level. When an unauthorized state occurs (for example, when the driver takes his / her hands off the steering wheel in a hands-on state), the automatic driving control device 150 notifies the driver to eliminate the state, and if the state is not eliminated, performs processing such as switching to manual driving. The automatic driving control device 150 outputs information related to the automatic driving level to the driving assistance device 100A.

[0075] The driving assistance device 100A refers to information about the autonomous driving level, and automatically stops operation if both hands-off and eyes-off are permitted. In this state, the driver is temporarily no longer required to monitor the periphery of the vehicle MA, and the autonomous driving control device 150 performs control to avoid contact with risky objects. Other functions are the same as those of the first or second embodiment.

[0076] According to the third embodiment described above, control that is highly compatible with autonomous driving can be performed.

[0077] <Fourth embodiment> The fourth embodiment will be described below. The driving support device of the fourth embodiment is mounted on a motorcycle that is ridden by a rider wearing a helmet. FIG. 15 is a diagram showing devices mounted on a motorcycle MB, focusing on a driving support device 100B according to the fourth embodiment. Instead of controlling the speaker unit 60, the driving support device 100B instructs the wireless communication device 80 to transmit instruction information related to a notification sound. The wireless communication device 80 communicates based on a communication standard such as Bluetooth (registered trademark).

[0078] A receiver 310, a left ear speaker 320-L, and a right ear speaker 320-R are attached to a helmet 300 worn by a rider. The receiver 310 outputs a notification sound from one or both of the left ear speaker 320-L and the right ear speaker 320-R based on instruction information received from the wireless communication device 80.

[0079] Except for the fact that it is difficult to express the front and rear directions when controlling the sound image localization, this is the same as the first or second embodiment. That is, the recognition unit 110 and the notification control unit 120 perform the same processing as that described in the first or second embodiment, determine the composition, tempo, and volume of the first notification sound NS1 and the second notification sound NS2, and transmit instruction information to the wireless communication device 80 for outputting the notification sounds from the left ear speaker 320-L and / or the right ear speaker 320-R accordingly.

[0080] In the fourth embodiment, instead of the speaker provided in the helmet, earphones with wireless functionality may be used.

[0081] According to the fourth embodiment described above, although the degree of freedom in sound image localization decreases, in other respects it is possible to achieve the same effects as the first or second embodiment.

[0082] The above-described embodiment can be expressed as follows. A storage device storing a program; a hardware processor coupled to the storage device; The hardware processor executes the program stored in the storage device, Based on the output of the object detection device, a risk object existing in the vicinity of the moving object is recognized, outputting a notification sound from at least one speaker based on the position of the risk object; When outputting the notification sound, if the risk object is present in a first area including a front area of ​​the moving body, a first notification sound is output from the at least one speaker, and if the risk object is present in a second area including a rear area of ​​the moving body, a second notification sound is output from the at least one speaker; The first notification sound is a sound having a different pitch from the second notification sound. Driving assistance device.

[0083] The above-described embodiment can also be expressed as follows. A storage device storing a program; a hardware processor coupled to the storage device; The hardware processor executes the program stored in the storage device, The hardware processor executes the program stored in the storage device, Based on the output of the object detection device, a risk object existing in the vicinity of the moving object is recognized, outputting a notification sound from at least one speaker based on the position of the risk object; When outputting the notification sound, if the risk object is present in a first area including a front area of ​​the moving body, a first notification sound is output from the at least one speaker, and if the risk object is present in a second area including a rear area of ​​the moving body, a second notification sound is output from the at least one speaker; One of the first notification sound and the second notification sound is a sound including a reference sound A and an integer harmonic of the reference sound A, and the other of the first notification sound and the second notification sound is a sound including a reference sound B and an additional sound having a frequency different from a frequency that is 1 / n times the frequency of the reference sound B (n is a natural number). Driving assistance device.

[0084] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0085] 10. Camera 12 Radar equipment 14 LIDAR 16 Object recognition device 40 Vehicle Sensors 60 Speaker unit 80 Wireless communication devices 100, 100A, 100B Driving support device 110 Recognition part 120 Notification control section 150 Automatic Driving Control Device 300 Helmet

Claims

1. a recognition unit that recognizes risk objects present around the moving object based on an output of the object detection device; A notification control unit that causes at least one speaker to output a notification sound based on the position of the risk object, The notification control unit causes the at least one speaker to output a first notification sound when the risk object is present in a first area including a front area of ​​the moving body, and causes the at least one speaker to output a second notification sound when the risk object is present in a second area including a rear area of ​​the moving body; The first notification sound is a sound having a different pitch from the second notification sound. Driving assistance device.

2. The first notification sound is a higher pitch sound than the second notification sound. The driving assistance device according to claim 1.

3. The first notification sound includes a first reference sound and a first additional sound that is an integer harmonic of the first reference sound.

3. A driving assistance device according to claim 1 or 2.

4. The second notification sound includes a second reference sound and a second additional sound having a frequency different from 1 / n times the frequency of the second reference sound (n is a natural number). A driving assistance device according to any one of claims 1 to 3.

5. a recognition unit that recognizes risk objects present around the moving object based on an output of the object detection device; A notification control unit that outputs a notification sound to at least one speaker based on the position of the risk object, The notification control unit causes the at least one speaker to output a first notification sound when the risk object is present in a first area including a front area of ​​the moving body, and causes the at least one speaker to output a second notification sound when the risk object is present in a second area including a rear area of ​​the moving body; One of the first notification sound and the second notification sound is a sound including a reference sound A and an integer harmonic of the reference sound A, and the other of the first notification sound and the second notification sound is a sound including a reference sound B and an additional sound having a frequency different from a frequency that is 1 / n times the frequency of the reference sound B (n is a natural number). Driving assistance device.

6. The first notification sound includes a first reference sound and a first additional sound that is an integer harmonic of the first reference sound. The driving assistance device according to claim 5.

7. When the risk object is present in the first area, the notification control unit changes a manner of the first additional sound based on a position of the risk object in the first area.

7. A driving assistance device according to claim 3 or 6.

8. the notification control unit, when the risk object is present in the first area, increases the number of types of frequencies of the first additional sound as the direction of the risk object as seen from the moving body approaches the traveling direction of the moving body; The driving assistance device according to claim 7.

9. The notification control unit increases the number of types of frequencies of the first additional sound when the risk object is present in a first central area that is in the center of the first area in the width direction of the moving body, compared to when the risk object is present in a first left area or a first right area that is on both the left and right sides of the first central area of ​​the first area. A driving assistance device according to claim 7 or 8.

10. The second notification sound includes a second reference sound and a second additional sound having a frequency different from a frequency that is 1 / n times the frequency of the second reference sound. A driving assistance device according to any one of claims 5 to 9.

11. When the risk object is present in the second area, the notification control unit changes a manner of the second additional sound based on a position of the risk object in the second area. A driving assistance device according to claim 4 or 10.

12. the notification control unit, when the risk object is present in the second area, increases the volume of the second additional sound as the direction of the risk object as seen from the moving body approaches a direction opposite to a traveling direction of the moving body; A driving assistance device according to claim 11.

13. When the risk object is present in a second central area that is in the center of the second area in the width direction of the moving body, the notification control unit increases the volume of the second notification sound compared to when the risk object is present in a second left area or a second right area that is on both the left and right sides of the second central area in the second area. A driving assistance device according to claim 11 or 12.

14. When the risk object is present in the first area, the notification control unit slows down the tempo of the first notification sound as the direction of the risk object as seen from the moving body approaches the traveling direction of the moving body. A driving assistance device according to any one of claims 1 to 13.

15. When the risk object is present in the second area, the notification control unit increases the tempo of the second notification sound as the direction of the risk object as seen from the moving body approaches a direction opposite to a traveling direction of the moving body. A driving assistance device according to any one of claims 1 to 14.

16. The notification control unit increases a tempo of the first notification sound or the second notification sound as a distance between the moving object and the risk object decreases. A driving assistance device according to any one of claims 1 to 15.

17. the at least one speaker includes a plurality of speakers; When the risk object is present in the first area, the notification control unit determines a position for localizing a sound image of the first notification sound based on a position of the risk object in the first area, and when the risk object is present in the second area, determines a position for localizing a sound image of the second notification sound based on the position of the risk object in the second area. A driving assistance device according to any one of claims 1 to 16.

18. The risk object is a traffic participant, A driving assistance device according to any one of claims 1 to 17.

19. A computer installed in a vehicle Recognizing risk objects present around the moving object based on an output from an object detection device; outputting a notification sound from at least one speaker based on the position of the risk object; When outputting the notification sound, the computer causes the at least one speaker to output a first notification sound if the risk object is present in a first area including a front area of ​​the moving body, and causes the at least one speaker to output a second notification sound if the risk object is present in a second area including a rear area of ​​the moving body; The first notification sound is a sound having a different pitch from the second notification sound. Driving assistance methods.

20. A computer installed in a vehicle Recognizing risk objects present around the moving object based on an output from an object detection device; outputting a notification sound from at least one speaker based on the position of the risk object; When outputting the notification sound, the computer causes the at least one speaker to output a first notification sound if the risk object is present in a first area including a front area of ​​the moving body, and causes the at least one speaker to output a second notification sound if the risk object is present in a second area including a rear area of ​​the moving body; One of the first notification sound and the second notification sound is a sound including a reference sound A and an integer harmonic of the reference sound A, and the other of the first notification sound and the second notification sound is a sound including a reference sound B and an additional sound having a frequency different from a frequency that is 1 / n times the frequency of the reference sound B (n is a natural number). Driving assistance methods.

21. A computer installed in a moving vehicle Based on the output of the object detection device, a risk object present in the vicinity of the moving body is recognized, and based on the position of the risk object, a notification sound is output from at least one speaker. When outputting the notification sound, the computer is caused to output a first notification sound from the at least one speaker if the risk object is present in a first area including a front area of ​​the moving body, and to output a second notification sound from the at least one speaker if the risk object is present in a second area including a rear area of ​​the moving body; The first notification sound is a sound having a different pitch from the second notification sound. program.

22. A computer installed in a moving vehicle Based on the output of the object detection device, a risk object present in the vicinity of the moving body is recognized, and based on the position of the risk object, a notification sound is output from at least one speaker. When outputting the notification sound, the computer is caused to output a first notification sound from the at least one speaker if the risk object is present in a first area including a front area of ​​the moving body, and to output a second notification sound from the at least one speaker if the risk object is present in a second area including a rear area of ​​the moving body; One of the first notification sound and the second notification sound is a sound including a reference sound A and an integer harmonic of the reference sound A, and the other of the first notification sound and the second notification sound is a sound including a reference sound B and an additional sound having a frequency different from a frequency that is 1 / n times the frequency of the reference sound B (n is a natural number). program.

Images