Driving assistance device, driving assistance method, and program

Abstract

To provide a drive assistance device, a drive assistance method and a program capable of making an occupant intuitively recognize a position or a direction of a risk object present around a mobile body.SOLUTION: A drive assistance device comprises: a recognition unit that recognizes a risk object present around a mobile body on the basis of an output of an object detection device; and a notification control unit that makes at least one speaker output a notification sound on the basis of a position of the risk object. The notification control unit makes the at least one speaker output a first notification sound when the risk object is present in a first region including a front region of the mobile body, and makes the at least one speaker output a second notification sound when the risk object is present in a second region including a rear region of the mobile body. The first notification sound is a sound having a pitch different from that of the second notification sound.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a driving support device, a driving support method, and a program.

Background Art

[0002] Conventionally, a technique for alerting a driver (occupant) by sound about an object to be noted existing around a moving body is known. Among them, there is one that generates sound at a position and in a direction where an alert is to be made by sound image localization (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, sound image localization in the interior of a moving body may not exhibit the effect as expected. For example, in the case of a vehicle, the sound generated near the rear seat may be reflected by the front glass and may be felt by the driver as being heard from the front. Thus, in the conventional technology, there are cases where the position or direction of a risk object existing around a moving body cannot be intuitively recognized by an occupant.

[0005] The present invention has been made in consideration of such circumstances, and one of its objects is to provide a driving support device, a driving support method, and a program that can intuitively make an occupant recognize the position or direction of a risk object existing around a moving body.

Means for Solving the Problems

[0006] The driving support device, driving support method, and program according to this invention adopt the following configuration.

[0007] (1) A driving assistance device according to one aspect of the present invention comprises a recognition unit that recognizes a risk object present around a moving body based on the output of an object detection device, and a notification control unit that causes at least one speaker to output a notification sound based on the position of the risk object, wherein the notification control unit causes at least one speaker to output a first notification sound when the risk object is in a first region including the front region of the moving body, and causes at least one speaker to output a second notification sound when the risk object is in a second region including the rear region of the moving body, and the first notification sound is a sound with a different pitch from the second notification sound.

[0008] (2) In the embodiment of (1) above, the first notification sound is a higher pitched sound than the second notification sound.

[0009] (3): In the embodiment of (1) or (2) above, the first notification tone includes a first reference tone and a first additional tone which is an integer harmonic of the first reference tone.

[0010] (4): In any embodiment of (1) to (3) above, the second notification sound includes a second reference sound and a second additional sound having a frequency different from that of 1 / n times the frequency of the second reference sound (where n is a natural number).

[0011] (5): A driving assistance device according to another aspect of the present invention comprises: a recognition unit that recognizes a risk object present around a moving body based on the output of an object detection device; and a notification control unit that causes at least one speaker to output a notification sound based on the position of the risk object, wherein the notification control unit causes at least one speaker to output a first notification sound when the risk object is in a first region including the front region of the moving body, and causes at least one speaker to output a second notification sound when the risk object is in a second region including the rear region of the moving body, wherein one of the first notification sound and the second notification sound is a sound that includes a reference sound A and integer harmonics of the reference sound A, and the other of the first notification sound and the second notification sound is a sound that includes a reference sound B and an additional sound with a frequency different from 1 / n times the frequency of the reference sound B (where n is a natural number).

[0012] (6): In the embodiment of (5) above, the first notification tone includes a first reference tone and a first additional tone which is an integer harmonic of the first reference tone.

[0013] (7) In the embodiment of (3) or (6) above, the notification control unit, when the risk object is present in the first area, causes the first additional sound to be different based on the position of the risk object within the first area.

[0014] (8) In the embodiment of (7) above, the notification control unit increases the number of types of first additional sound frequencies as the direction of the risk object as seen from the moving body is closer to the direction of travel of the moving body, when the risk object is in the first area.

[0015] (9): In the embodiment of (7) or (8) above, the notification control unit increases the types of frequencies of the first additional sound when the risk object is located in the first central region of the first region, which is in the center of the moving body in the width direction, compared to when the risk object is located in the first left region or the first right region, which are on either side of the first central region of the first region.

[0016] (10): In any embodiment of (5) to (9) above, the second notification sound includes a second reference sound and a second additional sound having a frequency different from that of 1 / n times the frequency of the second reference sound.

[0017] (11): In the embodiment of (4) or (10) above, the notification control unit, when the risk object is present in the second area, causes the second additional sound to be different based on the position of the risk object within the second area.

[0018] (12): In the embodiment of (11) above, the notification control unit increases the volume of the second additional sound when the risk object is in the second area, the closer the direction of the risk object as seen from the moving body is to the opposite direction of the moving body's direction of travel.

[0019] (13): In the embodiments of (11) or (12) above, the notification control unit increases the volume of the second notification sound when the risk object is located in the second central region, which is the central part of the second region in the width direction of the moving body, compared to when the risk object is located in the second left region or the second right region, which are on either side of the second central region.

[0020] (14): In any embodiment of (1) to (13) above, the notification control unit slows down the tempo of the first notification sound the closer the direction of the risk object is to the direction of travel of the moving body when the risk object is in the first area.

[0021] (15): In any embodiment of (1) to (14) above, the notification control unit increases the tempo of the second notification sound if the risk object is in the second area, the closer the direction of the risk object as seen from the moving body is to the opposite direction of the moving body's direction of travel.

[0022] (16) In any of the aspects (1) to (15) above, as the distance between the moving body and the risk object becomes smaller, the tempo of the first notification sound or the second notification sound is increased by the notification control unit.

[0023] (17) In any of the aspects (1) to (16) above, the at least one speaker includes a plurality of speakers, and when the risk object is present in the first area, the notification control unit determines a position for localizing the sound image of the first notification sound based on the position of the risk object in the first area, and when the risk object is present in the second area, the notification control unit determines a position for localizing the sound image of the second notification sound based on the position of the risk object in the second area.

[0024] (18) In any of the aspects (1) to (17) above, the risk object is a traffic participant.

[0025] (19) A driving support method according to another aspect of the present invention is such that a computer mounted on a moving body recognizes a risk object existing around the moving body based on an output of an object detection device, and based on the position of the risk object, causes a notification sound to be output to at least one speaker. When outputting the notification sound, when the risk object is present in a first area including a front area of the moving body, the computer causes the at least one speaker to output a first notification sound, and when the risk object is present in a second area including a rear area of the moving body, the computer causes the at least one speaker to output a second notification sound, and the first notification sound is a sound having a different pitch from the second notification sound.

[0026] (20): A driving assistance method according to another aspect of the present invention is a method in which a computer mounted on a mobile body recognizes a risk object present around the mobile body based on the output of an object detection device, and based on the position of the risk object, causes at least one speaker to output a notification sound, and when the computer outputs the notification sound, if the risk object is in a first region including the front region of the mobile body, it causes at least one speaker to output a first notification sound, and if the risk object is in a second region including the rear region of the mobile body, it causes at least one speaker to output a second notification sound, and either the first notification sound or the second notification sound is a sound that includes a reference sound A and integer harmonics of the reference sound A, and the other of the first notification sound or the second notification sound is a sound that includes a reference sound B and an additional sound with a frequency different from 1 / n times the frequency of the reference sound B (where n is a natural number).

[0027] (21): A program according to another aspect of the present invention causes a computer mounted on a mobile body to recognize a risk object present in the vicinity of the mobile body based on the output of an object detection device, and to output a notification sound to at least one speaker based on the position of the risk object, wherein when outputting the notification sound, the computer causes the computer to output a first notification sound to at least one speaker if the risk object is in a first region including the front region of the mobile body, and to output a second notification sound to at least one speaker if the risk object is in a second region including the rear region of the mobile body, wherein the first notification sound is a sound with a different pitch from the second notification sound.

[0028] (22): A program according to another aspect of the present invention causes a computer mounted on a mobile body to recognize risky objects present around the mobile body based on the output of an object detection device, and to output a notification sound to at least one speaker based on the position of the risky object, and when outputting the notification sound, causes the computer to output a first notification sound to at least one speaker if the risky object is in a first region including the front region of the mobile body, and to output a second notification sound to at least one speaker if the risky object is in a second region including the rear region of the mobile body, wherein one of the first notification sound and the second notification sound is a sound including a reference tone A and integer harmonics of the reference tone A, and the other of the first notification sound and the second notification sound is a sound including a reference tone B and an additional sound with a frequency different from 1 / n times the frequency of the reference tone B (where n is a natural number). [Effects of the Invention]

[0029] According to the embodiments described in (1) to (22) above, the position or direction of risky objects present around the moving object can be made to be intuitively recognized by the occupants. [Brief explanation of the drawing]

[0030] [Figure 1] This diagram shows the equipment mounted on a vehicle M, centering on the driver assistance system 100 according to the first embodiment. [Figure 2] This is a diagram showing the configuration of speaker unit 60. [Figure 3] This figure shows an example of the hardware configuration of speaker unit 60. [Figure 4] This figure shows another example of the hardware configuration of the speaker unit 60. [Figure 5] This is a diagram illustrating the first region A1 and the second region A2. [Figure 6] This diagram illustrates the meaning of the first notification sound NS1 being higher in pitch than the second notification sound NS2. [Figure 7]This diagram illustrates the configuration of the first notification sound NS1 and the second notification sound NS2. [Figure 8] This diagram illustrates an example of increasing the frequency range of the first additional tone NS1-p. [Figure 9] This diagram illustrates an example of increasing the volume of the second additional tone, NS2-q. [Figure 10] This figure shows the definition of parameters in the embodiment. [Figure 11] This diagram illustrates an example of varying the tempo of notification sounds. [Figure 12] This figure shows an example of volume control based on distance (Drisk). [Figure 13] This flowchart shows an example of the processing flow performed by the driver assistance device 100. [Figure 14] This diagram shows the equipment mounted on a vehicle M, centered around the driver assistance system 100A according to the third embodiment. [Figure 15] This figure shows the equipment installed on the motorcycle MB, centered around the driver assistance device 100B according to the fourth embodiment. [Modes for carrying out the invention]

[0031] The embodiments of the driver assistance device, driver assistance method, and program of the present invention will be described below with reference to the drawings. The driver assistance device is a device that assists in the driving of a mobile body. The mobile body includes vehicles such as three-wheeled or four-wheeled vehicles, two-wheeled vehicles, micromobiles, etc., and may include any mobile body on which a person (driver) is riding. In the first and third embodiments described below, the mobile body is assumed to be a four-wheeled vehicle, and the vehicle on which the driver assistance device is installed will be referred to as vehicle M. In the fourth embodiment, the mobile body is assumed to be a two-wheeled vehicle.

[0032] [composition] Figure 1 shows the equipment mounted on a vehicle M, centering on the driver assistance system 100 according to the first embodiment. The vehicle M may be an automobile powered by an internal combustion engine such as a diesel engine or a gasoline engine, an electric vehicle powered by an electric motor, or a hybrid vehicle that combines an internal combustion engine and an electric motor.

[0033] Vehicle M is equipped with, for example, a camera 10, a radar device 12, a LiDAR (Light Detection and Ranging) 14, an object recognition device 16, a vehicle sensor 40, a speaker unit 60, a display device 70, and a driver assistance device 100. In this embodiment, the components for driving vehicle M, such as the driver controls, drive devices such as the engine and motor, steering devices, and brake devices, are not shown or described, but these components may be mounted on vehicle M.

[0034] Camera 10 is a digital camera that utilizes a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 10 is mounted at any location on the vehicle M on which the vehicle system 1 is installed. When imaging the area in front, camera 10 is mounted on the top of the front windshield, behind the rearview mirror, etc. Camera 10 periodically and repeatedly images the area around vehicle M. Camera 10 may be a stereo camera. Multiple cameras 10 with different detection directions may be mounted on vehicle M.

[0035] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by objects (reflected waves) to determine at least the position (distance and direction) of an object. The radar device 12 can be mounted at any location on the vehicle M. The radar device 12 may detect the position and velocity of an object using the FM-CW (Frequency Modulated Continuous Wave) method. Multiple radar devices 12 with different detection directions may be mounted on the vehicle M.

[0036] LIDAR14 irradiates light (or electromagnetic waves with a wavelength close to light) around vehicle M and measures the scattered light. Based on the time from emission to reception, LIDAR14 detects the distance to the target. The irradiated light is, for example, pulsed laser light. LIDAR14 can be attached to any location on vehicle M.

[0037] The object recognition device 16 analyzes the image captured by the camera 10 to recognize an object. The object recognition device 16 then performs sensor fusion processing on some or all of the detection results from the camera 10, radar device 12, and LIDAR 14 to recognize the object's position (distance and angle), type, speed, etc. The object recognition device 16 outputs the recognition results to the driver assistance device 100. The object recognition device 16 may also output the detection results from the camera 10, radar device 12, and LIDAR 14 directly to the driver assistance device 100.

[0038] The group of devices including camera 10, radar device 12, LIDAR 14, and object recognition device 16 is an example of an "object detection device." A simpler configuration of the "object detection device," such as camera 10 and object recognition device 16, or camera 10 and radar device 12, may also be installed in the vehicle M. If the object recognition device 16 is omitted, the function of analyzing the images captured by camera 10 may be installed in the driver assistance device 100. Furthermore, the "object detection device" may also include an ultrasonic sensor.

[0039] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting angular velocity around the vertical axis, and an orientation sensor for detecting the orientation of the vehicle M.

[0040] Figure 2 is a diagram of the speaker unit 60. The speaker unit 60 includes, for example, four speakers: speakers 60FR, 60FL, 60RR, and 60RL. For example, speaker 60FR is located between the right end of the instrument panel and the right door inside the passenger compartment of vehicle M, speaker 60FL is located between the left end of the instrument panel and the right door inside the passenger compartment of vehicle M, speaker 60RR is located under the seat cushion of the right rear seat inside the passenger compartment of vehicle M, and speaker 60RL is located under the seat cushion of the left rear seat inside the passenger compartment of vehicle M. The speaker unit 60 may also include a pair of left and right speakers. Alternatively, a single speaker may be installed in the vehicle M instead of the speaker unit 60.

[0041] Figure 3 shows an example of the hardware configuration of the speaker unit 60. For example, control processors 62FR, 62FL, 62RR, and 62RL are attached to speakers 60FR, 60FL, 60RR, and 60RL, respectively. The speakers and control processors are connected via USB (Universal Serial Bus), and the control processors are connected to the driver assistance system 100 via a CAN (Controller Area Network) bus or the like.

[0042] Figure 4 shows another example of the hardware configuration of the speaker unit 60. For example, speakers 60FR and 60FL are connected analogously, and speaker 60FL is connected to the control processor 64FL via USB. Also, speakers 60RR and 60RL are connected analogously, and speaker 60RL is connected to the control processor 64RL via USB. The control processors 64FL and RL are then connected to the driver assistance device 100 via a CAN bus or the like.

[0043] Thus, a control processor may be installed for each speaker, or a control processor may be installed for each of multiple speakers. The control processor causes the corresponding speaker to output a notification sound (including a first notification sound and a second notification sound) having a frequency instructed by the driver assistance device 100 described later.

[0044] The display device 70 is, for example, a touch panel and is installed at any location inside the vehicle M. The display device 70 receives various operations for the driver assistance device 100 and displays images instructed by the driver assistance device 100, etc.

[0045] Returning to Figure 1, the driver assistance device 100 includes, for example, a recognition unit 110 and a notification control unit 120. These components are realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit), or by the cooperation of software and hardware. The program may be stored in advance in a storage device such as an HDD or flash memory (a storage device with a non-transient storage medium), or it may be stored in a removable storage medium such as a DVD or CD-ROM (a non-transient storage medium) and installed in the storage device when the storage medium is mounted in a drive device.

[0046] [Risk Object] The recognition unit 110 recognizes risk objects present around the vehicle M based on the outputs of at least some of the camera 10, radar device 12, LIDAR 14, and object recognition device 16. Risk objects are, for example, external mobile objects, in other words, traffic participants, other moving objects. Specifically, risk objects include pedestrians, bicycles or vehicles with people on them, etc. In addition, risk objects may also include static obstacles, such as abandoned objects, parked vehicles, utility poles, etc. "Recognizing a risk object" means recognizing the existence of a risk object classified as described above, and the relative position of that risk object with respect to the vehicle M. When the recognition unit 110 receives information on the type of object along with the position information of the object from the object recognition device 16, it may recognize the risk object by referring to the type of object, or it may recognize the risk object by analyzing the image captured by the camera 10 itself. The recognition unit 110 may also recognize risk objects based on the velocity of the object, the reflectivity of electromagnetic waves, etc.

[0047] [Notification target area] The notification control unit 120 causes the speaker unit 60 to output a notification sound (via the control processor) based on the location of the risk object. The notification control unit 120 causes the speaker unit 60 to output a first notification sound if the risk object is in a first region including the front region of the vehicle M, and causes the speaker unit 60 to output a second notification sound if the risk object is in a second region including the rear region of the vehicle M. Figure 5 is a diagram illustrating the first region A1 and the second region A2. In the figure, D M This is the direction of travel of vehicle M (forward direction, normal travel direction). The direction of travel of vehicle M is D. MThe direction may be the direction of the central axis of vehicle M, the direction of the instantaneous movement vector, or the direction toward the lane centerline of the road on which vehicle M is located. Region AF is the front region. For example, the front region AF is the region from the front end of vehicle M to a predetermined distance, with the left and right outer edges being lines extending from the left and right side ends of vehicle M toward the front of vehicle M. Region AR is the rear region. For example, the rear region AR is the region from the rear end of vehicle M to a predetermined distance (which may be the same as or different from the predetermined distance in the front region AF), with the left and right outer edges being lines extending from the left and right side ends of vehicle M toward the rear of vehicle M. The first region A1 is the region that includes at least the front region AF and extends from the side to the front of vehicle M. The second region A2 is the region that includes at least the rear region AR and extends from the side to the rear of vehicle M. The first region A1 and the second region A2 are set so as not to overlap with each other. The boundary line between the first region A1 and the second region A2 is the reference point R of vehicle M. M It passes through and extends to the left and right sides of vehicle M. This boundary line does not need to perfectly coincide with the width of vehicle M, and may be tilted forward or backward from the reference point of vehicle M. In the figure, the reference point R of vehicle M M The line segments Lr and Ll, which extend diagonally forward from the starting point, are assumed to be the boundary lines between the first region A1 and the second region A2. Furthermore, there may be a certain amount of space between the first region A1 and the second region A2.

[0048] [Notification sound] In the first embodiment, the first notification sound is a sound with a different pitch from the second notification sound. This allows the driver to intuitively grasp the location of the risk object by recognizing whether the high-pitched notification sound is output for the front or rear, and whether the low-pitched notification sound is output for the rear.

[0049] For example, the first notification sound is higher pitched than the second notification sound. "Higher pitched than the second notification sound" means, for example, that all the frequency components of the sound that make up the first notification sound are higher frequency than the frequency components of the sound that make up the second notification sound. Alternatively, it may mean that some of the frequency components of the sound that make up the first notification sound overlap with the frequency components of the sound that make up the second notification sound, but are generally higher frequency (for example, the weighted sum of the frequencies of the sound that make up the first notification sound, weighted by volume, is higher than that of the second notification sound). Figure 6 illustrates the meaning of the first notification sound NS1 being higher pitched than the second notification sound NS2. The upper part of Figure 6 shows that all the frequency components of the sound that make up the first notification sound NS1 are higher frequency than the frequency components of the sound that make up the second notification sound NS2, and the lower part of Figure 6 shows that some of the frequency components of the sound that make up the first notification sound NS1 overlap with the frequency components of the sound that make up the second notification sound NS2, but are generally higher frequency. The following explanation assumes that all frequency components of the sound constituting the first notification sound NS1 are higher in frequency than the frequency components of the sound constituting the second notification sound NS2.

[0050] Generally, higher-pitched sounds attract more attention than lower-pitched sounds. Therefore, for risk objects located in the first area A1, i.e., risk objects located in front of vehicle M, a high-pitched notification sound can be emitted to more quickly draw the driver's attention. Conversely, for risk objects located in the second area A2, i.e., risk objects located behind vehicle M, a low-pitched notification sound can be emitted to avoid bothering the driver with excessive notifications.

[0051] Conversely, the first notification sound may be lower in pitch than the second notification sound. As mentioned earlier, if the driver is aware of whether a higher-pitched notification sound is emitted for the front or rear, and which one for the rear, they can intuitively grasp the location of the risk object.

[0052] [Integer harmonics, etc.] The notification control unit 120 may output to the speaker unit 60 a sound as the first notification sound NS1, which includes a first reference tone NS1-b and a first additional tone NS1-p (p=1, 2, ...) that is an integer harmonic of the first reference tone NS1-b. The notification control unit 120 may also output to the speaker unit 60 a sound as the second notification sound NS2, which includes a second reference tone NS2-b and a second additional tone NS2-q (q=1, 2, ...) with a frequency different from 1 / n times the frequency of the second reference tone NS2-b (where n is a natural number). The second additional tone NS2-q may also be a sound that does not correspond to an integer harmonic of the second reference tone NS2-b. In this case, the first reference tone NS1-b is an example of "reference tone A", and the second reference tone NS2-b is an example of "reference tone B".

[0053] Figure 7 is a diagram illustrating the configuration of the first notification tone NS1 and the second notification tone NS2. The first reference tone NS1-b is, for example, a higher pitch than the second reference tone NS2-b, but it may be a tone of the same pitch (same frequency). In the illustrated example, the first notification tone NS1 includes the first additional tone NS1-1 and the first additional tone NS1-2. Since the first additional tone NS1-1 and the first additional tone NS1-2 are integer harmonics of the first reference tone NS1-b, they are higher pitches than the first reference tone NS1-b. The second notification tone NS2 also includes the second additional tone NS2-1 and the second additional tone NS2-2. The sound with a frequency different from 1 / n times the frequency of the second reference tone NS2-b can be either a sound higher than the second reference tone NS2-b or a sound lower than the second reference tone NS2-b. However, the notification control unit 120 outputs only a sound lower than the second reference tone NS2-b as the second additional tone NS2-q to the speaker unit 60.

[0054] Sounds composed of a reference tone and integer harmonics are clearly audible to the human ear, while sounds whose constituent tones are not integer harmonics sound disjointed to the human ear. By composing the first notification tone NS1 and the second notification tone in this manner, for risk objects located in the first region A1, i.e., risk objects located in front of vehicle M, a notification tone containing the reference tone and its integer harmonics can be emitted to more quickly draw the driver's attention. Furthermore, for risk objects located in the second region A2, i.e., risk objects located behind vehicle M, a notification tone whose constituent tones are not integer harmonics can be emitted to avoid bothering the driver with excessive notifications.

[0055] Conversely, the notification control unit 120 may output to the speaker unit 60 a sound as a first notification sound that includes a first reference sound and a first additional sound with a frequency different from 1 / n times the frequency of the first reference sound. The notification control unit 120 may also output to the speaker unit 60 a sound as a second notification sound that includes a second reference sound and a second additional sound that is an integer harmonic of the second reference sound. In this case, the first reference sound is an example of "reference sound B," and the second reference sound is an example of "reference sound A." In both the above and this case, if the driver recognizes whether the sound containing integer harmonics is output for the front or rear, and whether the sound containing an additional sound with a frequency different from 1 / n times the frequency of the reference sound is output for the rear, they can intuitively grasp the location of the risk object.

[0056] The notification control unit 120 may perform the following processing based on the above. For example, in the first embodiment, if a risk object is present in the first region A1, the notification control unit 120 may vary the characteristics of the first additional sound NS1-p based on the position of the risk object within the first region A1. More specifically, if a risk object is present in the first region A1, the notification control unit 120 may increase the variety of frequencies of the first additional sound NS1-p as the direction of the risk object as seen from the vehicle M is closer to the direction of travel of the moving object. Figure 8 is a diagram illustrating an example of increasing the variety of frequencies of the first additional sound NS1-p. For example, the notification control unit 120 may increase the number of frequencies of the first additional sound NS1-p when the risk object RO is located in the first region A1, specifically in the first central region A1-c, which is located in the center of the first region A1 in the width direction of the vehicle M, compared to when the risk object RO is located in the first left region A1-l or the first right region A1-r, which are located on either side of the first central region A1-c. The figure shows an example in which the speaker unit 60 outputs the first additional sounds NS1-1, NS1-2, and NS1-3 when the risk object RO is located in the first central region A1-c, and outputs the first additional sound NS1-1 to the speaker unit 60 when the risk object RO is located in the first right region A1-r. The notification control unit 120 may classify the position of the risk object RO in finer increments, rather than being limited to the divisions such as the first central region A1-c, the first left region A1-l, and the first right region A1-r, and may change the type of frequency of the first additional sound NS1-p.

[0057] Furthermore, the notification control unit 120 may perform the following processing. For example, in the first embodiment, if a risk object is located in the second region A2, the notification control unit 120 may change the mode of the second additional sound NS2-q based on the position of the risk object within the second region A2. More specifically, when a risk object is located in the second region A2, the notification control unit 120 may increase the volume of the second additional sound NS2-q as the direction of the risk object as seen from the vehicle M is closer to the opposite direction of the vehicle M's direction of travel. Figure 9 is a diagram illustrating an example of increasing the volume of the second additional sound NS2-q. For example, the notification control unit 120 may increase the volume of the second additional sound NS2-q when the risk object RO is located in the second central region A2-c, which is the central part of the second region A2 in the width direction of the moving object, compared to when the risk object RO is located in the second left region A2-l or the second right region A2-r, which are on either side of the second central region A2-c. The notification control unit 120 may also classify the position of the risk object RO in finer increments, rather than being limited to the divisions of the second central region A2-c, the second left region A2-l, and the second right region A2-r, and change the volume of the second additional sound NS2-q accordingly.

[0058] Alternatively, the notification control unit 120 may change the volume of the first additional sound NS1-p based on the position of the risk object RO within the first region A1 if the risk object RO is located in the first region A1, or change the frequency type of the second additional sound NS2-q based on the position of the risk object RO within the second region A2 if the risk object RO is located in the second region A2.

[0059] [Sound localization, control based on distance] The notification control unit 120 may control the output mode of one or both of the first notification sound NS1 and the second notification sound NS2 based on the distance between the vehicle M and the risk object, and the direction (angle) of the risk object as seen from the vehicle M, or both. Figure 10 is a diagram showing the definition of the parameters in the embodiment. In the following description, the direction of travel D of the vehicle M is... MThe direction coinciding with this is defined as 0 degrees, and angles are defined counterclockwise. Then, the direction of the boundary line between the first central region A1-c and the first left-side region A1-l is defined as θ1, the direction of the boundary line between the first left-side region A1-l and the second left-side region A2-l is defined as θ2, the direction of the boundary line between the second left-side region A2-l and the second central region A2-c is defined as θ3, the direction of the boundary line between the second central region A2-c and the second right-side region A2-r is defined as -θ3, the direction of the boundary line between the second right-side region A2-r and the first right-side region A1-r is defined as -θ2, and the direction of the boundary line between the first right-side region A1-r and the first central region A1-c is defined as -θ1. In this example, each region is set symmetrically, but each region may be set asymmetrically. Furthermore, the distance between vehicle M and risk object RO is defined as Drisk, and the direction of risk object RO as seen from vehicle M (assuming the above definitions) is defined as θrisk.

[0060] The notification control unit 120 may determine the position for localizing the sound image of the first notification sound NS1 based on the position of the risk object within the first region A1 if the risk object is located in the first region A1, and may determine the position for localizing the sound image of the second notification sound NS2 based on the position of the risk object within the second region A2 if the risk object is located in the second region A2.

[0061] Specifically, the notification control unit 120 localizes the sound image by determining the volume of sound output by each speaker based on the following formula. In the following description, SL-FR represents the volume of sound output by speaker 60FR, SL-FL represents the volume of sound output by speaker 60FL, SL-RR represents the volume of sound output by speaker 60RR, and SL-RL represents the volume of sound output by speaker 60RL. Also, Lbs is the reference volume. (a) If the risk object is in the first central region A1-c SL-FR = Lbs × (θ1 - θrisk) / 2θ1 SL - FL = Lbs × (θrisk + θ1) / 2θ1 SL-RR=SL-RL=0 (b) If the risk object is in the first left-hand region A1-l SL-FL=Lbs×(θ2-θrisk) / (θ2-θ1) SL-RL=Lbs×(θrisk-θ1) / (θ2-θ1) SL-FR=SL-RR=0 (c) If the risk object is in the second left-hand region A2-l SL-FL=Lbs×(θ3-θrisk) / (θ3-θ2) SL-RL=Lbs×(θrisk-θ2) / (θ3-θ2) SL-FR=SL-RR=0 (d) If the risk object is in the second central region A2-c SL-RR=Lbs×{θrisk+θ3+2×(180-θ3)} / {2×(180-θ3)} SL-RL=Lbs×(-θrisk+θ3) / {2×(180-θ3)} SL-FR=SL-FL=0 (e) If the risk object is in the second right-hand region A2-r SL-FR=Lbs×(θ3+θrisk) / (θ3-θ2) SL-RR=Lbs×(-θrisk-θ2) / (θ3-θ2) SL-FL=SL-RL=0 (f) If the risk object is in the first right-hand region A1-r SL-FR=Lbs×(θ2+θrisk) / (θ2-θ1) SL-RR=Lbs×(-θrisk-θ1) / (θ2-θ1) SL-FL=SL-RL=0

[0062] By localizing the sound image in this way, the driver can be intuitively informed of the location of the risk object RO, particularly in the lateral direction.

[0063] [Notification sound control based on direction and proximity] The notification control unit 120 may control the tempo and volume of the first notification sound NS1 or the second notification sound NS2 based on the direction θrisk and distance Drisk of the risk object RO, as follows:

[0064] For example, if the risk object is in the first region A1, the notification control unit 120 determines that the direction of the risk object RO as seen from the vehicle M is the direction of travel D of the vehicle M. M The closer it is, the slower the tempo of the first notification sound NS1 will be, and if the risk object RO is in the second region A2, the direction of the risk object RO as seen from vehicle M will be the direction of travel D of vehicle M. M The closer to the opposite direction, the faster the tempo of the second notification sound NS2 is set. Figure 11 is a diagram illustrating an example of varying the tempo of the notification sound. As shown in the figure, the notification control unit 120 may set the tempo of the first notification sound NS1 slower when the risk object RO is in the first central region A1-c compared to when the risk object RO is in the first left region A1-l or the first right region A1-r. This control is designed to avoid excessive notification, as drivers often pay attention to the front of the vehicle M. It is also based on the consideration that using a high-pitched sound containing integer harmonics for notification to the front is effective in attracting attention. This reduces the stress on the driver caused by outputting the first notification sound NS1.

[0065] The notification control unit 120 may set the tempo of the second notification sound NS2 faster when the risk object RO is in the second central region A2-c compared to when the risk object RO is in the second left region A2-l or the second right region A2-r. Furthermore, assuming the distance Drisk is the same, the notification control unit 120 may set the slowest tempo of the second notification sound NS2 when the risk object RO is in the second region A2 to be faster than the fastest tempo of the first notification sound NS1 when the risk object RO is in the first region A1.

[0066] The notification control unit 120 may classify the location of the risk object RO in finer increments, rather than being limited to the divisions such as the first central region A1-c, the first left region A1-l, the first right region A1-r, the second central region A2-c, the second left region A2-l, and the second right region A2-r, and may change the tempo of the first notification sound NS1 or the second notification sound NS2.

[0067] The notification control unit 120 may increase the tempo of the first notification sound NS1 or the second notification sound NS2 as the distance Drisk decreases, regardless of which region the risk object RO is located in. The notification control unit 120 may also increase the volume of the first notification sound NS1 or the second notification sound NS2 as the distance Drisk decreases. Figure 12 shows an example of volume control according to the distance Drisk. In the figure, Dth is the distance at which the output control of the notification sound begins, and may coincide with the distance from the vehicle M to the outer edge of regions A1 or A2.

[0068] Figure 13 is a flowchart showing an example of the processing flow performed by the driver assistance device 100. First, the recognition unit 110 recognizes the risk object (step S100). Next, the recognition unit 110 identifies the region where the risk object exists, the distance Drisk from the risk object, and the direction θrisk of the risk object (step S102).

[0069] Next, the notification control unit 120 determines the configuration, tempo, and volume of the notification sound (step S104), and causes the speaker unit 60 to output the notification sound (step S106).

[0070] Next, the recognition unit 110 determines whether the location of the risk object is outside the notification range (including not being recognized) (step S108). Being outside the notification range means that the distance from the vehicle M is greater than Dth as described above. If the location of the risk object is outside the notification range, the process in this flowchart ends. If the location of the risk object is not outside the notification range, the process returns to step S102.

[0071] According to the first embodiment described above, the position or direction of risky objects present around the moving object (vehicle M) can be intuitively recognized by the occupants.

[0072] <Second Embodiment> The second embodiment will now be described. The driving assistance device of the second embodiment does not perform the configuration change control of the first notification sound NS1 within the first region A1 and the configuration change control of the second notification sound NS2 within the second region A2, as described with reference to Figures 8 and 9 in the first embodiment, but instead makes the configuration of the notification sound different depending on whether the risk object RO is located in the first region A1 or the second region A2. In other words, the driving support device of the second embodiment outputs a first notification sound NS1 to the speaker unit 60, which includes a first reference tone NS1-b and a first additional tone NS1-p (p=1, 2, ...) that is an integer harmonic of the first reference tone NS1-b, when the risk object RO is located in the first region A1. When the risk object RO is located in the second region A2, the device outputs a second notification sound NS2 to the speaker unit 60, which includes a second reference tone NS2-b and a second additional tone NS2-q (q=1, 2, ...) with a frequency different from 1 / n times the frequency of the second reference tone NS2-b. Other controls may be the same as in the first embodiment. This makes it possible to achieve the same effects as in the first embodiment with simpler control.

[0073] <Third Embodiment> The third embodiment will now be described. The driver assistance system of the third embodiment is installed in an autonomous vehicle. Figure 14 is a diagram showing the equipment installed in the vehicle M, centering on the driver assistance system 100A according to the third embodiment. The driver assistance system 100A is installed in the vehicle MA together with the autonomous driving control system 150. Output data from the camera 10, radar system 12, LIDAR 14, object recognition system 16, and vehicle sensor 40 are also input to the autonomous driving control system 150. In addition to these, output data from the navigation system and MPU (Micro Processing Unit) may also be input to the autonomous driving control system 150. The autonomous driving control system 150 recognizes the surrounding conditions of the vehicle MA, recognizes the area where it can travel while avoiding contact with obstacles, generates a target trajectory (with a velocity element) for the vehicle MA, and controls the driving force output device 200, brake system 210, steering system 220, etc. so that the vehicle MA can travel along the target trajectory. The driving force output device 200 includes the engine and driving motor. Since details about autonomous driving technology are publicly available in various publications, further explanation will be omitted.

[0074] The automated driving control device 150 is further connected to a steering grip sensor 160 and an in-cabin camera 170. The steering grip sensor 160 detects whether the driver is gripping the steering wheel, which is the steering control element, and the in-cabin camera 170 captures an image of the driver's head from the front. The automated driving control device 150 determines whether the driver is gripping the steering wheel (hands-on) by referring to the output of the steering grip sensor 160, and determines whether the driver is looking in the direction of travel of the vehicle MA (eyes-on) by analyzing the image captured by the in-cabin camera 170. Based on the surrounding environment of the vehicle MA (type of road being traveled) and the speed of the vehicle MA, the automated driving control device 150 allows either or both of the following: releasing the steering wheel (hands-off) and looking away from the direction of travel of the vehicle MA (eyes-off). This permitted state is sometimes referred to as the automated driving level. If an unacceptable condition occurs (for example, if the driver takes their hands off the steering wheel while hands-on), the automatic driving control device 150 notifies the driver to resolve the condition, and if the condition is not resolved, it switches to manual driving. The automatic driving control device 150 outputs information regarding the automatic driving level to the driver assistance device 100A.

[0075] The driver assistance device 100A refers to information regarding the level of autonomous driving and automatically stops operating if both hands-off and eyes-off driving are permitted. In this state, the driver is temporarily relieved of the obligation to monitor the area around the vehicle MA, and the autonomous driving control device 150 takes control to avoid contact with hazardous objects. Other functions are the same as in the first or second embodiment.

[0076] According to the third embodiment described above, it is possible to perform control that is highly compatible with autonomous driving.

[0077] <Fourth Embodiment> The fourth embodiment will now be described. The driver assistance device of the fourth embodiment is mounted on a motorcycle in which the driver wears a helmet while riding. Figure 15 is a diagram showing the equipment mounted on the motorcycle MB, centering on the driver assistance device 100B according to the fourth embodiment. Instead of controlling the speaker unit 60, the driver assistance device 100B instructs the wireless communication device 80 to transmit instruction information regarding notification sounds. The wireless communication device 80 communicates based on a communication standard such as Bluetooth (registered trademark).

[0078] The helmet 300 worn by the driver is equipped with a receiver 310, a left ear speaker 320-L, and a right ear speaker 320-R. Based on instruction information received from the wireless communication device 80, the receiver 310 outputs a notification sound to one or both of the left ear speaker 320-L and the right ear speaker 320-R.

[0079] Except for the difficulty in expressing the front-to-back direction when controlling sound image localization, this is the same as the first or second embodiment. That is, the recognition unit 110 and the notification control unit 120 perform the same processing as described in the first or second embodiment to determine the configuration, tempo, and volume of the first notification sound NS1 and the second notification sound NS2, and send instruction information to the wireless communication device 80 to output the notification sound to the left ear speaker 320-L and / or the right ear speaker 320-R accordingly.

[0080] In the fourth embodiment, instead of speakers installed inside the helmet, wireless earphones or the like may be used.

[0081] According to the fourth embodiment described above, although the degree of freedom for sound image localization is reduced, the same effects as those of the first or second embodiment can be achieved in other respects.

[0082] The embodiments described above can be expressed as follows. A memory device that stores the program, The system comprises a hardware processor associated with the storage device, The hardware processor executes the program stored in the memory device, Based on the output of the object detection device, risky objects present around the moving object are recognized. Based on the location of the aforementioned risk object, a notification sound is output to at least one speaker. When outputting the notification sound, if the risk object is located in a first region including the front region of the moving body, the first notification sound is output to at least one speaker; if the risk object is located in a second region including the rear region of the moving body, the second notification sound is output to at least one speaker. The first notification sound is a sound with a different pitch from the second notification sound. Driving assistance system.

[0083] The embodiment described above can also be expressed as follows: A memory device that stores the program, The system comprises a hardware processor associated with the storage device, The hardware processor executes the program stored in the memory device, The hardware processor executes the program stored in the memory device, Based on the output of the object detection device, risky objects present around the moving object are recognized. Based on the location of the aforementioned risk object, a notification sound is output to at least one speaker. When outputting the notification sound, if the risk object is located in a first region including the front region of the moving body, the first notification sound is output to at least one speaker; if the risk object is located in a second region including the rear region of the moving body, the second notification sound is output to at least one speaker. Either the first notification sound or the second notification sound is a sound that includes a reference sound A and integer harmonics of the reference sound A, and the other of the first notification sound or the second notification sound is a sound that includes a reference sound B and an additional sound with a frequency different from 1 / n times the frequency of the reference sound B (where n is a natural number). Driving assistance system.

[0084] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0085] 10 Cameras 12 Radar equipment 14 LIDAR 16 Object recognition device 40 Vehicle Sensors 60 speaker units 80 Wireless communication devices 100, 100A, 100B Driving Assistance Devices 110 Recognition part 120 Notification Control Unit 150 Automatic Driving Control System 300 helmets

Claims

1. A recognition unit that recognizes risky objects present around a moving object based on the output of an object detection device, The system includes a notification control unit that causes at least one speaker to output a notification sound based on the location of the risk object, The notification control unit causes the at least one speaker to output a first notification sound when the risk object is located in a first region including the front region of the moving body, and causes the at least one speaker to output a second notification sound when the risk object is located in a second region including the rear region of the moving body. The first notification sound is a sound with a different pitch from the second notification sound. The first notification sound is a higher pitched sound than the second notification sound. The first notification tone includes a first reference tone and a first additional tone which is an integer harmonic of the first reference tone. The second notification sound includes a second reference sound and a second additional sound having a frequency different from that of 1 / n times the frequency of the second reference sound (where n is a natural number). The notification control unit, when the risk object is located in front of the moving body, slows down the tempo of the first notification sound as the direction of the risk object as seen from the moving body is closer to the direction of movement of the moving body. Driving assistance system.

2. The notification control unit shall If the risk object is located in front of the moving body, the closer the direction of the risk object is to the direction of travel of the moving body, the slower the tempo of the first notification sound will be. If the risk object is located behind the moving body, the closer the direction of the risk object is to the opposite direction of the moving body's movement, the faster the tempo of the second notification sound will be. The driving support device according to claim 1.

3. A recognition unit that recognizes risky objects present around a moving object based on the output of an object detection device, The system includes a notification control unit that causes at least one speaker to output a notification sound based on the location of the risk object, The notification control unit causes the at least one speaker to output a first notification sound when the risk object is located in a first region including the front region of the moving body, and causes the at least one speaker to output a second notification sound when the risk object is located in a second region including the rear region of the moving body. The first notification sound is a sound with a different pitch from the second notification sound. The first notification sound is a higher pitched sound than the second notification sound. The first notification tone includes a first reference tone and a first additional tone which is an integer harmonic of the first reference tone. The second notification sound includes a second reference sound and a second additional sound having a frequency different from that of 1 / n times the frequency of the second reference sound (where n is a natural number). The notification control unit slows down the tempo of the first notification sound when the risk object is in front of the moving object, compared to when the risk object is to the right or left of the front. Driving assistance system.

4. The aforementioned at least one speaker includes a plurality of speakers, The notification control unit determines, if the risk object is in the first region, the position for localizing the sound image of the first notification sound based on the position of the risk object within the first region, and if the risk object is in the second region, the position for localizing the sound image of the second notification sound based on the position of the risk object within the second region. The driving support device according to any one of claims 1 to 3.

5. The aforementioned risk object is a traffic participant, A driving support device according to any one of claims 1 to 4.

6. Computers Based on the output of the object detection device, risky objects present around the moving object are recognized. Based on the location of the aforementioned risk object, a notification sound is output to at least one speaker. If the risk object is located in a first region including the front region of the moving body, a first notification sound is output from at least one speaker; if the risk object is located in a second region including the rear region of the moving body, a second notification sound is output from at least one speaker. The first notification sound is a sound with a different pitch from the second notification sound. The first notification sound is a higher pitched sound than the second notification sound. The first notification tone includes a first reference tone and a first additional tone which is an integer harmonic of the first reference tone. The second notification sound includes a second reference sound and a second additional sound having a frequency different from that of 1 / n times the frequency of the second reference sound (where n is a natural number). If the risk object is located in front of the moving body, the tempo of the first notification sound is slowed down as the direction of the risk object as seen from the moving body is closer to the direction of movement of the moving body. Driving assistance methods.

7. On the computer, Based on the output of the object detection device, it recognizes risky objects present around the moving object. Based on the location of the aforementioned risk object, a notification sound is output to at least one speaker. If the risk object is located in a first region including the front region of the moving body, a first notification sound is output from at least one speaker; if the risk object is located in a second region including the rear region of the moving body, a second notification sound is output from at least one speaker. The first notification sound is a sound with a different pitch from the second notification sound. The first notification sound is a higher pitched sound than the second notification sound. The first notification tone includes a first reference tone and a first additional tone which is an integer harmonic of the first reference tone. The second notification sound includes a second reference sound and a second additional sound having a frequency different from that of 1 / n times the frequency of the second reference sound (where n is a natural number). If the risk object is located in front of the moving body, the tempo of the first notification sound is slowed down as the direction of the risk object as seen from the moving body is closer to the direction of movement of the moving body. program.

8. Computers Based on the output of the object detection device, risky objects present around the moving object are recognized. Based on the location of the aforementioned risk object, a notification sound is output to at least one speaker. If the risk object is located in a first region including the front region of the moving body, a first notification sound is output from at least one speaker; if the risk object is located in a second region including the rear region of the moving body, a second notification sound is output from at least one speaker. The first notification sound is a sound with a different pitch from the second notification sound. The first notification sound is a higher pitched sound than the second notification sound. The first notification tone includes a first reference tone and a first additional tone which is an integer harmonic of the first reference tone. The second notification sound includes a second reference sound and a second additional sound having a frequency different from that of 1 / n times the frequency of the second reference sound (where n is a natural number). When the risk object is located in front of the moving body, the tempo of the first notification sound is slowed down compared to when the risk object is located to the right or left of the front. Driving assistance methods.

9. On the computer, Based on the output of the object detection device, it recognizes risky objects present around the moving object. Based on the location of the aforementioned risk object, a notification sound is output to at least one speaker. If the risk object is located in a first region including the front region of the moving body, a first notification sound is output from at least one speaker; if the risk object is located in a second region including the rear region of the moving body, a second notification sound is output from at least one speaker. The first notification sound is a sound with a different pitch from the second notification sound. The first notification sound is a higher pitched sound than the second notification sound. The first notification tone includes a first reference tone and a first additional tone which is an integer harmonic of the first reference tone. The second notification sound includes a second reference sound and a second additional sound having a frequency different from that of 1 / n times the frequency of the second reference sound (where n is a natural number). When the risk object is located in front of the moving body, the tempo of the first notification sound is slowed down compared to when the risk object is located to the right or left of the front. program.

Images