Driving support device, driving support method, and storage medium

Abstract

Provided is a driving support device, a driving support method, and a storage medium that enable a passenger to visually recognize the position or direction of a risk object present in the periphery of a moving body. The driving support device includes an identification unit that identifies a risk object present in the periphery of a moving body based on the output of an object detection device, and a notification control unit that causes at least one speaker to output a notification sound based on the position of the risk object, the notification control unit causing the at least one speaker to output a first notification sound when the risk object is present in a first region including a front region of the moving body, and to output a second notification sound when the risk object is present in a second region including a rear region of the moving body, the first notification sound being a sound having a different height from the second notification sound.

Description

Technical Field

[0001] This invention relates to a driving support device, a driving support method, and a storage medium. Background Technology

[0002] Previously, techniques were known for using sound to attract the attention of a driver (occupant) to an object of attention present around a moving body. Among these techniques, there is one that uses sound localization to emit sound at the location and direction from which attention is to be attracted (Patent Document 1).

[0003] Prior technology literature

[0004] [Patent Documents]

[0005] Patent Document 1: Japanese Patent No. 6587776 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] However, acoustic localization within a moving vehicle sometimes does not work ideally. For example, in the case of a vehicle, sounds emitted near the rear seats may be reflected by the windshield, sometimes making the driver feel as if they are heard from the front. Thus, in conventional technologies, occupants sometimes cannot visually identify the location or direction of hazardous objects present around the moving vehicle.

[0008] The present invention was made in consideration of such circumstances, and one of its objectives is to provide a driving support device, driving support method, and storage medium that enables occupants to visually identify the location or orientation of hazardous objects present around a moving body.

[0009] Solution for solving the problem

[0010] The driving support device, driving support method, and storage medium of the present invention adopt the following structure.

[0011] (1): A driving support device according to one aspect of the present invention includes: an identification unit that identifies a risky object existing around a moving body based on the output of an object detection device; and a notification control unit that causes at least one speaker to output a notification tone based on the position of the risky object, wherein the notification control unit causes the at least one speaker to output a first notification tone when the risky object is present in a first region including a region in front of the moving body, and causes the at least one speaker to output a second notification tone when the risky object is present in a second region including a region behind the moving body, wherein the first notification tone is a sound with a different pitch than the second notification tone.

[0012] (2): In the above scheme (1), the first notification tone is a higher pitch than the second notification tone.

[0013] (3): In the above (1) scheme, the first notification tone includes a first reference tone and a first additional tone that is an integer multiple of the first reference tone.

[0014] (4): In the above scheme (1), the second notification tone includes a second reference tone and a second additional tone whose frequency is different from 1 / n times the frequency of the second reference tone, where n is a natural number.

[0015] (5): Another aspect of the driving support device of the present invention includes: an identification unit that identifies a risky object existing around a moving body based on the output of an object detection device; and a notification control unit that causes at least one speaker to output a notification tone based on the position of the risky object. The notification control unit causes the at least one speaker to output a first notification tone when the risky object is present in a first region including a region in front of the moving body, and causes the at least one speaker to output a second notification tone when the risky object is present in a second region including a region behind the moving body. Either the first notification tone or the second notification tone is a sound including a reference tone A and an integer multiple of the reference tone A, and the other of the first notification tone and the second notification tone is a sound including a reference tone B and an additional tone whose frequency is different from 1 / n times the frequency of the reference tone B, where n is a natural number.

[0016] (6): In the above scheme (5), the first notification tone includes a first reference tone and a first additional tone that is an integer multiple of the first reference tone.

[0017] (7): In the above scheme (3) or (6), the notification control unit makes the form of the first additional sound different based on the different positions of the risk object in the first area when the risk object exists in the first area.

[0018] (8): In the above (7) scheme, when the risky object is present in the first area, the notification control unit increases the variety of the first additional sound frequency as the direction of the risky object is closer to the direction of travel of the moving body when observed from the moving body.

[0019] (9): In the above (7) scheme, when the risky object exists in the first central region of the first region located in the width direction of the moving body, the notification control unit increases the types of frequencies of the first additional sound compared to the case where the risky object exists in the first left region or the first right region located on the left and right sides of the first central region.

[0020] (10): In the above scheme (5), the second notification tone includes a second reference tone and a second additional tone whose frequency is different from 1 / n times the frequency of the second reference tone.

[0021] (11): In the above scheme (4) or (10), the notification control unit makes the form of the second additional sound different based on the different positions of the risk object in the second area when the risk object exists in the second area.

[0022] (12): In the above (11) scheme, when the risky object is present in the second area, the notification control unit increases the volume of the second additional sound as the direction of the risky object is closer to the opposite direction of the direction of travel of the moving body when viewed from the moving body.

[0023] (13): In the above-described (11) scheme, when the risky object exists in the second central region of the second region located in the width direction of the moving body, the notification control unit increases the volume of the second notification sound compared to the case where the risky object exists in the second left region or the second right region located on the left or right sides of the second central region.

[0024] (14): In the above scheme (1) or (5), when the risky object is present in the first area, the notification control unit slows down the beat of the first notification sound as the direction of the risky object is closer to the direction of travel of the moving body when viewed from the moving body.

[0025] (15): In the above scheme (1) or (5), when the risky object is present in the second area, the notification control unit speeds up the beat of the second notification sound the more the direction of the risky object is closer to the opposite direction of the direction of travel of the moving body when viewed from the moving body.

[0026] (16): In the above scheme (1) or (5), the smaller the distance between the moving body and the risky object, the faster the notification control unit speeds up the beat of the first notification tone or the second notification tone.

[0027] (17): In the above scheme (1) or (5), the at least one loudspeaker includes a plurality of loudspeakers, and the notification control unit determines the location to which the sound image of the first notification sound is located based on the location of the risk object in the first area when the risk object exists in the first area, and determines the location to which the sound image of the second notification sound is located based on the location of the risk object in the second area when the risk object exists in the second area.

[0028] (18): In the above scheme (1) or (5), the risk object is a traffic participant.

[0029] (19): Another aspect of the driving support method of the present invention causes a computer mounted on a mobile body to perform the following processing: based on the output of an object detection device, identify a risky object existing around the mobile body; based on the position of the risky object, cause at least one speaker to output a notification tone; when the notification tone is output, if the risky object is in a first region including the area in front of the mobile body, the computer causes at least one speaker to output a first notification tone; if the risky object is in a second region including the area behind the mobile body, the computer causes at least one speaker to output a second notification tone; the first notification tone is a sound with a different pitch than the second notification tone.

[0030] (20): Another aspect of the driving support method of the present invention causes a computer mounted on a mobile body to perform the following processing: based on the output of an object detection device, identify a risky object existing around the mobile body; based on the position of the risky object, cause at least one speaker to output a notification tone; when the notification tone is output, if the risky object is in a first region including a region in front of the mobile body, the computer causes the at least one speaker to output a first notification tone; if the risky object is in a second region including a region behind the mobile body, the computer causes the at least one speaker to output a second notification tone; either the first notification tone or the second notification tone is a sound including a reference tone A and an integer multiple of the reference tone A; the other of the first notification tone and the second notification tone is a sound including a reference tone B and an additional tone whose frequency is different from 1 / n times the frequency of the reference tone B, where n is a natural number.

[0031] (21): In another aspect of the present invention, the storage medium stores a program, wherein the program causes a computer mounted on a mobile body to perform the following processing: based on the output of an object detection device, identify a risky object existing around the mobile body; based on the position of the risky object, cause at least one speaker to output a notification tone; when the notification tone is output, if the risky object is in a first region including the area in front of the mobile body, the computer causes the at least one speaker to output a first notification tone; if the risky object is in a second region including the area behind the mobile body, the computer causes the at least one speaker to output a second notification tone, wherein the first notification tone is a sound with a different pitch than the second notification tone.

[0032] (22): In another aspect of the present invention, the storage medium stores a program, wherein the program causes a computer mounted on a mobile body to perform the following processing: based on the output of an object detection device, identify a risky object existing around the mobile body; based on the position of the risky object, cause at least one speaker to output a notification tone; when the notification tone is output, if the risky object is in a first region including a region in front of the mobile body, the computer causes the at least one speaker to output a first notification tone; if the risky object is in a second region including a region behind the mobile body, the computer causes the at least one speaker to output a second notification tone; either the first notification tone or the second notification tone is a sound including a reference tone A and an integer multiple of the reference tone A; the other of the first notification tone and the second notification tone is a sound including a reference tone B and an additional tone whose frequency is different from 1 / n times the frequency of the reference tone B, wherein n is a natural number.

[0033] Invention Effects

[0034] According to the above schemes (1) to (22), the occupants can intuitively identify the location or direction of the risky objects around the moving body. Attached Figure Description

[0035] Figure 1 This diagram shows a device mounted on a vehicle M, centered on the driving support device 100 of the first embodiment.

[0036] Figure 2 This is a structural diagram of speaker unit 60.

[0037] Figure 3 This is a diagram illustrating an example of the hardware structure of the speaker unit 60.

[0038] Figure 4 This is another example of the hardware structure of the speaker unit 60.

[0039] Figure 5 This is a diagram used to illustrate the first region A1 and the second region A2.

[0040] Figure 6 This is a diagram illustrating that the first notification tone NS1 is higher than the second notification tone NS2.

[0041] Figure 7 This is a diagram used to illustrate the structure of the first notification tone NS1 and the second notification tone NS2.

[0042] Figure 8 This is a diagram illustrating one example of the types of frequencies that can be increased by adding the first additional tone NS1-p.

[0043] Figure 9 This is a diagram illustrating an example of increasing the volume of the second additional tone NS2-q.

[0044] Figure 10 This is a diagram illustrating the definition of parameters in the implementation method.

[0045] Figure 11 This is a diagram used to illustrate an example of how different beats can be used for notification sounds.

[0046] Figure 12 This is a diagram illustrating an example of volume control corresponding to distance from Drisk.

[0047] Figure 13 This is a flowchart illustrating an example of the processing flow performed by the driving support device 100.

[0048] Figure 14 This diagram shows the equipment mounted on the vehicle M, centered on the driving support device 100A of the third embodiment.

[0049] Figure 15 This diagram shows a device mounted on a two-wheeled vehicle MB, centered on the driving support device 100B of the fourth embodiment.

[0050] Explanation of reference numerals in the attached figures:

[0051] 10 cameras

[0052] 12 Radar Devices

[0053] 14 LIDAR

[0054] 16 Object recognition devices

[0055] 40 Vehicle Sensors

[0056] 60 speaker units

[0057] 80 Wireless communication devices

[0058] 100, 100A, 100B Driving Support Devices

[0059] 110 Identification Department

[0060] 120 Notification Control Department

[0061] 150 Automatic driving control device

[0062] 300 Helmet. Detailed Implementation

[0063] Hereinafter, with reference to the accompanying drawings, embodiments of the driving support device, driving support method, and storage medium of the present invention will be described. The driving support device is a device that supports driving a mobile body. A mobile body includes vehicles with three or four wheels, two-wheeled vehicles, miniature mobile bodies, etc., and can include all mobile bodies for a person (driver) to ride in. In the first and third embodiments described below, the mobile body is assumed to be a four-wheeled vehicle, and the vehicle equipped with the driving support device is referred to as vehicle M. Furthermore, in the fourth embodiment, the mobile body is assumed to be a two-wheeled vehicle.

[0064] [structure]

[0065] Figure 1 This diagram illustrates a device mounted on a vehicle M, centered on the driver support device 100 of the first embodiment. The vehicle M may be any of the following: a motor vehicle powered by an internal combustion engine such as a diesel engine or a gasoline engine; an electric motor vehicle powered by an electric motor; or a hybrid motor vehicle that combines an internal combustion engine and an electric motor.

[0066] The vehicle M may include, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) system 14, an object recognition device 16, a vehicle sensor 40, a speaker unit 60, a display device 70, and a driver support device 100. In this embodiment, structures used to move the vehicle M, such as driving controls, drive units like engines or motors, steering systems, and braking systems, are omitted from illustration and description, but these structures can be mounted on the vehicle M.

[0067] Camera 10 is, for example, a digital camera utilizing a solid-state imaging element such as CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor). Camera 10 is mounted anywhere on the vehicle M equipped with vehicle system 1. When taking pictures of the front, camera 10 is mounted on the upper part of the windshield, the back of the rearview mirror inside the vehicle, etc. Camera 10, for example, periodically and repeatedly takes pictures of the surroundings of vehicle M. Camera 10 can also be a stereo camera. Alternatively, multiple cameras 10 with different detection directions can be mounted on vehicle M.

[0068] Radar device 12 radiates millimeter-wave or other radio waves around the vehicle M and detects the radio waves reflected by objects (reflected waves) to detect at least the position (distance and orientation) of the objects. Radar device 12 can be installed at any location on the vehicle M. Radar device 12 can also detect the position and speed of objects using FM-CW (Frequency Modulated Continuous Wave) method. Alternatively, multiple radar devices 12 with different detection directions can be mounted on the vehicle M.

[0069] The LIDAR14 illuminates the periphery of vehicle M with light (or electromagnetic waves with wavelengths close to light) and measures the scattered light. The LIDAR14 determines the distance to the object based on the time from emission to reception. The illuminating light can be, for example, a pulsed laser. The LIDAR14 can be mounted at any location on vehicle M.

[0070] The object recognition device 16 analyzes the images captured by the camera 10 to identify objects. Furthermore, the object recognition device 16 performs sensor fusion processing on the detection results from some or all of the detections by the camera 10, radar device 12, and LIDAR 14 to identify the object's position (distance and angle), type, speed, etc. The object recognition device 16 outputs the recognition results to the driver support device 100. Alternatively, the object recognition device 16 may directly output the detection results from the camera 10, radar device 12, and LIDAR 14 to the driver support device 100.

[0071] The group of devices including camera 10, radar device 12, LIDAR 14, and object recognition device 16 is an example of an "object detection device." The "object detection device" can also have a simpler structure, such as a structure consisting of camera 10 and object recognition device 16, or camera 10 and radar device 12, mounted on vehicle M. Even without the object recognition device 16, the function of analyzing the images captured by camera 10 can be mounted on driver support device 100. Furthermore, the "object detection device" may also include an ultrasonic sensor.

[0072] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting angular velocity about the vertical axis, and an orientation sensor for detecting the orientation of the vehicle M.

[0073] Figure 2 This is a structural diagram of speaker unit 60. Speaker unit 60 includes, for example, four speakers: speakers 60FR, 60FL, 60RR, and 60RL. For example, speaker 60FR is located between the right end of the dashboard and the right door in the interior of vehicle M; speaker 60FL is located between the left end of the dashboard and the right door in the interior of vehicle M; speaker 60RR is located under the seat surface of the right rear seat in the interior of vehicle M; and speaker 60RL is located under the seat surface of the left rear seat in the interior of vehicle M. Speaker unit 60 may also include a pair of left and right speakers. Alternatively, a single speaker may be mounted in vehicle M instead of speaker unit 60.

[0074] Figure 3 This diagram illustrates an example of the hardware structure of the speaker unit 60. For example, control processors 62FR, 62FL, 62RR, and 62RL are installed on the speakers 60FR, 60FL, 60RR, and 60RL, respectively. The speakers are connected to the control processors via USB (Universal Serial Bus), and the control processors are connected to the driver support device 100 via a CAN (Controller Area Network) bus, etc.

[0075] Figure 4 This diagram illustrates another example of the hardware structure of the speaker unit 60. For instance, the speaker 60FR and 60FL are connected analogously, and the speaker 60FL is connected to the control processor 64FL via USB. Additionally, the speaker 60RR and 60RL are connected analogously, and the speaker 60RL is connected to the control processor 64RL via USB. Furthermore, the control processors 64FL and RL are connected to the driver support device 100 via a CAN bus, etc.

[0076] Thus, a control processor can be installed corresponding to each speaker, or a control processor can be installed for each plurality of speakers. The control processor causes the corresponding speaker to output a notification tone (including a first notification tone and a second notification tone) at the frequency indicated by the driving support device 100 described later.

[0077] The display device 70 is, for example, a touch panel, and is installed anywhere inside the vehicle interior of the vehicle M. The display device 70 accepts various operations performed on the driver support device 100 and displays images indicated by the driver support device 100, etc.

[0078] return Figure 1 The driving support device 100 includes, for example, an identification unit 110 and a notification control unit 120. These components are implemented by executing programs (software) through hardware processors such as CPUs (Central Processing Units). Some or all of these components can also be implemented using hardware (including circuitry) such as LSIs (Large Scale Integration), ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), or GPUs (Graphics Processing Units), or through the coordinated use of software and hardware. The program can be pre-saved in storage devices such as HDDs and flash memory (storage devices with non-transitory storage media), or it can be saved in removable storage media such as DVDs and CD-ROMs (non-transitory storage media), and installed in the storage device by mounting the storage media to a drive device.

[0079] [Risk Object]

[0080] The identification unit 110 identifies potential hazards around the vehicle M based on the output of at least a portion of the camera 10, radar device 12, LIDAR 14, and object recognition device 16. Potential hazards are, for example, external mobile objects, i.e., traffic participants or other moving bodies. Specifically, potential hazards include pedestrians, bicycles or vehicles carrying passengers, etc. Potential hazards may also include static obstacles, such as placed objects, parked vehicles, utility poles, etc. "Identifying potential hazards" means identifying the existence of potential hazards categorized as described above and their relative positions to the vehicle M. When the identification unit 110 receives information about the object's category from the object recognition device 16 along with information about the object's position, it can identify potential hazards by referring to the object's category, or it can identify potential hazards by analyzing images captured by the camera 10. Additionally, the identification unit 110 can also identify potential hazards based on the object's speed, electromagnetic reflectivity, etc.

[0081] [Notification Recipient Area]

[0082] The notification control unit 120 causes the speaker unit 60 (via the control processor) to output a notification tone based on the location of the hazardous object. The notification control unit 120 causes the speaker unit 60 to output a first notification tone when the hazardous object is located in a first area including the area in front of the vehicle M, and causes the speaker unit 60 to output a second notification tone when the hazardous object is located in a second area including the area behind the vehicle M. Figure 5 This is a diagram used to illustrate the first region A1 and the second region A2. In the diagram, D... M This refers to the direction of travel (forward direction, normal travel direction) of vehicle M. The direction of travel D of vehicle M. M The direction can be the direction of the vehicle M's central axis, the direction of the instantaneous movement vector, or the direction of the lane centerline of the road where the vehicle M exists. Region AF is the front region. For example, the front region AF is defined as the outer edges of lines extending from the left and right sides of the vehicle M towards the front, and extends a predetermined distance from the front end of the vehicle M to the front. Region AR is the rear region. For example, the rear region AR is defined as the outer edges of lines extending from the left and right sides of the vehicle M towards the rear, and extends a predetermined distance from the rear end of the vehicle M to the rear (this distance may be the same as or different from the predetermined distance in the front region AF). First region A1 is a region that at least includes the front region AF and extends from the side of the vehicle M to the front. Second region A2 is a region that at least includes the rear region AR and extends from the side of the vehicle M to the rear. First region A1 and second region A2 are set to not overlap. The boundary line between first region A1 and second region A2 passes through the reference point R of the vehicle M. M And it extends to the left and right sides of vehicle M. This boundary line does not need to be strictly aligned with the width direction of vehicle M; it can also be inclined forward or backward from a reference point of vehicle M. In the diagram, it is set to extend from the reference point R of vehicle M. M Line segments Lr and Ll pointing diagonally forward form the boundary between the first region A1 and the second region A2. Additionally, a certain degree of gap may exist between the first region A1 and the second region A2.

[0083] [Notification tone]

[0084] In the first embodiment, the first notification tone is a sound with a different pitch than the second notification tone. Therefore, if the driver can identify which direction (front or rear) is indicated by a higher notification tone and which by a lower one, they can intuitively grasp the location of the hazardous object.

[0085] As an example, the first notification tone is a sound that is higher than the second notification tone. "A sound that is higher than the second notification tone" means, for example, that all the frequency components of the sound constituting the first notification tone are higher than the frequency components of the sound constituting the second notification tone. Alternatively, it can mean that a portion of the frequency components of the sound constituting the first notification tone overlaps with the frequency components of the sound constituting the second notification tone, but is higher overall (for example, the weighted sum of the frequencies constituting the first notification tone by volume weighting is higher than the weighted sum of the frequencies constituting the second notification tone). Figure 6 This is a diagram illustrating that the first notification tone NS1 is higher than the second notification tone NS2. Figure 6 The diagram above shows the case where all the frequency components of the sound constituting the first notification tone NS1 are higher than the frequency components of the sound constituting the second notification tone NS2. Figure 6 The diagram below illustrates a case where a portion of the frequency components constituting the first notification tone NS1 overlaps with the frequency components constituting the second notification tone NS2, but is generally higher in frequency. In the following description, it is assumed that all frequency components constituting the first notification tone NS1 are higher than the frequency components constituting the second notification tone NS2.

[0086] Generally, higher-pitched sounds attract more attention than lower-pitched sounds. Therefore, by emitting a high-pitched notification sound for a risky object located in the first area A1, i.e., the front side of the vehicle M, the driver's attention can be drawn more quickly. Conversely, by emitting a low-pitched notification sound for a risky object located in the second area A2, i.e., the rear side of the vehicle M, the driver can be prevented from being annoyed by excessive notification.

[0087] It should be noted that, conversely, the first notification tone can also be lower than the second notification tone. As mentioned earlier, this is because if the driver can identify which direction (front or rear) has a higher notification tone and which has a lower one, the location of the hazardous object can be intuitively determined.

[0088] [integer overtones, etc.]

[0089] The notification control unit 120 can also function as a first notification tone NS1, causing the speaker unit 60 to output a sound including a first reference tone NS1-b and a first additional tone NS1-p (p = 1, 2, ...) that is an integer multiple of the first reference tone NS1-b. Alternatively, the notification control unit 120 can function as a second notification tone NS2, causing the speaker unit 60 to output a sound including a second reference tone NS2-b and a second additional tone NS2-q (q = 1, 2, ...) whose frequency is 1 / n times that of the second reference tone NS2-b (n is a natural number). The second additional tone NS2-q can also be a sound that does not correspond to an integer multiple of the second reference tone NS2-b. In this case, the first reference tone NS1-b is an example of "reference tone A," and the second reference tone NS2-b is an example of "reference tone B."

[0090] Figure 7 This diagram illustrates the structure of the first notification tone NS1 and the second notification tone NS2. The first reference tone NS1-b is, for example, a sound higher than the second reference tone NS2-b, but it can also be a sound of the same pitch (same frequency). In the illustrated example, the first notification tone NS1 includes a first additional tone NS1-1 and a first additional tone NS1-2. The first additional tone NS1-1 and the first additional tone NS1-2 are integer multiples of the first reference tone NS1-b, and therefore are sounds higher than the first reference tone NS1-b. Similarly, the second notification tone NS2 includes a second additional tone NS2-1 and a second additional tone NS2-2. A sound whose frequency is "different from 1 / n times the frequency of the second reference tone NS2-b" could be either a sound higher than the second reference tone NS2-b or a sound lower than the second reference tone NS2-b, but the notification control unit 120, for example, uses the second additional tone NS2-q to only cause the speaker unit 60 to output a sound lower than the second reference tone NS2-b.

[0091] Sounds composed of a reference tone and integer multiples thereof sound clear to the human ear. Conversely, sounds whose relationships are not integer multiples of a reference tone sound chaotic to the human ear. Therefore, by constructing the first notification tone NS1 and the second notification tone in this way, a notification tone containing the reference tone and its integer multiples is emitted for a risk object located in the first area A1, i.e., a risk object located in front of the vehicle M, thereby attracting the driver's attention more quickly. Furthermore, for a risk object located in the second area A2, i.e., a risk object located behind the vehicle M, a notification tone whose relationships are not integer multiples of a reference tone is emitted, thereby avoiding driver annoyance due to excessive notification.

[0092] Conversely, the notification control unit 120 can also output a sound from the speaker unit 60 as a first notification tone, including a first reference tone and a first additional tone whose frequency is 1 / n times different from the frequency of the first reference tone. Alternatively, the notification control unit 120 can also output a sound from the speaker unit 60 as a second notification tone, including a second reference tone and a second additional tone that is an integer multiple of the second reference tone. In this case, the first reference tone is an example of "reference tone B," and the second reference tone is an example of "reference tone A." If, in either the above or this case, the driver can identify which direction (front or rear) outputs a sound including an integer multiple of the second tone, and which direction outputs a sound including an additional tone whose frequency is 1 / n times different from the frequency of the reference tone, the location of the hazardous object can be intuitively determined.

[0093] The notification control unit 120 may further perform the following processing based on the above. For example, the notification control unit 120 of the first embodiment may, when a risky object exists in the first region A1, change the form of the first additional tone NS1-p based on the position of the risky object in the first region A1. More specifically, when a risky object exists in the first region A1, the notification control unit 120 may, when observing from the vehicle M, increase the variety of frequencies of the first additional tone NS1-p as the direction of the risky object is closer to the direction of travel of the moving body. Figure 8 This diagram illustrates one example of the types of frequencies that can be increased by adding the first additional tone NS1-p. For example, when the control unit 120 is notified that a risky object RO is present in the first region A1, and the risky object RO is present in the first central region A1-c of the first region A1 located in the width direction of the vehicle M, the frequency of the first additional tone NS1-p can be increased compared to when the risky object RO is present in the first left region A1-l or the first right region A1-r located on either side of the first central region A1-c. The diagram shows an example where the speaker unit 60 outputs the first additional tones NS1-1, NS1-2, and NS1-3 when the risky object RO is present in the first central region A1-c, and outputs the first additional tone NS1-1 when the risky object RO is present in the first right region A1-r. The notification control unit 120 is not limited to the distinction of the first central region A1-c, the first left region A1-l, and the first right region A1-r. It can also classify the position of the risk object RO with a finer scale to change the type of frequency of the first additional tone NS1-p.

[0094] Furthermore, the notification control unit 120 can also perform the following further processing. For example, the notification control unit 120 of the first embodiment can also, when a risky object exists in the second region A2, adjust the form of the second additional tone NS2-q based on the position of the risky object within the second region A2. More specifically, when a risky object exists in the second region A2, the notification control unit 120 can also increase the volume of the second additional tone NS2-q as the direction of the risky object, when viewed from vehicle M, is closer to the opposite direction of the vehicle M's direction of travel. Figure 9 This diagram illustrates an example of increasing the volume of the second additional tone NS2-q. For example, when the risk object RO is present in the second region A2, the notification control unit 120 may increase the volume of the second additional tone NS2-q compared to when the risk object RO is present in the second central region A2-c, which is located in the center of the width direction of the moving body within the second region A2, rather than when the risk object RO is present in the second left region A2-1 or the second right region A2-r, which are located to the left or right of the second central region A2-c. The notification control unit 120 is not limited to the distinction between the second central region A2-c, the second left region A2-1, and the second right region A2-r; it may also classify the position of the risk object RO with finer scales to change the volume of the second additional tone NS2-q.

[0095] Instead of the above, the notification control unit 120 may also change the volume of the first additional tone NS1-p based on the position of the risky object RO in the first region A1 when the risky object RO exists, and may also change the frequency type of the second additional tone NS2-q based on the position of the risky object RO in the second region A2 when the risky object RO exists.

[0096] [Controls corresponding to sound image localization and distance]

[0097] The notification control unit 120 can also control the output mode of one or both of the first notification tone NS1 and the second notification tone NS2 based on one or both of the distance between the vehicle M and the risky object and the direction (angle) of the risky object when viewed from the vehicle M. Figure 10 This is a diagram illustrating the definitions of parameters in the implementation method. In the following description, the parameters will be related to the travel direction D of the vehicle M. MThe uniform direction is set to 0 degrees, and the angle is defined counterclockwise. Furthermore, the direction of the boundary line between the first central region A1-c and the first left-side region A1-1 is defined as θ1; the direction of the boundary line between the first left-side region A1-1 and the second left-side region A2-1 is defined as θ2; the direction of the boundary line between the second left-side region A2-1 and the second central region A2-c is defined as θ3; the direction of the boundary line between the second central region A2-c and the second right-side region A2-r is defined as -θ3; the direction of the boundary line between the second right-side region A2-r and the first right-side region A1-r is defined as -θ2; and the direction of the boundary line between the first right-side region A1-r and the first central region A1-c is defined as -θ1. It should be noted that in this example, the regions are set to be symmetrical left and right, but the regions can also be set to be asymmetrical left and right. Additionally, the distance between vehicle M and the risk object RO is defined as Drisk, and the direction of the risk object RO as observed from vehicle M (based on the above definitions) is defined as θrisk.

[0098] The notification control unit 120 can also determine the location to which the sound image of the first notification tone NS1 is located based on the location of the risky object in the first area A1 when the risky object exists, and determine the location to which the sound image of the second notification tone NS2 is located based on the location of the risky object in the second area A2 when the risky object exists.

[0099] Specifically, the notification control unit 120 determines the volume of the sound output by each speaker based on the following formula, thereby enabling sound image localization. In the following explanation, SL-FR represents the volume of the sound output by speaker 60FR, SL-FL represents the volume of the sound output by speaker 60FL, SL-RR represents the volume of the sound output by speaker 60RR, and SL-RL represents the volume of the sound output by speaker 60RL. Additionally, Lbs is the reference volume.

[0100] (a) The risky object exists in the first central area A1-c

[0101] SL-FR=Lbs×(θ1-θrisk) / 2θ1

[0102] SL-FL=Lbs×(θrisk+θ1) / 2θ1

[0103] SL-RR = SL-RL = 0

[0104] (b) The risky object exists in the first left-side region A1-l.

[0105] SL-FL=Lbs×(θ2-θrisk) / (θ2-θ1)

[0106] SL-RL=Lbs×(θrisk-θ1) / (θ2-θ1)

[0107] SL-FR=SL-RR=0

[0108] (c) The risky object exists in the second left-side region A2-1.

[0109] SL-FL=Lbs×(θ3-θrisk) / (θ3-θ2)

[0110] SL-RL=Lbs×(θrisk-θ2) / (θ3-θ2)

[0111] SL-FR=SL-RR=0

[0112] (d) The risky object exists in the second central area A2-c.

[0113] SL-RR=Lbs×{θrisk+θ3+2×(180-θ3)} / {2×(180-θ3)}

[0114] SL-RL=Lbs×(-θrisk+θ3) / {2×(180-θ3)}

[0115] SL-FR = SL-FL = 0

[0116] (e) The risky object exists in the second right region A2-r

[0117] SL-FR=Lbs×(θ3+θrisk) / (θ3-θ2)

[0118] SL-RR=Lbs×(-θrisk-θ2) / (θ3-θ2)

[0119] SL-FL = SL-RL = 0

[0120] (f) The risky object exists in the first right-hand region A1-r

[0121] SL-FR=Lbs×(θ2+θrisk) / (θ2-θ1)

[0122] SL-RR=Lbs×(-θrisk-θ1) / (θ2-θ1)

[0123] SL-FL = SL-RL = 0

[0124] In this way, the location of the acoustic image, especially the position of the lateral hazardous object (RO), can be intuitively transmitted to the driver.

[0125] [Notification tone control corresponding to direction and proximity]

[0126] The notification control unit 120 can also control the beat and volume of the first notification tone NS1 or the second notification tone NS2 based on the direction θrisk and distance Drisk of the risk object RO, as follows.

[0127] For example, when a risky object exists in the first area A1, the control unit 120 notifies that the direction of the risky object RO, when viewed from vehicle M, is closer to the direction of travel D of vehicle M. M The slower the beat of the first notification tone NS1, the closer the direction of the risky object RO is to the direction of travel D of vehicle M when observed from vehicle M, given that the risky object RO exists in the second region A2. M The opposite direction will speed up the beat of the second notification tone NS2. Figure 11 This diagram illustrates an example of different tempos for the notification tone. Alternatively, as shown, when a hazard object RO is present in the first region A1, the notification control unit 120 may slow down the tempo of the first notification tone NS1 when the hazard object RO is present in the first central region A1-c, compared to when the hazard object RO is present in the first left region A1-l or the first right region A1-r. Since the driver often pays attention to the area in front of the vehicle M, the control must avoid excessive reporting. Furthermore, it is considered that reporting to the area in front with a high-pitched sound including integer multiples of a octave can effectively arouse attention. Therefore, outputting the first notification tone NS1 can reduce the stress on the driver.

[0128] Alternatively, when the risky object RO is present in the second region A2, the notification control unit 120 may speed up the tempo of the second notification tone NS2 when the risky object RO is present in the second central region A2-c, compared to when the risky object RO is present in the second left region A2-1 or the second right region A2-r. Furthermore, the notification control unit 120 may assume the distance to Drisk is the same and set the tempo of the slowest second notification tone NS2 when the risky object RO is present in the second region A2 to be faster than the tempo of the fastest first notification tone NS1 when the risky object RO is present in the first region A1.

[0129] The notification control unit 120 is not limited to the distinction of the first central region A1-c, the first left side region A1-l, the first right side region A1-r, the second central region A2-c, the second left side region A2-l, and the second right side region A2-r. It can also classify the position of the risk object RO with a finer scale and change the beat of the first notification tone NS1 or the second notification tone NS2.

[0130] The notification control unit 120 can, in any area where a risky object RO is present, increase the tempo of the first notification tone NS1 or the second notification tone NS2 as its distance from Drisk increases. Alternatively, the notification control unit 120 can also increase the volume of the first notification tone NS1 or the second notification tone NS2 as its distance from Drisk increases. Figure 12 This is a diagram illustrating an example of volume control corresponding to distance Drisk. In the diagram, Dth is the distance from which the output control of the notification tone begins, which can be the same as the distance from vehicle M to the outer edge of areas A1 and A2.

[0131] Figure 13 This is a flowchart illustrating an example of the processing flow performed by the driving support device 100. First, the identification unit 110 identifies a risky object (step S100). Next, the identification unit 110 determines the area where the risky object exists, the distance Drisk from the risky object, and the direction θrisk of the risky object (step S102).

[0132] Next, the notification control unit 120 determines the structure, rhythm, and volume of the notification tone (step S104), and causes the speaker unit 60 to output the notification tone (step S106).

[0133] Next, the identification unit 110 determines whether the location of the risk object is outside the notification target range (including non-recognition) (step S108). Outside the notification target range means that the distance from the vehicle M is greater than the aforementioned Dth. If the location of the risk object is outside the notification target range, the processing in this flowchart ends. If the location of the risk object is not outside the notification target range, the processing returns to step S102.

[0134] According to the first embodiment described above, occupants can intuitively identify the location or direction of hazardous objects present around the moving body (vehicle M).

[0135] <Second Implementation Method>

[0136] The second embodiment will now be described. The driving support device in the second embodiment does not perform the functions used in the first embodiment. Figure 8 and Figure 9The structure change control of the first notification tone NS1 in the first region A1 and the structure change control of the second notification tone NS2 in the second region A2, as described above, differs depending on whether the hazardous object RO is present in the first region A1 or the second region A2. Specifically, in the second embodiment, when the hazardous object RO is present in the first region A1, the speaker unit 60 outputs the first notification tone NS1, which includes a first reference tone NS1-b and a first additional tone NS1-p (p = 1, 2, ...), which is an integer multiple of the first reference tone NS1-b. When the hazardous object RO is present in the second region A2, the speaker unit 60 outputs the second notification tone NS2, which includes a second reference tone NS2-b and a second additional tone NS2-q (q = 1, 2, ...), whose frequency is 1 / n times that of the second reference tone NS2-b. Other controls are the same as in the first embodiment. Therefore, the same effect as the first embodiment can be achieved with simpler control.

[0137] <Third Implementation Method>

[0138] The third embodiment will now be described. The driving support device of the third embodiment is mounted on an autonomous vehicle. Figure 14 This diagram illustrates a device mounted on a vehicle M, centered on the driving support device 100A according to the third embodiment. The driving support device 100A, along with the automatic driving control device 150, is mounted on the vehicle MA. Output data from the camera 10, radar device 12, LIDAR 14, object recognition device 16, and vehicle sensors 40 are also input to the automatic driving control device 150. Additionally, output data from the navigation device and MPU (Micro Processing Unit) may also be input to the automatic driving control device 150. The automatic driving control device 150 identifies the surrounding conditions of the vehicle MA and generates a target trajectory (with speed parameters) for the vehicle MA by identifying an area where it can travel while avoiding contact with obstacles. It then controls the driving force output device 200, braking device 210, steering device 220, etc., so that the vehicle MA can travel along the target trajectory. The driving force output device 200 includes an engine, a driving motor, etc. Detailed information about automatic driving technology is available in various publications, therefore further explanation is omitted.

[0139] The automatic driving control unit 150 is also connected to a steering wheel grip sensor 160 and an in-vehicle camera 170. The steering wheel grip sensor 160 detects whether the driver is gripping the steering wheel, which serves as the steering input, while the in-vehicle camera 170 captures a frontal view of the driver's head. The automatic driving control unit 150 determines whether the driver is gripping (holding) the steering wheel by referring to the output of the steering wheel grip sensor 160, and analyzes the image captured by the in-vehicle camera 170 to determine whether the driver is visually recognizing (looking at) the direction of travel of the vehicle MA. Based on the surrounding environment of the vehicle MA (the type of road), the speed of the vehicle MA, etc., the automatic driving control unit 150 allows the driver to release (not hold) the steering wheel, or to look towards one or both directions other than the direction of travel of the vehicle MA (not looking at). This permitted state is sometimes referred to as the level of automatic driving. If an unpermitted state occurs (e.g., the driver releases their hands from the steering wheel while holding it), the automatic driving control unit 150 notifies the driver to clear the state; if the state is not cleared, it performs processing such as switching to manual driving. The automatic driving control device 150 outputs information related to the level of automatic driving to the driving support device 100A.

[0140] The driver assistance device 100A, referring to information related to the level of autonomous driving, automatically stops operating when both hands-free and gaze-free operation are permitted. This is because, in this state, the driver is temporarily not obligated to monitor the surroundings of the vehicle MA, and the autonomous driving control device 150 performs controls to avoid contact with hazardous objects. Other functions are the same as in the first or second embodiment.

[0141] According to the third embodiment described above, control with high affinity for autonomous driving can be achieved.

[0142] <Fourth Implementation Method>

[0143] The fourth embodiment will now be described. The driving support device of the fourth embodiment is mounted on a two-wheeled vehicle in which the driver is wearing a helmet. Figure 15 This diagram illustrates a device mounted on a two-wheeled vehicle MB, centered on the driver support device 100B according to the fourth embodiment. The driver support device 100B, instead of controlling the speaker unit 60, instructs the wireless communication device 80 to send instruction information related to notification tones. The wireless communication device 80 communicates based on communication standards such as Bluetooth (registered trademark).

[0144] A receiver 310, a left ear speaker 320-L, and a right ear speaker 320-R are mounted on the helmet 300 worn by the driver. The receiver 310 causes one or both of the left ear speaker 320-L and the right ear speaker 320-R to output a notification tone based on instruction information received from the wireless communication device 80.

[0145] Except for the difficulty in expressing the front-back direction when controlling the sound image localization, it is otherwise the same as the first or second embodiment. That is, the identification unit 110 and the notification control unit 120 perform the same processing as described in the first or second embodiment, determine the structure, rhythm, and volume of the first notification tone NS1 and the second notification tone NS2, and cause the wireless communication device 80 to send instruction information for causing the left ear speaker 320-L and / or the right ear speaker 320-R to output notification tones.

[0146] In the fourth embodiment, a wireless headset or similar device can be used instead of a speaker installed inside the helmet.

[0147] According to the fourth embodiment described above, although the degree of freedom of sound image localization is reduced, it can achieve the same effect as the first or second embodiment for other points.

[0148] The above description illustrates specific embodiments of the present invention, but the present invention is not limited to such embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. A driving support device, wherein, The driving support device includes: The identification unit identifies risky objects present around the moving object based on the output of the object detection device. as well as The notification control unit, based on the location of the risky object, causes at least one speaker to output a notification tone. The notification control unit, when the risky object is present in a first region including the area in front of the moving body, causes the at least one speaker to output a first notification tone; and when the risky object is present in a second region including the area behind the moving body, causes the at least one speaker to output a second notification tone. The first notification tone is a sound with a different pitch than the second notification tone. The first notification tone includes a first reference tone and a first additional tone that is an integer multiple of the first reference tone. The second notification tone includes a second reference tone and a second additional tone whose frequency is different from 1 / n times the frequency of the second reference tone and is not an integer multiple of the second reference tone, where n is a natural number.

2. The driving support device according to claim 1, wherein, The first notification tone is a higher pitch than the second notification tone.

3. The driving support device according to claim 1 or 2, wherein, When the risky object is present in the first area, the notification control unit adjusts the form of the first additional sound based on the location of the risky object within the first area.

4. The driving support device according to claim 3, wherein, When the risky object is present in the first area, the notification control unit increases the variety of frequencies of the first additional sound as the direction of the risky object is closer to the direction of travel of the moving body when observed from the moving body.

5. The driving support device according to claim 3, wherein, When the risky object is present in the first central region of the first region, which is located in the center of the width direction of the moving body, the notification control unit increases the variety of frequencies of the first additional sound compared to when the risky object is present in the first left region or the first right region located on either side of the first central region.

6. The driving support device according to claim 1 or 2, wherein, When the risky object is present in the second area, the notification control unit adjusts the form of the second additional sound based on the different positions of the risky object within the second area.

7. The driving support device according to claim 6, wherein, When the risky object is present in the second area, the notification control unit increases the volume of the second additional sound as the direction of the risky object, when viewed from the moving body, is closer to the opposite direction of the moving body's direction of travel.

8. The driving support device according to claim 6, wherein, When the risky object is present in the second central region of the second region, which is located in the center of the width direction of the moving body, the notification control unit increases the volume of the second additional sound compared to when the risky object is present in the second left region or the second right region located on either side of the second central region.

9. The driving support device according to claim 1 or 2, wherein, When the risky object is present in the first area, the notification control unit slows down the tempo of the first notification sound as the direction of the risky object is closer to the direction of travel of the moving body when viewed from the moving body.

10. The driving support device according to claim 1 or 2, wherein, When the risky object is present in the second area, the notification control unit speeds up the beat of the second notification sound as the direction of the risky object, when viewed from the moving body, is closer to the opposite direction of the moving body's direction of travel.

11. The driving support device according to claim 1 or 2, wherein, The smaller the distance between the moving body and the risky object, the faster the notification control unit speeds up the beat of the first notification tone or the second notification tone.

12. The driving support device according to claim 1 or 2, wherein, The at least one speaker includes multiple speakers. When the risky object is present in the first area, the notification control unit determines the location to which the sound image of the first notification sound is located based on the location of the risky object in the first area; when the risky object is present in the second area, the notification control unit determines the location to which the sound image of the second notification sound is located based on the location of the risky object in the second area.

13. The driving support device according to claim 1 or 2, wherein, The risky object is a participant in traffic.

14. A driving support method, wherein, The driving support method causes a computer mounted on a mobile body to perform the following processing: Based on the output of the object detection device, risky objects existing around the moving body are identified. Based on the location of the hazardous object, at least one speaker will output a notification tone. When the notification tone is output, if the risky object is present in a first region containing the area in front of the moving body, the computer causes the at least one speaker to output a first notification tone; if the risky object is present in a second region containing the area behind the moving body, the computer causes the at least one speaker to output a second notification tone. The first notification tone is a sound with a different pitch than the second notification tone. The first notification tone includes a first reference tone and a first additional tone that is an integer multiple of the first reference tone. The second notification tone includes a second reference tone and a second additional tone whose frequency is different from 1 / n times the frequency of the second reference tone and is not an integer multiple of the second reference tone, where n is a natural number.

15. A storage medium storing a program, wherein, The program causes the computer mounted on the mobile device to perform the following processing: Based on the output of the object detection device, risky objects existing around the moving body are identified. Based on the location of the hazardous object, at least one speaker will output a notification tone. When the notification tone is output, if the risky object is present in a first region containing the area in front of the moving body, the computer causes the at least one speaker to output a first notification tone; if the risky object is present in a second region containing the area behind the moving body, the computer causes the at least one speaker to output a second notification tone. The first notification tone is a sound with a different pitch than the second notification tone. The first notification tone includes a first reference tone and a first additional tone that is an integer multiple of the first reference tone. The second notification tone includes a second reference tone and a second additional tone whose frequency is different from 1 / n times the frequency of the second reference tone and is not an integer multiple of the second reference tone, where n is a natural number.

Images