Driving assistance device, driving assistance method, and program

Abstract

To provide a driving support device, a driving support method, and a program capable of performing more appropriate driving support through a sound notification according to a surrounding situation.SOLUTION: A driving support device according to an embodiment comprises: a recognition section for recognizing a surrounding situation of a movable body; a determination section for determining whether or not to prompt a driver of the movable body to perform either acceleration or deceleration on the basis of the surrounding situation; and a notification control section for causing a speaker to output a notification sound for prompting the driver of the movable body to perform either acceleration or deceleration on the basis of a result of the determination. The notification sound is obtained by continuously changing a sound structure.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a driving assistance device, a driving assistance method, and a program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable traffic participants have been gaining momentum. To achieve this, efforts are being made to further improve traffic safety and convenience through research and development of driving assistance technologies. In this regard, technology has been disclosed that enables vehicle occupants to identify information to be notified (information presentation, warning, alarm) by sound expression (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-158935 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the driving assistance technology, the provision of sounds to guide the occupant to drive appropriately in response to the various differences in the surrounding conditions of the vehicle has not been taken into consideration. Therefore, there is a problem that the occupant may not be provided with appropriate driving assistance.

[0005] In order to solve the above-mentioned problems, one of the objects of the present application is to provide a driving assistance device, a driving assistance method, and a program that can provide more appropriate driving assistance by notifying the driver of sounds according to the surrounding conditions, thereby contributing to the development of sustainable transportation systems. [Means for solving the problem]

[0006] A driving assistance device, a driving assistance method, and a program according to the present invention employ the following configuration. (1): A driving assistance device according to one aspect of the present invention includes a recognition unit that recognizes a surrounding situation of a moving body, a decision unit that decides whether to prompt a driver of the moving body to accelerate or decelerate based on the surrounding situation, and a notification control unit that outputs a notification sound to a speaker that prompts the driver of the moving body to accelerate or decelerate based on a result of the decision, the notification control unit outputs a notification sound to a speaker to prompt the driver of the moving body to either accelerate or decelerate when a difference between the speed of the moving body and a target speed becomes equal to or greater than a threshold based on the result of the determination, and when a difference between the speed of the moving body and the target speed becomes less than the threshold while the notification sound is being output and it is recognized that the moving body is traveling on a specific road section where a change in speed of the moving body is predicted based on the surrounding circumstances, the notification control unit continues to output the notification sound, the target speed being a target speed of the vehicle that is set based on the legal speed or speed limit of the lane in which the moving body is traveling, or that is set according to the speeds of surrounding vehicles; The notification sound is The above includes the frequency or pitch of the sound. The composition of sound Successively louder or softer This is a driving assistance device that has been transformed.

[0007] (2): In the above aspect (1), the notification control unit continuously changes the notification speed depending on whether the moving object is accelerating or decelerating. The aforementioned Notification sounds with different sound configurations are output from the speaker.

[0009] ( 3 ):the above( 1 In the above aspect, the notification control unit is configured to The aforementioned Deviation from target speed The smaller , said frequency or pitch So that It is something that causes change.

[0010] ( 4 ):the above( 1 In the above aspect, the notification control unit is configured to The aforementioned Deviation from target speed The smaller , the period of the notification sound to be repeatedly output So that it becomes longer It is something that changes.

[0011] ( 5 ):the above( 1 ) In the above aspect, a plurality of speakers are provided on the moving body, and the notification control unit controls the output from the plurality of speakers so that the sound image of the notification sound moves from behind to in front of the driver, or from in front to behind the driver, depending on whether the moving body is accelerating or decelerating.

[0014] ( 6 ):the above( 1 ) In the above aspect, the specific road section includes a merging section, and the notification control unit terminates output of the notification sound when the moving body completes merging from the merging lane to the main road in the merging section.

[0015] ( 7 ):the above( 1 In the above aspect, the notification control unit is configured to The aforementioned The difference from the target speed The aforementioned When the threshold value is less than the threshold value and the mobile object is not traveling on the specific road section, the output of the notification sound is terminated.

[0016] ( 8 ):the above( 1 In the above aspect, the notification control unit, after outputting the notification sound, The aforementioned The difference from the target speed The aforementioned If it is below the threshold, To notify that the current speed is acceptable, The notification sound different from the notification sound is output from the speaker.

[0017] ( 9 ):the above( 1 In the above aspect, the notification control unit is configured to, while the notification sound is being output, determine the speed of the moving object and The aforementioned The difference from the target speed The aforementioned When the threshold is exceeded, the and information that predetermines whether or not to terminate the notification sound when the difference between the speed of the moving object and the target speed becomes less than the threshold value. Whether or not to end the notification sound is determined based on the above.

[0018] ( 10 ):the above( 1 In the above aspect, the notification control unit is configured to, while the notification sound is being output, determine the speed of the moving object and The aforementioned The difference from the target speed The aforementioned When the threshold is exceeded, , the output of the notification sound Upon receiving an end instruction, the output of the notification sound is ended.

[0019] (11 ):the above( 1 ) In the aspect of the present invention, the notification control unit further includes a learning unit that learns a notification sound to be notified to the driver based on a degree of change in the traveling of the moving body after the notification sound is output, and the notification control unit is configured to The aforementioned The difference from the target speed The aforementioned When the threshold value is less than the threshold value, a notification sound associated with the driver that has been learned in advance by the learning unit is output from the speaker.

[0020] ( 12 ) A driving assistance method according to another aspect of the present invention includes a computer recognizing a surrounding situation of a moving object, determining whether to prompt a driver of the moving object to accelerate or decelerate based on the recognized surrounding situation, and outputting a notification sound from a speaker to prompt the driver of the moving object to accelerate or decelerate based on the result of the determination; Based on the result of the determination, when a difference between the speed of the moving body and a target speed becomes equal to or greater than a threshold, a notification sound is output from a speaker to prompt the driver of the moving body to either accelerate or decelerate, and when a difference between the speed of the moving body and the target speed becomes less than the threshold while the notification sound is being output, and when it is recognized that the moving body is traveling on a specific road section where a change in speed of the moving body is predicted based on the surrounding circumstances, the notification sound is continued to be output, and the target speed is a target speed of the vehicle that is set based on the legal speed or speed limit of the lane in which the moving body is traveling, or that is set according to the speed of surrounding vehicles, The notification sound is The above includes the frequency or pitch of the sound. The composition of sound Successively louder or softer This is a driving assistance method that has been changed.

[0021] ( 13 ) A program according to another aspect of the present invention causes a computer to recognize a surrounding situation of a moving object, determine whether to prompt a driver of the moving object to accelerate or decelerate based on the recognized surrounding situation, and output a notification sound to a speaker to prompt the driver of the moving object to accelerate or decelerate based on the result of the determination; Based on the result of the determination, when a difference between the speed of the moving body and a target speed becomes equal to or greater than a threshold, a notification sound is output from a speaker to prompt the driver of the moving body to either accelerate or decelerate, and when a difference between the speed of the moving body and the target speed becomes less than the threshold while the notification sound is being output, and when it is recognized that the moving body is traveling on a specific road section where a change in speed of the moving body is predicted based on the surrounding circumstances, the notification sound is continued to be output, and the target speed is a target speed of the vehicle that is set based on the legal speed or speed limit of the lane in which the moving body is traveling, or that is set according to the speed of surrounding vehicles, The notification sound is The above includes the frequency or pitch of the sound. The composition of sound Successively louder or softer It is the program that has changed. [Effects of the Invention]

[0022] According to the above aspects (1) to (16), more appropriate driving assistance can be provided by providing a sound notification according to the surrounding situation. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a diagram showing devices mounted on a vehicle M, focusing on a driving assistance device 100 according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing an example of the contents of scene-specific notification sound information 182. [Figure 3] 10A and 10B are diagrams for explaining the configuration of an acceleration guiding notification sound. [Figure 4] 10A and 10B are diagrams for explaining the configuration of a deceleration guidance notification sound. [Figure 5] FIG. 10 is a diagram illustrating the configuration of an OK notification sound (neutral notification sound). [Figure 6] FIG. 10 is a diagram for explaining notification content in the first scene. [Figure 7] FIG. 10 is a diagram for explaining notification content in the second scene. [Figure 8] 3 is a flowchart showing an example of a flow of processing executed by the driving assistance device 100 of the embodiment. [Figure 9] 10 is a diagram showing devices mounted on a vehicle M, focusing on a driving assistance device 100A according to a second embodiment. FIG. [Figure 10] FIG. 10 is a diagram showing devices mounted on a two-wheeled vehicle MB, focusing on a driving assistance device 100B according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0024] Hereinafter, with reference to the drawings, embodiments of a driving assistance device, a driving assistance method, and a program of the present invention will be described. The driving assistance device is a device that assists driving of a mobile object. The mobile object may include any mobile object on which a person (driver) rides, including three-wheeled or four-wheeled vehicles, two-wheeled vehicles, micromobility, etc. In the first and second embodiments described below, the mobile object is a four-wheeled vehicle, and in the third embodiment, the mobile object is a two-wheeled vehicle. Hereinafter, a vehicle equipped with a driving assistance device will be referred to as "vehicle M." Furthermore, the following description will be given for a case where laws stipulating left-hand traffic apply, but if laws stipulating right-hand traffic apply, the left and right can be reversed.

[0025] First Embodiment [composition] FIG. 1 is a diagram showing devices mounted on a vehicle M, focusing on a driving assistance device 100 according to the first embodiment. The vehicle M may be any of an automobile powered by an internal combustion engine such as a diesel engine or a gasoline engine, an electric automobile powered by an electric motor, or a hybrid automobile equipped with both an internal combustion engine and an electric motor. In this embodiment, illustrations and descriptions of components for driving the vehicle M, such as driving controls, drive devices such as an engine or a motor, a steering device, and a brake device, are omitted, but these components may be mounted on the vehicle M.

[0026] The vehicle M is equipped with, for example, a vehicle sensor 10, a navigation device 20, an object detection device 30, a speaker 60, a display device 70, a driving assistance device 100, and the like.

[0027] The vehicle sensor 10 includes some or all of a vehicle speed sensor that detects the speed of the vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, a direction sensor that detects the orientation of the vehicle M, a gradient sensor that detects the gradient of the location where the vehicle M is located, etc. The vehicle sensor 10 may also be provided with a position sensor that detects the position of the vehicle M. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. The position sensor may also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 21 of the navigation device 20.

[0028] The navigation device 20 includes, for example, a GNSS receiver 21, a navigation HMI (Human Machine Interface) 22, and a route determination unit 23. The navigation device 20 stores map information in a storage device such as a hard disk drive (HDD) or flash memory. The GNSS receiver 21 identifies the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be identified or supplemented by an inertial navigation system (INS) that uses the output of the vehicle sensors 10. The navigation HMI 22 includes a display device, a speaker, a touch panel, keys, etc. The route determination unit 23 determines, for example, a route (hereinafter referred to as a map route) from the position of the vehicle M identified by the GNSS receiver 21 (or any input position) to a destination input by the occupant using the navigation HMI 22, with reference to map information. The map information is, for example, information that represents road shapes using links indicating roads and nodes connected by the links. The map information may include road curvature and gradient (uphill, downhill), the number of lanes, legal speed limit, type information (e.g., information indicating urban areas, expressways, national highways, etc., merging sections, branching sections, toll gates, and whether the roadway and sidewalk are clearly separated), POI (Point of Interest) information, etc. The navigation device 20 may provide route guidance using the navigation HMI 22 based on the route on the map. The navigation device 20 may be realized, for example, by a terminal device such as a smartphone or tablet device carried by the occupant that is communicatively connected to the driving assistance device 100. The navigation device 20 may transmit the current position and destination to a navigation server via a communication device and obtain a route equivalent to the route on the map from the navigation server.

[0029] The object detection device 30 includes, for example, a camera, a radar device, a LIDAR (Light Detection and Ranging), a sensor fusion device, etc. The object detection device 30 uses known functions to detect the type and position of an object (other vehicles, motorcycles, bicycles, pedestrians, road dividing lines, road signs, markings, etc.) present around the vehicle M. The object detection device 30 may also be capable of detecting the speed of an object.

[0030] The speaker 60 outputs sound into the cabin of the vehicle M. The speaker 60 may be a speaker unit including multiple sub-speakers, in which case the position of the sound image may be set arbitrarily. For example, if the speaker 60 is a speaker unit, the sub-speakers are provided at least in front of the driver's seat where the driver sits (for example, on an instrument panel inside the cabin) and at least one at the rear (for example, on the headrest of the driver's seat, on the rear seat, or at the rear of the cabin). Furthermore, multiple sub-speakers may be provided as surround speakers surrounding the driver's seat so as to provide surround sound inside the cabin.

[0031] The display device 70 is, for example, a touch panel, and is attached to any location within the cabin of the vehicle M. The display device 70 accepts various operations for the driving assistance device 100 and displays images instructed by the driving assistance device 100, etc. The speaker 60 and the display device 70 may be provided integrally with the navigation HMI 22 or may be provided separately.

[0032] The driving assistance device 100 includes, for example, a recognition unit 110, a target speed setting unit 120, an acceleration / deceleration guidance determination unit 130, a notification control unit 140, a learning unit 150, and a storage unit 180. The components other than the storage unit 180 are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as an HDD (Hard Disk Drive) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device. The acceleration / deceleration guidance determining unit 130 is an example of a "determining unit."

[0033] The storage unit 180 may be realized by the various storage devices described above, or a solid state drive (SSD), an electrically erasable programmable read only memory (EEPROM), a read only memory (ROM), or a random access memory (RAM). The storage unit 180 stores, for example, scene-specific notification sound information 182, programs, and various other information. The contents of the scene-specific notification sound information 182 will be described later. The storage unit 180 may also store the map information described above.

[0034] The recognition unit 110 recognizes the surrounding situation (driving scene) of the vehicle M based on the results of detection by the object detection device 30. For example, the recognition unit 110 recognizes the position, speed, acceleration, and other states of objects present around the vehicle M. The position of the object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. For example, if the object is a moving body such as another vehicle, the "state" of the object may include the acceleration or jerk of the moving body, or the "behavioral state" (for example, whether the other vehicle is changing lanes or is about to change lanes).

[0035] For example, the recognition unit 110 performs known analysis processing (e.g., edge extraction, feature extraction, pattern matching, character recognition, etc.) on an image captured by a camera of the object detection device 30 (hereinafter, a camera image), and recognizes landmarks (lane boundaries, road boundaries) that can identify lane positions, including road dividing lines, road shapes, shoulders, curbs, medians, guardrails, fences, walls, etc., from the image analysis results. The recognition unit 110 may also recognize the gradient of the road from the road shapes and road signs obtained from the analysis results of the camera image. The recognition unit 110 may also recognize that the road shape on which the vehicle M is traveling (which may include the road shape on which the vehicle M is predicted to travel in the near future) is a specific road section where the speed of the vehicle M is predicted to change. Examples of specific road sections include a merging section, a branching section, an uphill section, a downhill section, and a toll booth section. The following description uses a merging section as an example of a specific road section. For example, the recognition unit 110 may recognize that the shape of the road on which the vehicle M is traveling is a merging section or not a merging section. The recognition unit 110 may also recognize stop lines, obstacles, red lights, and other road phenomena.

[0036] Furthermore, the recognition unit 110 may refer to map information based on the position of the vehicle M detected by the vehicle sensor 10 and the GNSS receiver 21, and recognize lanes around the vehicle M, including the lane in which the vehicle M is traveling. Furthermore, the recognition unit 110 may recognize whether the lane in which the vehicle M is traveling or the shape of the road on which the vehicle M is predicted to travel in the future is a merging section or not a merging section.

[0037] The target speed setting unit 120 sets a target speed V# at which the vehicle M should travel. The target speed setting unit 120 acquires, for example, information on the legal speed of the lane on which the vehicle M is traveling from the navigation device 20 or the recognition unit 110, and sets the acquired legal speed as the target speed V#. The target speed setting unit 120 may set the target speed V# to a value smaller than the legal speed depending on the curvature and gradient of the road. The target speed setting unit 120 may also set the target speed V# to a value smaller than the legal speed depending on the degree of congestion of objects detected by the object detection device 30 (for example, the degree of congestion based on the number of vehicles ahead). On a road where the roadway and the sidewalk are not clearly separated, if the object detection device 30 detects pedestrians equal to or greater than a threshold, the target speed setting unit 120 may set the target speed V# to a value smaller than the legal speed in order to autonomously slow down. Information that vehicle M is traveling on a road where the carriageway and the sidewalk are not clearly separated may be obtained from navigation device 20, or may be obtained by driving assistance device 100 analyzing images captured by the camera of object detection device 30.

[0038] The acceleration / deceleration guidance determination unit 130 determines whether to prompt the driver of the vehicle M to accelerate or decelerate based on the state of the vehicle M. Here, the state includes, for example, the speed V and the target speed V#. For example, the acceleration / deceleration guidance determination unit 130 determines whether to prompt the driver of the vehicle M to accelerate or decelerate based on a comparison between the speed V of the vehicle M and the target speed V#. For example, the acceleration / deceleration guidance determination unit 130 determines to prompt the driver of the vehicle M to decelerate when the speed V is greater than the target speed V# and the difference therebetween is equal to or greater than a threshold. Furthermore, the acceleration / deceleration guidance determination unit 130 determines to prompt the driver of the vehicle M to accelerate when the speed V is smaller than the target speed V# and the difference therebetween is equal to or greater than a threshold. Furthermore, the acceleration / deceleration guidance determination unit 130 determines not to prompt the driver of the vehicle M to accelerate or decelerate when the difference between the speed V and the target speed V# is less than the threshold. However, the present invention is not limited to this, and the acceleration / deceleration guidance determination unit 130 may determine whether to encourage acceleration or deceleration based on the curvature and gradient of the road, the degree of congestion around the vehicle M, and the like.

[0039] Furthermore, the acceleration / deceleration guidance determination unit 130 may determine the degree of acceleration / deceleration to be prompted to the driver of the vehicle M. For example, the acceleration / deceleration guidance determination unit 130 determines the degree of acceleration / deceleration to be increased as the magnitude of the difference (degree of deviation) between the speed V and the target speed V# increases. For example, the acceleration / deceleration guidance determination unit 130 determines the degree of acceleration / deceleration to be prompted to the driver of the vehicle M in two stages, high and low, by setting the degree of acceleration / deceleration to be prompted to the driver of the vehicle M to be high when the degree of deviation is equal to or greater than a reference value (a value greater than the threshold value described above), and setting the degree of acceleration / deceleration to be prompted to the driver of the vehicle M to be low when the degree of deviation is less than the reference value. By setting a plurality of the above-mentioned reference values, the degree of acceleration can be determined in multiple stages.

[0040] When the acceleration / deceleration guidance determination unit 130 determines to issue a notification prompting the driver to either accelerate or decelerate, the notification control unit 140 causes the speaker 60 to output a corresponding notification sound (acceleration guidance notification sound, deceleration guidance notification sound). Furthermore, when the difference between the speed V and the target speed V# is less than a threshold, the notification control unit 140 may cause the speaker 60 to output a notification sound (OK notification sound) that notifies the driver of the vehicle M that the current speed is sufficient (neither acceleration nor deceleration is necessary), instead of issuing a notification prompting the driver to either accelerate or decelerate. The OK notification sound is different from the acceleration guidance notification sound and the deceleration guidance notification sound. The OK notification sound may be output, for example, when the difference between the speed V and the target speed V# becomes less than a threshold after the acceleration guidance notification sound or the deceleration guidance notification sound has been output. When outputting these notification sounds, the notification control unit 140 refers to the scene-specific notification sound information 182 based on the surrounding conditions of the vehicle recognized by the recognition unit 110, and determines which notification sound to use and how to notify based on the surrounding conditions (scene type) and the notification content (notification type), and outputs the determined notification sound to the speaker 60.

[0041] FIG. 2 is a diagram illustrating an example of the content of the scene-specific notification sound information 182. The scene-specific notification sound information 182 is information in which notification sound information is associated with a scene type and a notification type. The scene type includes information about the surrounding situation (driving scene) of the vehicle M. The scene type includes, for example, merging (driving in a merging section) and driving on a straight road section (an example of a section other than a specific road section). The notification type includes information about the content of the notification (e.g., acceleration guidance, deceleration guidance, maintaining the current status quo because the speed difference is less than a threshold (OK notification)). The notification sound information includes information about a notification sound corresponding to the scene type and the notification type (e.g., acceleration guidance notification sound, deceleration guidance notification sound, OK notification sound). The notification sound in the embodiment is, for example, a sound whose composition is continuously changed. "Continuously changing the composition of the sound" means, for example, continuously changing the frequency or pitch of the sound. More specifically, the pitch is the pitch (melodic pitch) of sounds that are output sequentially. The notification sound information may also include a notification sound whose frequency or pitch is changed depending on the degree of acceleration or deceleration to be prompted to the driver of the vehicle M, or the degree of deviation between the speed V of the vehicle M and the target speed V#. The degree of change refers to, for example, the magnitude of change in at least one of the repetition period, frequency magnitude, frequency change amount, and volume of the notification sound. The notification sound information may also include information about a sound that conveys the output of a notification (notification end sound). The notification sound information may also include audio information (human voice or mechanical voice) in addition to the notification sound.

[0042] The information included in the scene-specific notification sound information 182 may be preset as fixed information or may be arbitrarily set by the driver. When set by the driver, for example, information (such as notification sound information) received by the display device (touch panel) 70 is registered (or updated) in the scene-specific notification sound information 182.

[0043] Furthermore, the notification control unit 140 may cause the speaker to output the notification sound set in the scene-specific notification sound information 182 when the difference between the speed V of the vehicle M and the target speed V# is equal to or greater than a threshold, and may determine whether to continue outputting the notification sound according to the surrounding conditions of the vehicle M (notification type) when the difference between the speed V of the vehicle M and the target speed V# is less than the threshold. For example, when the vehicle M is traveling in a specific road section, the notification control unit 140 may cause the speaker to continue outputting the notification sound even if the difference between the speed V of the vehicle M and the target speed V# is less than the threshold, and when the vehicle M is traveling in a road other than the specific road section, the notification control unit 140 may end output of the notification sound when the difference becomes less than the threshold. When a merging section is included in the specific road section, the notification control unit 140 may end output of the notification sound when the vehicle M has completed merging from the merging lane in the merging section to the main lane (for example, when the vehicle M has passed the merging completion point).

[0044] Furthermore, when the difference between the speed of the vehicle M and the target speed V# becomes less than a threshold value while the acceleration guidance notification sound or the deceleration guidance notification sound is being output, the notification control unit 140 may determine whether to terminate the notification sound based on the content set by the driver. Furthermore, when the difference between the speed of the vehicle M and the target speed V# becomes less than a threshold value while the acceleration guidance notification sound or the deceleration guidance notification sound is being output, the notification control unit 140 may accept an end instruction from the driver and terminate the output of the notification sound. This allows the notification control instructed by the driver to be executed.

[0045] Furthermore, instead of (or in addition to) obtaining a notification sound according to the vehicle state or surrounding circumstances by referring to the scene-specific notification sound information 182, the notification control unit 140 may cause the speaker 60 to output the notification sound via a device capable of generating and synthesizing a desired sound, such as a synthesizer. Furthermore, in addition to (or instead of) the notification sound, the notification control unit 140 may generate an image urging the driver to accelerate or decelerate or an image for maintaining the current speed, and display the image on the display device 70.

[0046] The learning unit 150 learns the notification sound to be used to notify the driver based on the degree of change in the traveling of the vehicle M due to manual driving after the notification sound is output. For example, the learning unit 150 learns the notification sound preferred by the driver (in other words, the notification sound that is likely to induce the driver to perform a specific driving behavior (accelerate, decelerate, maintain the current state)) based on notification sounds output in the past and the result of guidance of the vehicle M after the notification, and updates the scene-specific notification sound information 182 based on the result of learning. For example, the learning unit 150 causes the notification control unit 140 to output a predetermined notification sound that prompts the driver to accelerate or decelerate the vehicle M, and learns the notification sound preferred by the driver based on the acceleration or deceleration of the vehicle M obtained from the vehicle sensor 10 after the notification sound is output, the time until the difference between the speed V of the vehicle M and the target speed v# becomes less than a threshold, and the like. For example, the learning unit 150 learns the notification sound preferred by the driver based on the difference between the speed V of the vehicle M and the target speed V# after the notification sound is output. The difference The learning unit 150 may learn the notification sound that takes the shortest time for the difference between the speed V of the vehicle M and the target speed v# to become less than the threshold value as the notification sound that the driver prefers. Furthermore, the learning unit 150 may learn the notification sound that keeps the difference less than the threshold value for the longest time as the notification sound that the driver prefers.

[0047] The learning unit 150 performs the above learning by surrounding situation (by scene) and registers (or updates) the notification sound learned for each in the scene-specific notification sound information 182, thereby enabling the driver to be notified of the notification sound of his / her preference according to each scene, making it easier for the driver to drive in a comfortable mood.

[0048] The learning unit 150 may generate a trained model that receives the surrounding conditions of the vehicle M recognized by the recognition unit 110 as input and outputs a notification sound. This trained model is updated using, for example, AI (Artificial Intelligence) functions such as machine learning (neural network) and deep learning using teacher (correct answer) data. The trained model may be stored in the storage unit 180 or may be acquired from an external device via a network. The notification control unit 140 uses the trained model to output the notification sound. If the notification sound is not updated through the above-described learning, the driving assistance device 100 does not need to have the function of the learning unit 150.

[0049] The vehicle M is equipped with driving controls such as an accelerator pedal, a brake pedal, and a steering wheel, as well as a power output device and a steering device that perform acceleration / deceleration and steering in accordance with the amount of operation of the driving controls, but these are not shown in the drawings or described in detail below. In the following description, the accelerator opening, which is the amount of operation of the accelerator pedal, may be referred to as Qap.

[0050] [About driving assistance via notification sounds] Next, driving assistance using a notification sound according to the embodiment will be described. Note that, below, as an example, a case will be described in which the driving assistance device 100 assists the driver in smoothly merging onto a main roadway. For example, when the legal speed limits of the merging lane and the main road are different, or when there are other vehicles near the merging point, it is necessary to adjust the speed of the vehicle M according to the legal speed limit of the lane in which the vehicle is traveling and the speed of the other vehicles. However, if the driver is inexperienced in driving or is not good at merging, the driver may not be able to adjust the speed appropriately and may not be able to merge smoothly. Therefore, the driving assistance device 100 according to the embodiment outputs an acceleration guidance notification sound, a deceleration guidance notification sound, an OK notification sound, or the like from the speaker 60 according to the surrounding conditions when merging, thereby enabling the driver to intuitively understand what speed the vehicle M should be at, and thus urging the driver to appropriately adjust the speed (accelerate, decelerate, maintain the current speed). Note that, in this embodiment, similar notification control can be performed not only in merging sections, but also in other specific road sections such as forks and slopes, and roads that are not specific road sections (for example, straight expressways).

[0051] [Notification sounds] For example, when urging the driver to accelerate the vehicle M, the notification control unit 140 outputs an acceleration-inducing notification sound that encourages the driver to accelerate, and when urging the driver to decelerate the vehicle M, the notification control unit 140 outputs a deceleration-inducing notification sound that encourages the driver to decelerate. Furthermore, when the notification control unit 140 wants to maintain the current speed of the vehicle M, it outputs a stable neutral notification sound (OK notification sound) that does not make the driver aware of acceleration or deceleration. An example of a notification sound corresponding to a speed adjustment that encourages the driver will be described below. Note that the notification sound output in the embodiment is not limited to the following example.

[0052] FIG. 3 is a diagram illustrating the configuration of the acceleration-guiding notification sound. In the diagram, the horizontal axis represents time, and the vertical axis represents the frequency of the sound. The same applies to FIGS. 4 and 5 described below. SA1 and SA2 in FIG. 3 are each a notification sound included in the acceleration-guiding notification sound. The acceleration-guiding notification sound may include three or more notification sounds. Unless each of the multiple notification sounds SA1, SA2, ... is specifically classified and described, they will be collectively referred to as "notification sound SA." The notification sound SA shown in FIG. 3 indicates a change in frequency over time, and the arrows on the notification sound SA are shown to make the frequency change trend easier to understand. The notification sound SA includes multiple sounds. For example, the notification sound SA has a continuously changing sound configuration, more specifically, the sound frequency is changed significantly continuously (or in stages). The frequency increase trend of the notification sound SA may change nonlinearly (for example, a downwardly convex curve) as shown in FIG. 3, or it may change linearly. Furthermore, the minimum frequency Fmin, maximum frequency Fmax, frequency change amount ΔF, output time of one notification sound SA (period from notification start time Ts to notification end time Te) ΔT, repetition period ΔC, frequency increase trend (linear, non-linear), etc. can be changed depending on the surrounding situation (scene type) and acceleration level (or deviation degree), and this information is stored, for example, in scene-specific notification sound information 182. Furthermore, in the example of Fig. 3, the same notification sound SA is repeatedly output as the acceleration-guided notification sound, but instead, the minimum frequency Fmin, maximum frequency Fmax, frequency change amount ΔF, etc. may be increased with each repetition, the output time ΔT or repetition period ΔC may be shortened, or the volume may be increased up to an upper limit value.

[0053] As shown in Figure 3, when guiding the driver to accelerate, by repeatedly outputting a notification sound with a continuously increasing frequency, the continuous sound that gradually gets higher in pitch makes it easier for the driver to associate acceleration with the sound, and therefore it is possible to output an acceleration guiding notification sound that makes the driver want to accelerate.

[0054] FIG. 4 is a diagram illustrating the configuration of the deceleration guidance notification sound. Each of SD1 and SD2 in FIG. 4 is one notification sound included in the deceleration guidance notification sound. The deceleration guidance notification sound may include three or more notification sounds. Except when each of the multiple notification sounds SD1, SD2, ... is specifically classified and described, they will be collectively referred to as "notification sound SD." The notification sound SD shown in FIG. 4 includes multiple sounds, similar to the notification sound SA described above. For example, the notification sound SD is a sound whose configuration changes continuously, and more specifically, the frequency of the sound is changed continuously (or in small steps). The decreasing trend of the frequency of the notification sound SD may change nonlinearly (for example, an upwardly convex curve) as shown in FIG. 4, or may change linearly. Furthermore, the minimum frequency Fmin, maximum frequency Fmax, frequency change amount ΔF, output time ΔT of the notification sound SD, repetition period ΔC, frequency decrease tendency (linear, non-linear), etc. can be changed depending on the surrounding circumstances (scene type) and deceleration rate (or deviation rate), and this information is stored, for example, in scene-specific notification sound information 182. Furthermore, in the example of Fig. 4, the same notification sound SD is repeatedly output as the deceleration guidance notification sound, but instead, the frequency change amount ΔF may be increased each time it is repeated, the output time ΔT or repetition period ΔC may be shortened, or the volume may be increased up to an upper limit.

[0055] As shown in Fig. 4, during deceleration guidance, a notification sound having a different sound structure that changes continuously from that during acceleration guidance is output from the speaker 60. For example, during deceleration guidance, by repeatedly outputting a notification sound with a continuously decreasing frequency, the continuous sound that gradually gets lower makes it easier for the driver to associate deceleration with the need to decelerate, and therefore it is possible to output a deceleration guidance notification sound that encourages the driver to decelerate.

[0056] FIG. 5 is a diagram illustrating the configuration of an OK notification sound (neutral notification sound). SO1 and SO2 in FIG. 5 are each a notification sound included in the OK notification sound. The OK notification sound may include three or more notification sounds. Unless specifically classifying and describing the multiple notification sounds SO1, SO2, ..., they will be collectively referred to as "notification sound SO." Unlike the notification sounds SA and SD described above, the notification sound SO shown in FIG. 5 has a constant continuous sound configuration (within a predetermined range). The frequency Fok of the notification sound SO is linear, as shown in FIG. 4. The value of the frequency Fok, the output time ΔT, the repetition period ΔC, and the like of the notification sound SO can be changed depending on the surrounding circumstances (scene type), etc., and this information is stored, for example, in the scene-specific notification sound information 182. In the example of FIG. 4, the notification sound SO repeatedly output as the OK notification sound is the same, but the frequency Fok, the output time ΔT, the repetition period ΔC, the volume, and the like may be changed within a predetermined range.

[0057] As shown in Fig. 5, when acceleration guidance or deceleration guidance is not performed, different notification sounds are output from the speaker 60 for acceleration guidance and deceleration guidance. For example, when an OK notification is given, a stable sound with little change in frequency is output, which makes it easier for the driver to associate that acceleration or deceleration is unnecessary (to maintain the current speed). In this way, by using different notification sounds depending on the state of the vehicle M, more appropriate driving assistance can be provided regarding speed adjustment of the vehicle M.

[0058] 3 to 5, the "frequency" on the vertical axis may be interpreted as "pitch." In this case, for example, the increasing or decreasing trend of frequency may be interpreted as the trend of change in pitch, and the amount of frequency change may be interpreted as the pitch difference. In other words, when guiding to accelerate or decelerate, the notification control unit 140 causes the speaker 60 to output a notification sound in which the frequency or pitch of the sound is continuously changed in volume or volume. In this way, by using different notification sounds depending on the state and driving conditions of the vehicle M, more appropriate driving assistance can be provided regarding the speed adjustment of the vehicle M.

[0059] Furthermore, when the speaker 60 is configured with multiple sub-speakers provided in the vehicle cabin, the notification control unit 140 may control the output of sound from the multiple sub-speakers so that the sound image of the notification sound flows (moves) from behind the driver (driver's seat) to the front or from in front of the driver to the rear when the vehicle M is accelerating and decelerating. The movement of the sound image can be achieved, for example, by adjusting the volume according to the installation positions of the multiple sub-speakers. For example, when outputting an acceleration guidance notification sound, the notification control unit 140 outputs the sound image so that it flows from behind the driver to the front, and when outputting a deceleration guidance notification sound, the notification control unit 140 outputs the sound image so that it flows from in front of the driver to the rear. Note that the flow direction (movement direction) may be reversed. When outputting an OK notification sound, the notification control unit 140 outputs the sound image from either the front or the rear, or both, without flowing (moving) the sound image. In this way, by varying the output mode of the sound image depending on the content of the speed adjustment, it is possible to more reliably prompt the driver to adjust the speed.

[0060] Next, a specific output mode of the notification sound output from the speaker 60 by the notification control unit 140 will be described in detail. Note that the following description will be divided into two driving scenes.

[0061] <Scene 1: Reuniting> FIG. 6 is a diagram for explaining the notification content in a first scene. The first scene shows a scene in which a vehicle M merges from a merging lane onto a main lane in a merging section. The example of FIG. 6 shows a road in a merging section having a merging lane L11 and main lanes L12 and L13. In the example of FIG. 6, time T11 is assumed to be the earliest, followed by times T12, T13, and T14 in the order of latest. In the example of FIG. 6, the position and speed of the host vehicle M at time T* are represented as M(T*) and V(T*). The same applies to FIG. 7 described later. At time T11, the vehicle M(T11) is assumed to be traveling in the merging lane L11 at a speed V(T11) toward the main lane L2.

[0062] The recognition unit 110 recognizes the vehicle's surrounding conditions and recognizes that the vehicle M is traveling in a merging section. The recognition unit 110 also references map information based on the vehicle M's position information to acquire the legal speed limit for the main lane and recognizes the acquired legal speed as the target speed V# of the vehicle M. The recognition unit 110 may also recognize the numerical value of a road sign MK1 indicating the speed limit for the main lanes L12 and L13, detected by the object detection device 30, as the target speed of the vehicle M. The target speed V# may be set, for example, so that the difference between the target speed V# and the legal speed or the speed limit is less than a predetermined speed. The recognition unit 110 may also recognize the target speed V# based on the speed Vm1 of another nearby vehicle traveling on the main lane (in the figure, another vehicle m1 traveling on lane L13). The recognition unit 110 may also recognize a speed appropriate for merging (for merging onto the main lane without coming into contact with other vehicles) as the target speed V# based on the conditions of the surrounding vehicles (e.g., traffic congestion).

[0063] When the difference between the speed V of the vehicle M obtained from the vehicle sensor 10 and the target speed V# is equal to or greater than a threshold, the notification control unit 140 causes the speaker 60 to output a notification sound that prompts the driver to accelerate or decelerate the vehicle M. At time T11, the speed V of the vehicle M is assumed to be smaller than the target speed V# by equal to or greater than the threshold. In this case, the notification control unit 140 refers to the scene-specific notification sound information 182, acquires notification sound information (acceleration guidance notification sound) for urging the driver to accelerate for the scene type "merging", and causes the speaker 60 to output the acquired notification sound.

[0064] 6 shows that at time T11, one or more notification sounds SA whose frequencies continuously increase over time are repeatedly output as acceleration guiding notification sounds from speaker 60. Note that the minimum frequency Fmin and maximum frequency Fmax of the notification sounds SA, the amount of frequency change ΔF, the output time ΔT of the notification sounds SA, the repetition period ΔC, the frequency increase trend (linear or nonlinear), and the like may be changed depending on various conditions such as the difference (degree of deviation) between the speed V and the target speed V# of vehicle M, the degree of acceleration, the degree of congestion, the remaining distance to the point P1 where merging should be completed (hereinafter referred to as the "merging completion point"), and the speed V of vehicle M.

[0065] By outputting such a notification sound SA, the driver can be prompted to accelerate by sound. Furthermore, since the notification is not a direct request by voice such as "Please accelerate," the driver can be prevented from being overly conscious of accelerating when there is still a distance (a distance greater than a predetermined distance) from the merging completion point P1, as at time T11, and more appropriate driving assistance can be provided.

[0066] Note that, if the speed change of the vehicle M is less than a predetermined amount even after a predetermined time has elapsed since the acceleration guidance notification sound was output, or if the distance to the merging completion point P1 is less than a predetermined distance, the notification control unit 140 may increase the frequency increase rate, shorten the repetition period, or increase the frequency used to output a notification sound SA composed of a higher pitch. The notification control unit 140 may also increase the volume of the notification sound. Furthermore, the notification control unit 140 may output a voice message such as "Please accelerate" to the speaker 60, or may output an image encouraging acceleration to the display device 70. This allows the driver to be more clearly notified that acceleration is required.

[0067] At time T12, the speed V of the vehicle M is lower than the target speed V# by a threshold or more, but the speed difference (deviation degree) is smaller than that at time T11. In this case, the notification control unit 140 changes the acceleration guiding notification sound compared to that at time T11. For example, as shown in FIG. 6, the notification control unit 140 outputs a notification sound from the speaker 60, in which the amount of change in frequency is small and the maximum frequency Fmax of the notification sound SA is smaller than that of the notification sound SA at time T11. The notification control unit 140 may output a notification sound with a longer repetition period (interval) compared to that at time T11. This allows the driver to understand that the speed V of the vehicle M is approaching the target speed V# using only the notification sound SA. Furthermore, compared to when a voice such as "The vehicle speed is approaching the target speed" is output, the driver does not become overly aware of the speed of the vehicle M, and more appropriate driving assistance can be provided.

[0068] At time T13, the difference between the speed V of vehicle M and the target speed V# is assumed to be less than the threshold. In this case, the notification control unit 140 refers to the scene-specific notification sound information 182 and causes the speaker 60 to output a notification sound SO (OK notification sound) indicating that the speed is less than the threshold. This notifies the driver that the current speed is appropriate for merging.

[0069] During merging, the notification control unit 140 continues to output the OK notification sound until merging is complete (for example, until the vehicle M passes the merging completion point P1, as at time T14). This allows the driver to understand that the current speed V is an appropriate speed for merging. Even if the target speed V# suddenly changes due to a change in the behavior of other vehicles during merging, the change in the notification sound that is continuously output allows the driver to easily understand the situation and drive more appropriately.

[0070] <Scene 2: When driving on a road that is not a merging section> FIG. 7 is a diagram for explaining the notification content in the second scene. The second scene shows a scene in which a notification sound is output to prompt vehicle M to accelerate or decelerate in a road section that is not a merging section. Here, a "road section that is not a merging section" refers to a road section other than a specific road section (for example, a straight road section). In the example of FIG. 7, lanes L21 and L22 are shown that vehicle M can travel in the same direction (X-axis direction in the figure), and vehicle M is traveling in lane L21 at a speed V. In the example of FIG. 7, the position and speed of vehicle M at time T* are represented as M(T*) and V(T*). In the example of FIG. 7, time T21 is the earliest, followed by times T22, T23, and T24 in that order.

[0071] At time T21, vehicle M (T21) is traveling on lane L21 at speed V (T21). The recognition unit 110 recognizes the vehicle's surroundings and recognizes that vehicle M is not traveling in a merging section (i.e., traveling on a straight road section). Furthermore, the recognition unit 110 references map information based on the vehicle M's position information to acquire the legal speed limit for the main lane, and recognizes the acquired legal speed as vehicle M's target speed V#. Alternatively, the recognition unit 110 may recognize the numerical value of road sign MK2, which indicates the speed limit for main lanes L21 and L22 and is detected by object detection device 30, as vehicle M's target speed. Note that target speed V# is set, for example, so that the difference between target speed V# and the legal speed or the speed limit is less than a predetermined speed. Furthermore, the recognition unit 110 may recognize target speed V# based on the speed Vm2 of another vehicle traveling on lane L21 or L22 (i.e., another vehicle m2 traveling on lane L22 in the figure).

[0072] When the difference between the speed V of the vehicle M obtained from the vehicle sensor 10 and the target speed V# is equal to or greater than a threshold, the notification control unit 140 causes the speaker 60 to output a notification sound that prompts the driver to accelerate or decelerate the vehicle M. At time T21, the speed V of the vehicle M is assumed to be greater than the target speed V# by equal to or greater than the threshold. In this case, the notification control unit 140 refers to the scene-specific notification sound information 182, acquires a deceleration guidance notification sound for urging the driver to decelerate when traveling on a straight road section, and outputs the acquired deceleration guidance notification sound.

[0073] 7 shows that at time T21, one or more notification sounds SD whose frequencies continuously decrease over time are repeatedly output as deceleration guidance notification sounds from speaker 60. Note that the minimum frequency Fmin and maximum frequency Fmax of the notification sounds SD, the amount of frequency change ΔF, the output time ΔT of the notification sounds SA, the repetition period ΔC, the frequency decrease trend (linear, non-linear), etc. may be changed depending on various conditions such as the difference (degree of deviation) between the speed V of vehicle M and the target speed V#, the degree of deceleration, the degree of congestion, the speed V of vehicle M, etc.

[0074] By outputting this notification sound SD, the driver can be prompted to slow down by sound. Also, since it is not a direct request by voice such as "Please slow down," the driver can be provided with more appropriate driving assistance without being overly conscious of slowing down.

[0075] Note that, if a predetermined time has elapsed since the deceleration guidance notification sound was output, or if the change in the speed of the vehicle M is less than a predetermined amount even after traveling a predetermined distance, the notification control unit 140 may increase the frequency reduction rate, shorten the repetition cycle of the notification sound, or increase the frequency used to output a notification sound composed of a higher-pitched sound. The notification control unit 140 may also increase the volume of the notification sound SD. Furthermore, the notification control unit 140 may output a voice message such as "Please slow down" to the speaker 60, or may output an image encouraging the driver to slow down to the display device 70. This makes it possible to more clearly notify the driver that deceleration is required.

[0076] At time T22, the speed V of the vehicle M is greater than the target speed V# by a threshold value or more, but the speed difference (deviation degree) is smaller than that at time T21. In this case, the notification control unit 140 changes the deceleration guidance notification sound compared to that at time T21. For example, as shown in FIG. 7, the notification control unit 140 outputs a notification sound from the speaker 60, in which the amount of change in frequency is small and the maximum frequency of one notification sound is lower than the maximum frequency of the notification sound at time T21. The notification control unit 140 may output one notification sound with a longer repetition period (interval) compared to that at time T21. This allows the driver to understand that the speed V of the vehicle M is approaching the target speed V# using only the notification sound SD. Furthermore, compared to when a voice such as "The vehicle speed is approaching the target speed" is output, the driver does not become overly aware of the speed, and more appropriate driving assistance can be provided.

[0077] At time T23, it is assumed that the difference between the speed V of vehicle M and the target speed V# is less than the threshold value. In this case, the notification control unit 140 references the scene-specific notification sound information 182 and outputs the acquired OK notification sound to the speaker 60. The OK notification sound SO may be the same as or different from the OK notification sound for merging. In the case of a straight road section (not a merging section), the notification control unit 140 outputs the OK notification sound for a predetermined period of time and then ends the output. In other words, even if vehicle M has not passed through a straight road section, as at time T24, the output of the OK notification sound ends. This is because vehicle M is traveling at a speed that is appropriate for the legal speed limit and traffic flow of the traveling section, and because large speed changes are unlikely to occur on straight roads, etc. This makes it possible to prevent excessive notifications to the driver.

[0078] In the above example, the acceleration-inducing notification sound is composed of one or more notification sounds with successively increasing frequencies, and the deceleration-inducing notification sound is composed of one or more notification sounds with successively decreasing frequencies, but this is not limiting. The notification sound may be a single tone or a chord. The notification control unit 140 may switch between a single tone and a chord depending on the degree of deviation of the speed difference or the number of repetitions of the notification sound.

[0079] Furthermore, when the output of the notification sound is terminated, the notification control unit 140 may output a sound (notification end sound) that indicates the output of the notification. This makes it easier for the driver to understand that the notification regarding the speed adjustment has ended. Note that the notification end sound may be the same after the acceleration guidance notification, the deceleration guidance notification, and the OK notification. This makes it easier for the driver to recognize that it is the notification end sound.

[0080] [Processing flow] Fig. 8 is a flowchart showing an example of the flow of processing executed by the driving assistance device 100 of the embodiment. In the example of Fig. 8, the processing executed by the driving assistance device 100 will be mainly described, focusing on processing related to issuing a notification sound according to the state and surrounding circumstances of the vehicle M. The processing shown below may be repeatedly executed at a predetermined timing or at a predetermined cycle, for example, while the driver is driving the vehicle M.

[0081] In the example of FIG. 8 , the recognition unit 110 recognizes the surrounding conditions of the vehicle M (step S100). Next, the acceleration / deceleration guidance determination unit 130 acquires the speed V and the target speed V# of the vehicle M (step S102) and determines whether the difference between the speed V and the target speed V# of the vehicle M is equal to or greater than a threshold (step S104). If it is determined that the difference is equal to or greater than the threshold, the notification control unit 140 determines a notification sound to prompt the driver to accelerate or decelerate according to the content of the notification (notification type) and the surrounding conditions of the vehicle M recognized by the recognition unit 110 (step S106). Next, the notification control unit 140 causes the speaker 60 to output a notification sound (acceleration guidance notification sound or deceleration guidance notification sound) to prompt the driver to accelerate or decelerate in order to execute speed control to bring the speed V of the vehicle M closer to the target speed V# (step S108).

[0082] Furthermore, if it is determined in the processing of step S104 that the difference between the vehicle speed V and the target speed V# is not equal to or greater than the threshold (is less than the threshold), the notification control unit 140 causes the speaker 60 to output a notification sound (OK notification sound) indicating that the current speed V is appropriate (step S110). In the processing of step S110, the notification control unit 140 may control whether or not to continue outputting the OK notification sound based on the surrounding situation recognized by the recognition unit 110. This ends the processing of this flowchart.

[0083] [Variations] The notification control unit 140 may switch whether or not to issue the notification sound described above in response to an instruction from the driver. Furthermore, the acceleration / deceleration guidance determination unit 130 may store past driving history in the storage unit 180, and when it is determined that output of the notification sound is unnecessary based on the number of driving times included in the driving history, may determine not to perform acceleration / deceleration guidance even if the difference between the vehicle speed V and the target speed V# is equal to or greater than a threshold value. This makes it possible to suppress the notification sound from being issued when driving through a familiar merging section.

[0084] Furthermore, the melody or instrument of the notification sound in the embodiment may be changed depending on the state or surrounding circumstances of the vehicle M, and a braking sound may be used as the deceleration induction notification sound, and an engine sound may be used as the acceleration induction notification sound. Furthermore, the notification control unit 140 may adjust the frequency, volume, rhythm, intonation, etc. of the notification sound depending on the age of the driver so that the notification sound is easier to hear.

[0085] According to the first embodiment described above, the driving assistance device 100 includes a recognition unit 110 that recognizes the surrounding conditions of a vehicle (an example of a moving body) M, an acceleration / deceleration guidance determination unit (an example of a determination unit) 130 that determines whether to prompt the driver of the vehicle M to accelerate or decelerate based on the surrounding conditions, and a notification control unit 140 that causes a speaker to output a notification sound that prompts the driver of the moving body to accelerate or decelerate based on the result of the determination. The notification sound has a continuously changing sound configuration, so that more appropriate driving assistance can be provided by notification of the sound according to the surrounding conditions. Therefore, it can contribute to the development of a sustainable transportation system.

[0086] For example, according to the first embodiment, when acceleration or deceleration of the vehicle M is necessary depending on the surrounding conditions, etc., a sound that encourages the driver to accelerate or decelerate can be output, thereby making it possible to give the driver more appropriate instructions to accelerate or decelerate. Therefore, according to the first embodiment, more appropriate driving assistance can be achieved by sound-based acceleration / deceleration guidance.

[0087] Second Embodiment The second embodiment will be described below. The driving assistance device of the second embodiment is mounted on an autonomous vehicle. Autonomous driving, for example, refers to automatically controlling one or both of the steering and speed of the vehicle to perform driving control. An autonomous vehicle may be driven by manual operation by a vehicle user (e.g., a driver) (so-called manual driving). FIG. 9 is a diagram showing devices mounted on a vehicle M, focusing on a driving assistance device 100A according to the second embodiment. The driving assistance device 100A is mounted on the vehicle MA together with an autonomous driving control device 160. Output data from the vehicle sensor 10, the navigation device 20, and the object detection device 30 is also input to the autonomous driving control device 160. In addition to these, output data from the navigation device and an MPU (Micro Processing Unit) may also be input to the autonomous driving control device 160. The automatic driving control device 160 recognizes the surrounding conditions of the vehicle MA, recognizes the area in which the vehicle MA can travel while avoiding contact with obstacles, generates a target trajectory (which includes a speed element) for the vehicle MA, and controls the driving force output device 200, braking device 210, steering device 220, etc. so that the vehicle MA can travel along the target trajectory. The driving force output device 200 includes an engine, a driving motor, etc. Details of automatic driving technology are publicly known in various documents, so further explanation will be omitted.

[0088] The autonomous driving control device 160 is further connected to a steering grip sensor 170 and an interior camera 175. The steering grip sensor 170 detects whether the driver is gripping the steering wheel, which is a steering control element, and the interior camera 175 captures an image of the driver's head from the front. The autonomous driving control device 160 determines whether the driver is gripping the steering wheel (hands-on) by referring to the output of the steering grip sensor 170, and determines whether the driver is looking at the direction in which the vehicle MA is traveling (eyes-on) by analyzing the image captured by the interior camera 175. The autonomous driving control device 160 allows either or both of the following, based on the surrounding environment of the vehicle MA (the type of road on which the vehicle is traveling) and the speed of the vehicle MA. This allowable state is sometimes referred to as the autonomous driving level. When an impermissible state occurs (for example, when the driver takes his / her hands off the steering wheel in a hands-on state), the automatic driving control device 160 notifies the driver to resolve the state, and if the state is not resolved, performs processing such as switching to manual driving. The automatic driving control device 160 outputs information related to the automatic driving level (automatic driving level information) to the driving assistance device 100A.

[0089] The driving assistance device 100A refers to the autonomous driving level information and automatically stops operation if both hands-off and eyes-off are permitted. In this state, the driver is temporarily no longer required to monitor the periphery of the vehicle MA, and the autonomous driving control device 160 performs control to avoid contact with risky objects. Other functions are the same as those of the first embodiment. In other words, when the driving assistance device 100A determines based on the autonomous driving level information that manual driving will be performed in the autonomously driven vehicle (vehicle MA), it performs notification control of the notification sound based on the state and surrounding circumstances of the vehicle MA, similar to the driving assistance device 100 of the first embodiment.

[0090] According to the second embodiment described above, the driving assistance device 100A of the embodiment can be applied to an autonomous driving vehicle. Therefore, according to the second embodiment, in addition to achieving the same effects as the first embodiment, it is possible to perform control that is highly compatible with autonomous driving.

[0091] <Third embodiment> The third embodiment will be described below. The driving assistance device of the third embodiment is mounted on a motorcycle that is ridden by a rider wearing a helmet. FIG. 10 is a diagram showing devices mounted on a motorcycle MB, focusing on a driving assistance device 100B according to the third embodiment. The driving assistance device 100B differs from the driving assistance device 100 of the first embodiment in that it includes a wireless communication device 80 instead of controlling the speaker 60. The driving assistance device 100B instructs the wireless communication device 80 to transmit instruction information related to a notification sound. The wireless communication device 80 communicates based on a communication standard such as Bluetooth (registered trademark).

[0092] A receiver 310, a left ear speaker 320-L, and a right ear speaker 320-R are attached to a helmet 300 worn by a driver. The helmet 300 may be provided with at least one sub-speaker each at the front and rear of the head. A plurality of sub-speakers may be provided as surround speakers surrounding the periphery of the head so that surround sound can be provided inside the helmet 300.

[0093] The driving assistance device 100B performs the same processing as described in the first embodiment to determine a notification sound according to the state of the motorcycle MB and the surrounding circumstances, and transmits instruction information to the wireless communication device 80 to cause the left ear speaker 320-L and / or the right ear speaker 320-R to output the notification sound accordingly. The receiver 310 outputs the notification sound from the left ear speaker 320-L, the right ear speaker 320-R, and / or one or both of the sub-speakers based on the instruction information received from the wireless communication device 80. In the third embodiment, earphones with wireless functionality may be used instead of the speakers provided in the helmet.

[0094] According to the third embodiment described above, although the degree of freedom in sound image localization is somewhat reduced, in other respects it is possible to achieve the same effects as the first embodiment.

[0095] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: Recognize the surrounding situation of the moving object, determining whether to prompt the driver of the moving object to accelerate or decelerate based on the recognized surrounding conditions; Based on the result of the determination, a notification sound is output from a speaker to prompt the driver of the moving body to either accelerate or decelerate; The notification sound has a continuously changing sound configuration. Driving assistance device.

[0096] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0097] 10 Vehicle Sensors 20 Navigation devices 30 Object detection device 60 speakers 80 Wireless communication equipment 100, 100A, 100B Driving support device 110 Recognition part 120 Target speed setting section 130 Acceleration / Deceleration Guidance Decision Unit 140 Notification control section 150 Learning Department 160 Automatic driving control device 170 Steering grip sensor 175 In-car camera 300 helmet

Claims

1. a recognition unit that recognizes the surrounding situation of the moving object; a decision unit that decides whether to prompt the driver of the moving object to accelerate or decelerate based on the surrounding situation; a notification control unit that outputs a notification sound from a speaker to prompt the driver of the moving body to either accelerate or decelerate based on the result of the determination, The notification control unit outputs a notification sound to a speaker to prompt the driver of the moving body to either accelerate or decelerate when a difference between the speed of the moving body and a target speed becomes equal to or greater than a threshold based on the result of the determination, and when a difference between the speed of the moving body and the target speed becomes less than the threshold while the notification sound is being output and it is recognized that the moving body is traveling on a specific road section where a change in the speed of the moving body is predicted based on the surrounding circumstances, the notification control unit continues to output the notification sound. the target speed is a target speed of the vehicle that is set based on the legal speed or speed limit of the lane in which the moving object is traveling, or that is set in accordance with the speed of surrounding vehicles; The notification sound is a sound whose structure, including the frequency or pitch of the sound, is continuously changed from louder to softer. Driving assistance device.

2. the notification control unit causes the speaker to output the notification sound, the notification sound having a continuously changing sound configuration that differs depending on whether the moving object is accelerating or decelerating. The driving assistance device according to claim 1 .

3. the notification control unit changes the frequency or the pitch so that the smaller the deviation between the speed of the moving object and the target speed, the smaller the frequency or the pitch becomes; The driving assistance device according to claim 1 .

4. the notification control unit changes the period of the notification sound to be repeatedly output so that the period becomes longer as the degree of deviation between the speed of the moving object and the target speed becomes smaller; The driving assistance device according to claim 1 .

5. a plurality of the speakers are provided on the moving body; The notification control unit controls the output from the plurality of speakers so that a sound image of the notification sound moves from behind to in front of the driver or from in front to behind the driver depending on whether the moving body is accelerating or decelerating. The driving assistance device according to claim 1 .

6. the specific road section includes a merging section, the notification control unit terminates output of the notification sound when the moving object has completed merging from the merging lane to the main lane in the merging section. The driving assistance device according to claim 1 .

7. the notification control unit terminates output of the notification sound when a difference between the speed of the moving object and the target speed becomes less than the threshold value and when the moving object is not traveling on the specific road section. The driving assistance device according to claim 1 .

8. When a difference between the speed of the moving object and the target speed becomes less than the threshold value after the notification control unit has output the notification sound, the notification control unit outputs, from the speaker, a notification sound different from the notification sound for notifying that the current speed is acceptable. The driving assistance device according to claim 1 .

9. When a difference between the speed of the moving body and the target speed becomes less than the threshold value while the notification sound is being output, the notification control unit determines whether to terminate the notification sound based on information set in advance by the driver as to whether to terminate the notification sound when the difference between the speed of the moving body and the target speed becomes less than the threshold value. The driving assistance device according to claim 1 .

10. When a difference between a speed of the moving object and the target speed becomes less than the threshold value while the notification sound is being output, the notification control unit receives an instruction from the driver to terminate output of the notification sound and terminates output of the notification sound. The driving assistance device according to claim 1 .

11. a learning unit that learns a notification sound to be notified to the driver based on a degree of change in the traveling of the moving body after the notification sound is output; When a difference between a speed of the moving body and the target speed becomes less than the threshold value while the notification sound is being output, the notification control unit outputs, from the speaker, a notification sound associated with the driver that has been learned in advance by the learning unit. The driving assistance device according to claim 1 .

12. The computer Recognize the surrounding situation of the moving object, determining whether to prompt the driver of the moving object to accelerate or decelerate based on the recognized surrounding conditions; Based on the result of the determination, a notification sound is output from a speaker to prompt the driver of the moving body to either accelerate or decelerate; If, based on the result of the determination, the difference between the speed of the moving body and the target speed is equal to or greater than a threshold value, a notification sound is output from a speaker to prompt the driver of the moving body to either accelerate or decelerate; if, while the notification sound is being output, the difference between the speed of the moving body and the target speed is less than the threshold value, and if, based on the surrounding conditions, it is recognized that the moving body is traveling on a specific road section where the speed of the moving body is expected to change, the notification sound is continued to be output; the target speed is a target speed of the vehicle that is set based on the legal speed or speed limit of the lane in which the moving object is traveling, or that is set in accordance with the speed of surrounding vehicles; The notification sound is a sound whose structure, including the frequency or pitch of the sound, is continuously changed from louder to softer. Driving assistance methods.

13. On the computer, It allows the vehicle to recognize its surroundings, The system determines whether to prompt the driver of the moving object to accelerate or decelerate based on the recognized surrounding conditions; outputting a notification sound from a speaker to prompt the driver of the moving body to either accelerate or decelerate based on the determined result; If, based on the result of the determination, the difference between the speed of the moving body and the target speed is equal to or greater than a threshold value, a notification sound is output from a speaker to prompt the driver of the moving body to either accelerate or decelerate; if, while the notification sound is being output, the difference between the speed of the moving body and the target speed is less than the threshold value, and if, based on the surrounding conditions, it is recognized that the moving body is traveling on a specific road section where the speed of the moving body is expected to change, the notification sound is continued to be output; the target speed is a target speed of the vehicle that is set based on the legal speed or speed limit of the lane in which the moving object is traveling, or that is set in accordance with the speed of surrounding vehicles; The notification sound is a sound whose structure, including the frequency or pitch of the sound, is continuously changed from louder to softer. program.

Images