Driving assistance device, driving assistance method, and program

Abstract

To provide appropriate drive assistance via a sound notification dependent on a surrounding situation.SOLUTION: A drive assistance device according to an embodiment comprises: a recognition unit that recognizes a surrounding situation of a mobile object; a determination unit that determines on the basis of the surrounding situation whether or not an operator of the mobile object is prompted to perform a steering operation and determines whether there is a possibility that a risk occurs with the mobile body; and a notification control unit that causes a speaker to output, on the basis of a determination result of the determination unit, a guiding sound that prompts the operator of the mobile object for a steering operation and an alarm sound for the risk. The notification control unit outputs the guiding sound and the alarm sound in different manners.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a driving assistance device, a driving assistance method, and a program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable traffic participants have been gaining momentum. To achieve this, efforts are being made to further improve traffic safety and convenience through research and development of driving assistance technologies. In this regard, technology has been disclosed that enables vehicle occupants to identify information to be notified (information presentation, warning, alarm) by sound expression (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-158935 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the driving assistance technology, there has been no consideration given to providing a sound that guides the driver to drive appropriately according to the situation when steering a moving object. Therefore, there is a problem that the driver may not be provided with appropriate driving assistance.

[0005] In order to solve the above-mentioned problems, one of the objects of the present application is to provide a driving assistance device, a driving assistance method, and a program that can provide more appropriate driving assistance by providing sound notifications according to the situation, thereby contributing to the development of sustainable transportation systems. [Means for solving the problem]

[0006] A driving assistance device, a driving assistance method, and a program according to the present invention employ the following configuration. (1): A driving assistance device according to one embodiment of the present invention includes a recognition unit that recognizes the surrounding conditions of a moving body, a judgment unit that judges, based on the surrounding conditions, whether to prompt the driver of the moving body to steer and whether there is a possibility of a risk to the moving body, and a notification control unit that, based on the judgment result by the judgment unit, causes a speaker to output a guidance sound that prompts the driver of the moving body to steer and a warning sound regarding the risk, and the notification control unit differentiates the output manner of the guidance sound and the warning sound.

[0007] (2) In the above aspect (1), the notification control unit causes the speaker to output the warning sound having a higher frequency than the guiding sound.

[0008] (3) In the above aspect (2), the guiding sound has fewer harmonic components than the warning sound.

[0009] (4): In the above aspect (1), the notification control unit positions the sound image so that the warning sound is output from the direction of the risk based on the driver, and controls the output of the guidance sound so that the sound image moves in the direction in which the driver is desired to steer.

[0010] (5): In the above aspect (1), the notification control unit localizes a sound image so that the warning sound is output from the direction of the risk relative to the driver.

[0011] (6): In the above aspect (1), a target steering angle determination unit is further provided that determines a target steering angle for the traveling direction of the moving body based on the surrounding conditions, and the notification control unit controls the output mode of the guidance sound based on the degree of deviation between the target steering angle and the current steering angle of the moving body.

[0012] (7) In the above aspect (1), the notification control unit changes at least one of the moving position and the moving speed of the sound image of the guiding sound based on the degree of deviation.

[0013] (8) In the aspect (6) above, the notification control unit determines a start point and an end point when moving the sound image of the guiding sound based on the degree of deviation.

[0014] (9): In the above aspect (8), the start point is determined based on the current steering angle of the moving body, and the end point is determined based on the target steering angle.

[0015] (10): In the aspect (6) above, the notification control unit outputs a notification sound from the speaker indicating that the current steering angle should be maintained when the degree of deviation between the target steering angle and the current steering angle of the moving body is less than a threshold value.

[0016] (11): In the aspect (10) above, the notification control unit localizes the sound image of the notification sound at a position corresponding to the target steering angle or the current steering angle of the moving body.

[0017] (12): In the above aspect (1), when the notification control unit moves the sound image of the guiding sound, it moves the sound image horizontally or in the rotation direction of the driving operator that performs the steering operation.

[0018] (13): In the above aspect (1), when the notification control unit causes the speaker to output the guiding sound when the moving body is reversing, the notification control unit generates the guiding sound based on the constituent components of the reversing sound output from the speaker when the moving body is reversing.

[0019] (14): A driving assistance device according to another aspect of the present invention includes a recognition unit that recognizes the surrounding conditions of a moving body, a target steering angle determination unit that determines a target steering angle for the direction of travel of the moving body based on the surrounding conditions, and a notification control unit that causes a speaker to output a guidance sound that prompts the driver of the moving body to perform a steering operation based on the target steering angle determined by the target steering angle determination unit and the current steering angle of the moving body, wherein the notification control unit varies the output mode of the guidance sound based on the degree of deviation between the target steering angle and the current steering angle of the moving body.

[0020] (15): Another aspect of the present invention is a driving assistance method in which a computer recognizes the surrounding conditions of a moving body, and determines, based on the surrounding conditions, whether to prompt the driver of the moving body to steer and whether there is a possibility of a risk to the moving body. Based on the determination result, the computer causes a speaker to output a guidance sound to prompt the driver of the moving body to steer and a warning sound regarding the risk, and the output manner of the guidance sound and the warning sound is different.

[0021] (16): Another aspect of the present invention is a driving assistance method in which a computer recognizes the surrounding conditions of a moving body, determines a target steering angle for the moving body's direction of travel based on the surrounding conditions, outputs a guidance sound to a speaker to encourage the driver of the moving body to steer based on the determined target steering angle and the moving body's current steering angle, and varies the output mode of the guidance sound based on the degree of deviation between the target steering angle and the moving body's current steering angle.

[0022] (17): Another aspect of the present invention is a program that causes a computer to recognize the surrounding conditions of a moving body, and based on the surrounding conditions, determines whether or not to prompt the driver of the moving body to steer and whether or not there is a possibility of a risk to the moving body, and based on the determination results, causes a speaker to output a guidance sound that prompts the driver of the moving body to steer and a warning sound regarding the risk, and differentiates the output manner of the guidance sound and the warning sound.

[0023] (18): Another aspect of the present invention is a program that causes a computer to recognize the surrounding conditions of a moving body, determine a target steering angle for the direction of travel of the moving body based on the surrounding conditions, output a guidance sound to a speaker to encourage the driver of the moving body to steer based on the determined target steering angle and the current steering angle of the moving body, and vary the output manner of the guidance sound based on the degree of deviation between the target steering angle and the current steering angle of the moving body. [Effects of the Invention]

[0024] According to the above aspects (1) to (18), more appropriate driving assistance can be provided by providing a sound notification according to the surrounding situation. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a diagram showing devices mounted on a vehicle M, focusing on a driving assistance device 100 according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing an example of the contents of a scene-specific output mode table 182. [Figure 3] FIG. 1 is a diagram showing a scene in which a vehicle M is traveling on a curved road. [Figure 4] FIG. 1 is a diagram for explaining the localization of a sound image. [Figure 5] 10A and 10B are diagrams for explaining how the sound image position MP of the guiding sound moves when the degree of deviation is large. [Figure 6] 10A and 10B are diagrams for explaining how the sound image position MP of the guiding sound moves when the degree of deviation is small. [Figure 7] FIG. 10 is a diagram for explaining the sound image position in the case of an OK notification sound. [Figure 8] 10A and 10B are diagrams for explaining the positions of sound image localization of a guiding sound and a warning sound. [Figure 9] 10 is a diagram for explaining how a sound image position MP is moved based on the rotation direction of a steering wheel. FIG. [Figure 10] 3 is a flowchart showing an example of a flow of processing executed by the driving assistance device 100 of the embodiment. [Figure 11] 10 is a diagram showing devices mounted on a vehicle M, focusing on a driving assistance device 100A according to a second embodiment. FIG. [Figure 12] FIG. 10 is a diagram showing devices mounted on a two-wheeled vehicle MB, focusing on a driving assistance device 100B according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0026] Hereinafter, with reference to the drawings, embodiments of a driving assistance device, a driving assistance method, and a program of the present invention will be described. The driving assistance device is a device that assists driving of a mobile object. The mobile object may include any mobile object on which a person (a passenger such as a driver) rides, including three-wheeled or four-wheeled vehicles, two-wheeled vehicles, micromobility, etc. In the first and second embodiments described below, the mobile object is a four-wheeled vehicle, and in the third embodiment, the mobile object is a two-wheeled vehicle. Hereinafter, a vehicle equipped with a driving assistance device will be referred to as "vehicle M." Furthermore, the following description will be given for a case where laws stipulating left-hand traffic apply, but if laws stipulating right-hand traffic apply, left and right can be reversed.

[0027] First Embodiment [composition] FIG. 1 is a diagram showing devices mounted on a vehicle M, focusing on a driving assistance device 100 according to the first embodiment. The vehicle M may be any of an automobile powered by an internal combustion engine such as a diesel engine or a gasoline engine, an electric automobile powered by an electric motor, or a hybrid automobile equipped with both an internal combustion engine and an electric motor. In this embodiment, illustrations and descriptions of components for driving the vehicle M, such as driving controls, drive devices such as an engine or a motor, a steering device, and a brake device, are omitted, but these components may be mounted on the vehicle M.

[0028] The vehicle M is equipped with, for example, a vehicle sensor 10, a navigation device 20, an object detection device 30, driving controls 40, a speaker 60, a display device 70, a driving assistance device 100, and the like.

[0029] The vehicle sensor 10 includes, for example, a vehicle speed sensor that detects the speed of the vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, a direction sensor that detects the orientation of the vehicle M, and a steering angle sensor that detects the steering angle of the vehicle M (which may be the angle of the steering wheels or the operating angle of the steering wheel). The vehicle sensor 10 may also be provided with a position sensor that detects the position of the vehicle M. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. The position sensor may also be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 21 of the navigation device 20.

[0030] The navigation device 20 includes, for example, a GNSS receiver 21, a navigation HMI (Human Machine Interface) 22, and a route determination unit 23. The navigation device 20 stores map information 24 in a storage device such as a hard disk drive (HDD) or flash memory. The GNSS receiver 21 identifies the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be identified or supplemented by an inertial navigation system (INS) that uses the output of the vehicle sensors 10. The navigation HMI 22 includes a display device, a speaker, a touch panel, keys, etc. The route determination unit 23 determines, for example, a route (hereinafter referred to as a map route) from the position of the vehicle M identified by the GNSS receiver 21 (or any input position) to a destination input by a driver or other occupant using the navigation HMI 22, with reference to the map information. The map information 24 is, for example, information representing road shapes using links indicating roads and nodes connected by the links. The map information 24 may include road curvature and gradient (uphill, downhill), the number of lanes, legal speed limits, road type information (e.g., urban area, expressway, national highway, etc., merging section, branching section, toll gate, curved road, crank, intersection, T-junction), POI (Point of Interest) information, etc. The navigation device 20 may provide route guidance using the navigation HMI 22 based on the route on the map. The route guidance may include, for example, a notification that there is a curved road in the direction of travel of the vehicle M, or information encouraging the driver to make a right or left turn or change lanes to reach the destination. The navigation device 20 may be realized by, for example, a terminal device such as a smartphone or tablet device owned by the driver that is communicatively connected to the driving assistance device 100. The navigation device 20 may obtain the latest map information 24 from an external device via a communication device (not shown) mounted on the vehicle M, or may transmit the current position and destination to a navigation server and obtain a route equivalent to the route on the map from the navigation server.

[0031] The object detection device 30 includes, for example, a camera, a radar device, a LIDAR (Light Detection and Ranging), a sensor fusion device, etc. The object detection device 30 uses known functions to detect the type and position of objects (other vehicles, motorcycles, bicycles, pedestrians, road structures (road shoulders, curbs, medians, road dividing lines, road signs), etc.) present around the vehicle M. The object detection device 30 may also be capable of detecting the speed of an object.

[0032] The driving operators 40 include, for example, a steering wheel, an accelerator pedal, a brake pedal, a shift lever, and other operators. The steering wheel is a steering operator that accepts steering operations of the vehicle M by the driver. A sensor is attached to the driving operators 40 to detect the amount of steering or the presence or absence of operation, and the detection results are output to the driving assistance device 100 or some or all of the driving force output device, braking device, and steering device (not shown). The operators do not necessarily have to be annular, and may be in the form of an irregular steering wheel, a joystick, a button, or the like.

[0033] The speaker 60 outputs sound into the cabin of the vehicle M. The speaker 60 may be a speaker unit including multiple sub-speakers, in which case the position of the sound image may be set arbitrarily. For example, if the speaker 60 is a speaker unit, the sub-speakers are provided at least in front of the driver's seat where the driver sits (e.g., on the instrument panel inside the cabin), on each side (e.g., on the left and right doors), and behind (e.g., on the headrest of the driver's seat, on the rear seat, or at the rear of the cabin). Furthermore, multiple sub-speakers may be provided as surround speakers surrounding the driver's seat so as to provide surround sound inside the cabin.

[0034] The display device 70 is, for example, a touch panel, and is attached to any location within the cabin of the vehicle M. The display device 70 accepts various operations for the driving assistance device 100 and displays images instructed by the driving assistance device 100, etc. The speaker 60 and the display device 70 may be provided integrally with the navigation HMI 22 or may be provided separately.

[0035] The driving assistance device 100 includes, for example, a recognition unit 110, a target steering angle determination unit 120, a determination unit 130, a notification control unit 140, and a storage unit 180. The components other than the storage unit 180 are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as an HDD (Hard Disk Drive) or a flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or a CD-ROM, and installed by inserting the storage medium into a drive device.

[0036] The storage unit 180 may be realized by the various storage devices described above, or a solid state drive (SSD), an electrically erasable programmable read only memory (EEPROM), a read only memory (ROM), a random access memory (RAM), or the like. The storage unit 180 stores, for example, a scene-specific output mode table 182, programs, and various other information. The scene-specific output mode table 182 stores, for example, for each driving scene of the vehicle M, information on guidance sounds for guiding the driver of the vehicle M to steer, and information on warning sounds (risk alerts) for warning that there is a possibility of a risk occurring to the vehicle M. The contents of the scene-specific output mode table 182 will be described later. The storage unit 180 may also store map information 24.

[0037] The recognition unit 110 recognizes the surrounding situation (driving scene) of the vehicle M based on the results of detection by the object detection device 30. For example, the recognition unit 110 recognizes the position, speed, acceleration, and other states of objects present around the vehicle M. The position of the object is recognized as a position on an absolute coordinate system with a reference point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of the object may be expressed by a representative point such as the center of gravity or a corner of the object, or may be expressed by a represented area. For example, if the object is a moving body such as another vehicle, the "state" of the object may include the acceleration or jerk of the moving body, or the "behavioral state" (for example, whether the other vehicle is changing lanes or is about to change lanes).

[0038] For example, the recognition unit 110 performs known analysis processing (for example, edge extraction, feature extraction, pattern matching processing, character recognition processing, etc.) on an image captured by a camera of the object detection device 30 (hereinafter, camera image), and recognizes landmarks (lane boundaries, road boundaries) that can identify lane positions, including road dividing lines, road shapes, shoulders, curbs, medians, guardrails, fences, walls, etc. from the image analysis results. The recognition unit 110 may also recognize the curvature, gradient, etc. of the road from the road shapes and road signs obtained from the analysis results of the camera image.

[0039] Furthermore, the recognition unit 110 may recognize that the shape of the road on which the vehicle M is traveling (which may include the shape of the road on which the vehicle M is predicted to travel in the near future) is a specific road section, based on the results of detection by the object detection device 30. A specific road section is a road section on which it is predicted that the steering angle of the vehicle M will need to be changed while traveling, such as a curved road, a crank, an intersection, a T-junction, a branching section, or a merging section. The following description will mainly use a curved road as an example of a specific road section. For example, the recognition unit 110 may recognize that the road on which the vehicle M is traveling or the road on which the vehicle M is predicted to travel in the near future (a road in the traveling direction of the vehicle M and within a predetermined distance from the vehicle M) is a curved road or is not a curved road. The recognition unit 110 may also recognize stop lines, obstacles, traffic lights, and other road phenomena.

[0040] In addition, instead of (or in addition to) recognizing based on the results detected by the object detection device 30, the recognition unit 110 may refer to map information 24 based on the position of vehicle M detected by the vehicle sensor 10 or the GNSS receiver 21, and recognize the road shape around vehicle M, including the lane in which vehicle M is traveling, recognize the curvature of the road, and recognize whether or not it is a specific road section.

[0041] The target steering angle determination unit 120 determines a target steering angle for the traveling direction of the vehicle M based on the surrounding conditions of the vehicle M recognized by the recognition unit 110. For example, the target steering angle determination unit 120 determines a target steering angle for driving the vehicle M in the center of the driving lane based on the shape of the road on which the vehicle M is traveling and the position information of the vehicle M.

[0042] For example, the target steering angle determination unit 120 generates a target trajectory along which the vehicle M will travel in the future, based on the road shape on which the vehicle M is currently traveling, based on the surrounding conditions recognized by the recognition unit 110. The target trajectory may be, for example, a trajectory corresponding to the center of a lane, or a trajectory for turning right or left at an intersection toward a destination. The target trajectory is expressed as a sequence of points (trajectory points) to be reached by the vehicle M, and may include a speed element, etc. Then, the target steering angle determination unit 120 determines a target steering angle for the current position of the vehicle M on the lane so that a reference point (center of gravity, center, etc.) of the vehicle M passes through the generated target trajectory. The target steering angle determination unit 120 continuously determines the target steering angle at predetermined intervals. This makes it possible to determine an appropriate target steering angle according to changes in the road shape. The target steering angle determination unit 120 may also determine a target angle of view when the road on which the vehicle M is traveling is a specific road section. Furthermore, the target steering angle determination unit 120 may determine the target steering angle when the road is curved and the curvature of the road (or the driving lane) is equal to or greater than a threshold value.

[0043] The determination unit 130 includes a steering guidance determination unit 132 and a risk presence / absence determination unit 134. The steering guidance determination unit 132 determines whether or not to provide steering guidance to the driver of the vehicle M based on the target steering angle determined by the target steering angle determination unit 120 and the actual steering angle of the vehicle M detected by the steering angle sensor of the vehicle sensor 10 (hereinafter referred to as the "actual steering angle"). For example, the steering guidance determination unit 132 determines to provide steering guidance when the degree of deviation between the target steering angle and the actual steering angle is equal to or greater than a threshold value (first threshold value), and determines not to provide steering guidance when the degree of deviation is less than the threshold value. The degree of deviation is derived, for example, from the angle difference between the target steering angle and the actual steering angle, and the greater the difference, the greater the degree.

[0044] The risk determination unit 134 determines whether or not there is a possibility that a risk will occur for the vehicle M in the near future (within a predetermined time) based on the surrounding conditions recognized by the recognition unit 110. The risk is, for example, contact between the vehicle M and another object. For example, the risk determination unit 134 derives a time to collision (TTC) using the relative position (relative distance) and relative speed between the vehicle M and another object (e.g., another vehicle), and determines that there is a possibility that a risk will occur (there is a possibility of contact) if the derived time to collision TTC is less than a threshold value (second threshold value), and determines that there is no possibility that a risk will occur (there is no possibility of contact) if the derived time to collision TTC is equal to or greater than the threshold value. The time to collision TTC is a value calculated, for example, by dividing the relative speed from the relative distance.

[0045] The notification control unit 140 causes the speaker 60 to output a corresponding notification sound (a guidance sound, a warning sound, or an OK notification sound, which will be described later) based on the determination result by the determination unit 130. The notification sound is basically composed of a sound that does not include sound (a human voice or a mechanical voice). For example, the notification control unit 140 causes the speaker 60 to output a guidance sound when the steering guidance determination unit 132 determines that steering guidance should be provided to the driver of the vehicle M. Furthermore, the notification control unit 140 causes the speaker 60 to output a warning sound when the risk presence / absence determination unit 134 determines that there is a possibility of a risk occurring. Note that the notification control unit 140 differentiates the output manner between the guidance sound and the warning sound so that the driver can distinguish between them. The output manner is, for example, the type, component, tempo, period, etc. of the sound. Furthermore, the output manner may include the moving speed and moving direction (the positions of the start and end points) of the sound image, etc.

[0046] As an example of a different output mode, the notification control unit 140 causes the speaker 60 to output a warning sound with a higher frequency than the guidance sound. In this case, the notification control unit 140 reduces the harmonic components of the guidance sound compared to the warning sound. Specifically, for example, if the warning sound is composed of a combination of second and third harmonics, the ratio or proportion of the second and third harmonics contained in the guidance sound is reduced compared to the warning sound. Furthermore, if the warning sound is composed only of third harmonics, the guidance sound is configured as a second harmonic or a fundamental tone (single tone), which is less than three times the harmonic. Furthermore, for example, in the case of a warning sound, the notification control unit 140 localizes a sound image so that the warning sound is output from a direction where there is a risk, and in the case of a guidance sound, controls the output of the guidance sound so that the sound image moves in a direction in which the driver is desired to steer. This output mode allows the warning sound to be output at a higher frequency and a higher sound than the guidance sound, making it easier for the driver to notice the warning sound even in a situation where not only one but both the guidance sound and the warning sound are output simultaneously.

[0047] Furthermore, for example, after outputting a guidance sound, if the driver performs a steering operation as guided and the state after the operation is satisfactory, the notification control unit 140 may cause the speaker 60 to output a notification indicating this (OK notification sound). Furthermore, if the steering guidance determination unit 132 determines that the degree of deviation between the target steering angle and the actual steering angle is less than a threshold value and that steering guidance should not be provided to the driver of the vehicle M, the notification control unit 140 causes the speaker 60 to output an OK notification sound in order to maintain the current steering angle. In this case, the OK notification sound has an output manner different from that of the guidance sound and the warning sound, and for example, has fewer harmonic components than the guidance sound and the warning sound.

[0048] Furthermore, the notification control unit 140 may output different notification sounds for different scenes in addition to different sound types of notification sounds. For example, when outputting these notification sounds, the notification control unit 140 refers to the scene-specific output mode table 182 based on the surrounding situation of the vehicle recognized by the recognition unit 110, and determines what kind of sound (sound type) and how to notify (for example, sound image information) based on the surrounding situation (scene type) and the notification content (notification type), and causes the speaker 60 to output the determined sound.

[0049] FIG. 2 is a diagram showing an example of the contents of the scene-specific output mode table 182. In the scene-specific output mode table 182, sound types and sound image information are associated with scene types and notification types. The scene types include information about the surrounding conditions (driving scene) of the vehicle M. The scene types include, for example, types of specific road sections such as curved roads, intersections, and cranks. The notification types include information about the content of the notification (for example, steering guidance, warning (risk alert), and maintaining the current state (OK notification)). The sound types may include the type of notification sound (guidance sound, warning sound, OK notification sound) as well as the constituent content of the sound, and may store the sound data itself. The sound image information includes, for example, information about whether to fix the sound image, the moving position (starting point, ending point positions), the moving speed, the period, etc. The moving position is three-dimensional position information and may include information about the moving direction. The sound image information may also include information about the tempo and volume of the sound. Furthermore, the scene-specific output mode table 182 may include audio information (human voice or mechanical voice) in addition to the notification sound.

[0050] The information included in the scene-specific output mode table 182 may be preset as fixed information or may be arbitrarily set by the driver. In the case where it is set by the driver, for example, information received by the display device (touch panel) 70 is registered (or updated) in the scene-specific output mode table 182.

[0051] Furthermore, instead of (or in addition to) obtaining a notification sound according to the vehicle state or surrounding circumstances by referring to the scene-specific output mode table 182, the notification control unit 140 may cause the speaker 60 to output the notification sound via a device capable of generating and synthesizing a desired sound, such as a synthesizer. Furthermore, in addition to (or instead of) the notification sound, the notification control unit 140 may generate an image that prompts the driver to perform a steering operation or an image that prompts the driver to maintain the current actual steering angle, and display the image on the display device 70.

[0052] [About driving assistance via notification sounds] Next, a description will be given of driving assistance using a notification sound according to an embodiment. In the following, a curved road is used as an example of a specific road section, and assistance will be provided to enable the driver to drive the vehicle M without deviating from the curved road.

[0053] Fig. 3 is a diagram showing a scene in which a vehicle M is traveling on a curved road. In the example of Fig. 3, the vehicle M is traveling at a speed VM in a lane L1 defined by left and right road dividing lines LL and RL. In the example of Fig. 3, it is assumed that a target trajectory K1 is generated by the target steering angle determination unit 120, and a target steering angle is determined based on the target trajectory K1 and the position of the vehicle M.

[0054] In the scene shown in Figure 3, for example, a conventional navigation device may output a voice message such as "Curved road. Please be careful." However, with such a message, the driver cannot grasp the curvature of the curved road, and in the case of a relatively long curved road or a meandering curved road (S-shaped curve) such as a mountain road, the driver may not be able to grasp when and how much steering is required, which may prevent appropriate driving assistance from being provided.

[0055] Therefore, the driving assistance device 100 of the embodiment changes the output mode of the guidance sound depending on the degree of deviation between the target steering angle and the actual steering angle, thereby providing the driver with appropriate steering guidance. Also, the driving assistance device 100 outputs the guidance sound and the warning sound in different output modes so that the driver does not confuse them. This allows for more appropriate driving assistance.

[0056] For example, the notification control unit 140 localizes the sound images of the guidance sound and the warning sound, and further moves the localization position of the sound image of the guidance sound. Localizing a sound image means, for example, determining the spatial position of the sound source perceived by the driver by adjusting the volume of the sound transmitted to the driver's left and right ears. The sound image can be determined, for example, based on the sound characteristics originally possessed by the sound source, information about the vehicle interior environment, and a head-related transfer function (HRTF). Using such principles, the sound image can be localized at a predetermined position. The notification control unit 140 has a function, for example, to adjust the volume of sounds output from multiple sub-speakers provided in the vehicle interior and to synthesize multiple input sounds.

[0057] Fig. 4 is a diagram for explaining the localization of a sound image. In Fig. 4, as an example, a sub speaker 60A is installed in front of a driver P1 who drives a vehicle M (for example, near the center of an instrument panel in the vehicle compartment or on the ceiling), a sub speaker 60B is installed to the left of the driver P1 (for example, on the left door or on the left side of the instrument panel as seen from the driver P1), and a sub speaker 60C is installed to the right of the driver P1 (for example, on the right door or on the right side of the instrument panel as seen from the driver P1), but the sound image may be localized using other sub speakers provided in the vehicle compartment, or the localization position may be moved spatially.

[0058] For example, when a sound image of the guiding sound is localized at a spatial position MPa defined by three-dimensional coordinates (X, Y, Z) shown in FIG. 4, the notification control unit 140 outputs the guiding sound at 5% of the maximum intensity from the sub-speaker 60C, at 80% of the maximum intensity from the sub-speaker 60B, and at 15% of the maximum intensity from the sub-speaker 60A. As a result, from the position of the driver P1, the sound image feels as if it is localized at the spatial position MPa shown in FIG. 4. Similarly, when a sound image of the guiding sound is localized at a spatial position MPb, the notification control unit 140 outputs the guiding sound at 45% of the maximum intensity from the sub-speakers 60B and 60C, and at 45% of the maximum intensity from the sub-speaker 60A. As a result, from the position of the driver P1, the sound image feels as if it is localized at the spatial position MPb shown in FIG. 4. In this way, by adjusting the multiple sub-speakers installed in the vehicle cabin and the volume of the sound output from each sub-speaker, the position where the sound image is localized can be fixed or moved. In the following explanation, the position in space where the sound image is localized will be referred to as the "sound image position MP."

[0059] For example, the notification control unit 140 moves the sound image position of the guiding sound according to the degree of deviation between the target steering angle and the actual steering angle. FIG. 5 is a diagram for explaining how the sound image position MP of the guiding sound moves when the degree of deviation is large (when the degree of deviation is equal to or greater than a first predetermined value). The example of FIG. 5 shows how the sound image position MP moves when the driver P1 wants to increase the actual steering angle of the vehicle M to the left by steering. In the example of FIG. 5, time t1 is assumed to be the earliest, followed by times t2, t3, and t4 in that order. In addition, in FIG. 5, MP(t*) represents the sound image position at time t*. The same applies to the subsequent figures.

[0060] For example, when the actual steering angle is smaller than the target steering angle and the driver P1 is prompted to steer to the left (when the driver P1 is prompted to turn the steering wheel to the left), the notification control unit 140 determines the start point and end point for moving the sound image of the guiding sound based on the degree of deviation. For example, the start point is determined based on the actual steering angle of the vehicle M, and the end point is determined based on the target steering angle. Therefore, the greater the degree of deviation, the greater the distance D1 from the start point to the end point. Note that the start point and the end point may be set to fixed positions based on the position of the driver P1. In the example of FIG. 5, the sound image position MP(t1) indicates the start point, and the sound image position MP(t4) indicates the end point.

[0061] Next, the notification control unit 140 determines a moving speed V1 of the sound image position MP according to the degree of deviation, and moves the sound image position MP of the guiding sound at the determined moving speed V1. In the example of Fig. 5, the sound image position MP is moved in the horizontal direction from the start point to the end point at the speed V1 from time t1 to t4. The guiding sound is continuously output during the movement. The notification control unit 140 also repeatedly outputs the movement of the sound image positions MP(t1) to MP(t4) of the guiding sound at a period according to the degree of deviation.

[0062] 6 is a diagram illustrating the movement of the sound image position MP of the guiding sound when the degree of deviation is small (less than the first predetermined value and equal to or greater than the second predetermined value) (note that the first predetermined value>the second predetermined value). In this case, the notification control unit 140 reduces at least one of the movement speed V1 and the movement distance D1 of the sound image position MP compared to the movement shown in FIG. 5. Furthermore, instead of (or in addition to) the above-described contents of the movement speed V1 and the movement distance D1, the notification control unit 140 may slow down the cycle (output cycle) at which the guiding sound is output.

[0063] That is, the notification control unit 140 increases the distance between the start point and the end point (distance D1 shown in FIG. 5) or the movement speed V1, or shortens the output cycle, as the degree of deviation increases. Furthermore, the notification control unit 140 may control the volume of the guiding sound to increase or the frequency of the guiding sound to be output, as the degree of deviation increases. Furthermore, the notification control unit 140 may control the movement distance D1 or the movement speed V1 of the sound image position MP to increase or the output cycle to shorten as the curvature of the driving lane L1 increases. Furthermore, the notification control unit 140 may adjust at least one of the movement distance D1, the movement speed V1, and the output cycle of the sound image position MP, depending on the width of the driving lane L1, the size of the vehicle M, etc.

[0064] In this way, by changing the output mode of the guidance sound according to the degree of deviation between the target steering angle and the actual steering angle, the driver P1 can be made to more clearly understand the degree of deviation. Furthermore, by moving the sound image position of the guidance sound to the left, the driver P1 can be made to more accurately recognize that a steering operation that increases the actual steering angle of the vehicle M to the left (more specifically, an operation to turn the steering wheel to the left) is necessary. For example, if a message voice urging a steering operation to the left is output, the driver P1 may confuse the left and steer the vehicle M. However, by instructing the steering direction by moving the sound image position of the guidance sound as in the present invention, it is possible to prevent the driver P1 from confusing the left and right (misrecognition).

[0065] Furthermore, if the degree of deviation is less than the second threshold value (if the target steering angle and the actual steering angle match (within a predetermined tolerance range)), the notification control unit 140 stops outputting the guiding sound. Furthermore, the notification control unit 140 may output an OK notification sound from the speaker 60 instead of the guiding sound. Whether or not to output the OK notification sound may be set by the driver P1, or may be set based on the past driving history of the driver P1.

[0066] FIG. 7 is a diagram for explaining the sound image position in the case of an OK notification sound. When outputting the OK notification sound, the notification control unit 140 sets a sound image position MP according to the target steering angle or the actual steering angle, and repeatedly outputs the OK notification sound from the set position (fixed position). In the example of FIG. 7, the sound image is localized at a position on the left front according to the actual steering angle of the driver P1. Outputting the OK notification sound can make the driver P1 recognize that the current steering operation is appropriate.

[0067] Furthermore, regardless of whether a guiding sound or an OK notification sound is being output, the notification control unit 140 outputs a warning sound to a predetermined sound image position when the risk presence determination unit 134 determines that there is a possibility of a risk occurring. The predetermined position may be, for example, a direction in which there is a risk relative to a reference point (center of gravity or center) of the vehicle M, or a direction in which there is a risk relative to the position of the driver P1 (position of the driver's seat). The direction in which there is a risk is, for example, a direction in which there is an object determined to have the possibility of coming into contact with the vehicle M.

[0068] FIG. 8 is a diagram for explaining the positions of sound image localization of the guiding sound and the warning sound. In the example of FIG. 8, sound image position MP1 indicates the sound image position of the guiding sound, and sound image position MP2 indicates the sound image position of the warning sound. The example of FIG. 8 also shows how the sound image position MP1 of the guiding sound moves when the degree of deviation is large (greater than or equal to a first predetermined value). When the guiding sound is output in the output mode shown in FIG. 8 and the risk presence / absence determination unit 134 determines that there is a possibility of a risk occurring, the notification control unit 140 localizes the sound image of the warning sound in the direction of an object (target object) that may come into contact with the driver P1 while outputting the guiding sound. This makes it possible to more accurately notify the driver P1 of the possibility of a risk occurring and the direction of the risk. Note that in the example of FIG. 8, the sound image position MP2 of the warning sound is fixed, but the sound image position MP2 may be moved depending on the relative positions of the vehicle M and the target object. Furthermore, the notification control unit 140 may adjust the volume and cycle of the warning sound according to the time to contact TTC.

[0069] Furthermore, as shown in Fig. 8, even when both the guidance sound (or OK notification sound) and the warning sound are notified, the driver P1 can be made aware of both the steering guidance and the warning because the guidance sound and the warning sound are different sounds. Furthermore, the warning sound is output as a higher sound (high-frequency sound) with more harmonic components than the guidance sound, so the risk alert (warning of the possibility of contact) can be more reliably notified to the driver P1.

[0070] When outputting a warning sound, the notification control unit 140 may terminate output of the guiding sound or the OK notification sound, or may output the guiding sound but terminate output of the OK notification sound because it is less important. The notification control unit 140 may also increase the volume of the warning sound higher than the volume of the guiding sound or the OK notification sound, or may decrease the volume of the guiding sound or the OK notification sound lower than the volume of the warning sound.

[0071] [Variations] In the embodiment, the sound image position MP of the notification sound may be moved with the steering wheel as a reference, instead of moving it horizontally as described above. FIG. 9 is a diagram for explaining how the sound image position MP is moved with the rotation direction of the steering wheel as a reference. The example of FIG. 9 shows the steering wheel SW that the driver P1 grips and steers. The example of FIG. 9 also shows how the sound image position MP moves from time t1 to time t3. For example, when outputting a guidance sound to increase the steering angle of the vehicle M to the left, the notification control unit 140 moves the sound image position MP near the steering wheel SW that is operated (within a predetermined distance from the steering wheel SW). For example, the notification control unit 140 rotates the sound image position MP counterclockwise from the start point of the sound image (sound image position MP(t1)) set above the steering wheel SW around the same rotation axis as the rotation axis C1 of the steering wheel SW. The angle θ1 of rotation during one cycle and the movement speed V1 are determined according to the degree of deviation.

[0072] In this way, by moving the sound image position MP so as to rotate the vicinity of the steering wheel SW on the same rotation axis as the rotation axis C1 of the steering wheel SW, the driver P1 can easily understand that an operation on the steering wheel SW, that is, a steering operation, is required, and the direction of movement of the sound image position MP can easily help the driver P1 to understand the steering direction (the direction in which to turn the steering wheel SW). This allows the driver P1 to drive appropriately without making a mistake in the steering direction.

[0073] Furthermore, when the vehicle M is moving backward (reverse driving, backing up), the notification control unit 140 may output from the speaker 60 a notification sound (reverse sound, reverse sound) that notifies the driver that the vehicle M is moving backward. When moving backward, for example, the shift lever is positioned at a position for moving the vehicle M backward (for example, the "R" position). Furthermore, when performing steering guidance when moving the vehicle M backward, the notification control unit 140 may output the above-mentioned guidance sound, OK notification sound, and warning sound in the above-mentioned output manner to provide driving assistance. In this case, the guidance sound and OK notification sound may be generated based on the constituent components of the reverse sound. In other words, the guidance sound and notification sound that are notified when the vehicle M is moving backward are composed of sound components similar to the reverse sound. This makes it possible to perform steering guidance while notifying the driver that the vehicle M is moving backward.

[0074] In addition, in the above embodiment, an example was shown in which the actual steering angle was increased to the left, but when the actual steering angle is increased to the right, or when the vehicle passes through a left curve and enters a straight road, a guidance sound is notified in which the sound image position moves to the right.

[0075] Furthermore, because the reaction speed after the notification sound is output varies depending on the driver, the notification control unit 140 may adjust the timing to start outputting the notification sound for each driver. In this case, the notification control unit 140 stores the past driving history for each driver and adjusts the timing to start outputting the notification sound based on the change trend of the error between the target steering angle and the actual steering angle based on the stored driving history. The change trend may be derived for each scene. For example, when the notification control unit 140 determines that the driver's reaction speed is slower than normal (reference speed) based on the change trend, the notification control unit 140 reduces the first threshold value, which is compared with the deviation degree to determine whether steering guidance should be performed, below a reference value, to make it easier to output the guidance sound at an earlier timing. When the notification control unit 140 determines that the driver's reaction speed is slow, the notification control unit 140 increases the second threshold value, which is compared with the time to contact (TTC) to determine whether there is a possibility of a risk, above a reference value to make it easier to output a warning at an earlier timing.

[0076] Furthermore, the notification control unit 140 may learn the change tendency for each driver in response to updates to the driving history, and adjust the output timing of the notification sound based on the learning results. During the learning, the change tendency for each driver may be updated using, for example, AI (Artificial Intelligence) functions such as machine learning (neural network) and deep learning using teacher (correct answer) data. By adjusting the start timing of the notification sound in this way, more appropriate driving assistance can be provided for each driver.

[0077] Furthermore, the notification control unit 140 may change the output mode of the guidance sound or warning sound depending on the speed of the vehicle M. In this case, the notification control unit 140 may output a guidance sound (speed guidance sound) for speed guidance from the speaker 60 in addition to (or instead of) steering guidance using the guidance sound. The speed guidance sound is configured as a sound different from the guidance sound for steering guidance, the OK notification sound, and the warning sound. For example, the determination unit 130 determines whether to prompt the driver to either accelerate or decelerate depending on the degree of deviation between the speed of the vehicle M and a target speed (for example, the legal speed limit for the driving lane or the average speed of surrounding vehicles). Then, when it is determined that the driver should be prompted to perform either acceleration or deceleration, the notification control unit 140 outputs a notification sound (speed guidance sound) from the speaker 60 that prompts the driver to either accelerate or decelerate. In this case, the speed guidance sound is, for example, a sound whose configuration is continuously changed, and the continuously changing sound configuration is made different when the vehicle M is accelerating and when the vehicle M is decelerating. The continuously changing sound is, for example, a sound whose frequency or pitch is continuously increased or decreased. Furthermore, the speed guidance sound, for example, localizes the sound image in front of the driver when guiding acceleration, and localizes the sound image behind the driver when guiding deceleration. This allows for driving assistance through acceleration / deceleration guidance.

[0078] Furthermore, in addition to outputting the above-mentioned notification sound, the notification control unit 140 may obtain and output audio information (human voice or mechanical voice) corresponding to the notification content from the scene-specific output mode table 182. For example, if the degree of deviation between the target steering angle and the actual steering angle does not decrease even after a predetermined time has elapsed since the output of the guidance sound, the notification control unit 140 outputs audio information corresponding to the guidance content from the speaker 60. This allows the driver to understand the content of the guidance sound more accurately.

[0079] [Processing flow] Fig. 10 is a flowchart showing an example of the flow of processing executed by the driving assistance device 100 of the embodiment. Note that, in the example of Fig. 10, the explanation will be focused mainly on processing related to steering guidance and notification of a notification sound in response to a risk alert, among the processing executed by the driving assistance device 100. The processing shown below may be repeatedly executed at a predetermined timing or at a predetermined cycle, for example, while the driver is driving the vehicle M.

[0080] In the example of FIG. 10, the recognition unit 110 recognizes the surrounding conditions of the vehicle M (step S100). Next, the recognition unit 110 determines whether the road on which the vehicle M is traveling is a curved road (an example of a specific road section) (step S102). If it is determined that the road is a curved road, the target steering angle determination unit 120 determines a target steering angle based on the shape of the road on which the vehicle M is traveling and the position of the vehicle M (step S104). Next, the steering guidance determination unit 132 determines whether to provide steering guidance to the driver of the vehicle M based on the degree of deviation between the target steering angle and the actual steering angle obtained from the vehicle sensor 10 (step S106). If it is determined that steering guidance is to be provided, the notification control unit 140 causes the speaker 60 to output a guidance sound in an output mode corresponding to the degree of deviation between the target steering angle and the actual steering angle (step S108).

[0081] After the processing of step S108, if it is determined in the processing of step S102 that the road is not a curved road, or if it is determined in the processing of step S106 that steering guidance will not be performed, the risk presence / absence determination unit 134 determines whether or not there is a possibility of a risk occurring to the vehicle M based on the situation around the vehicle M (step S110). If it is determined that there is a possibility of a risk occurring, the notification control unit 140 causes the speaker 60 to output a warning sound in an output mode different from the guidance sound (step S112). After the processing of step S112, or if it is determined that there is no possibility of a risk occurring in the processing of step S110, the processing of this flowchart ends.

[0082] 10, the processing of steps S102 to S108 and the processing of steps S110 to S112 may be executed in the reverse order or may be executed in parallel. Also, when it is determined in the processing of step S106 that steering guidance will not be performed, the notification control unit 140 may cause the speaker 60 to output an OK notification sound.

[0083] According to the first embodiment described above, the driving assistance device 100 includes a recognition unit 110 that recognizes the surrounding conditions of a vehicle (an example of a moving body) M, a judgment unit 130 that judges, based on the surrounding conditions, whether to prompt the driver of the vehicle M to perform a steering operation and whether there is a possibility of a risk to the vehicle M, and a notification control unit 140 that, based on the judgment result by the judgment unit 130, causes a speaker to output a guidance sound that prompts the driver of the vehicle M to perform a steering operation and a warning sound for the risk, and the notification control unit 140 differentiates the output manner for the guidance sound and the warning sound. The driving assistance device 100 also includes the above-described recognition unit 110, a target steering angle determination unit 120 that determines a target steering angle relative to the traveling direction of the vehicle M based on the surrounding circumstances, and a notification control unit 140 that causes a speaker to output a guidance sound that prompts the driver of the vehicle M to perform a steering operation based on the target steering angle determined by the target steering angle determination unit 120 and the current steering angle of the vehicle M, and the notification control unit 140 varies the output mode of the guidance sound based on the degree of deviation between the target steering angle and the current steering angle of the moving object. This allows for more appropriate driving assistance to be provided by notification using a sound that corresponds to the surrounding circumstances. This can therefore contribute to the development of a sustainable transportation system.

[0084] According to the first embodiment, the guidance sound can be output while changing the output mode according to the degree of steering operation. According to the first embodiment, sound-based steering guidance is provided in each scene of a specific road section such as a curved road or an intersection, thereby helping to improve the driver's driving ability.

[0085] Second Embodiment The second embodiment will be described below. The driving assistance device of the second embodiment is mounted on an autonomous vehicle. Autonomous driving, for example, refers to automatically controlling one or both of the steering and speed of the vehicle to perform driving control. An autonomous vehicle may be driven by manual operation by a vehicle user (e.g., a driver) (so-called manual driving). FIG. 11 is a diagram showing devices mounted on a vehicle M, focusing on a driving assistance device 100A according to the second embodiment. The driving assistance device 100A is mounted on the vehicle MA together with an autonomous driving control device 160. Output data from the vehicle sensor 10, the navigation device 20, and the object detection device 30 is also input to the autonomous driving control device 160. In addition to these, output data from the navigation device and an MPU (Micro Processing Unit) may also be input to the autonomous driving control device 160. For example, the automatic driving control device 160 recognizes the surrounding conditions of the vehicle MA, recognizes an area in which the vehicle MA can travel near a predetermined position relative to the lane (for example, the center of the lane) while avoiding contact with obstacles, generates a target trajectory (with a speed element attached) for the vehicle MA, and controls the traveling drive force output device 200, braking device 210, steering device 220, etc. so that the vehicle MA can travel along the target trajectory. The traveling drive force output device 200 includes an engine, a traveling motor, etc. Details of automatic driving technology are publicly known in various documents, so further explanation will be omitted.

[0086] The autonomous driving control device 160 is further connected to a steering grip sensor 170 and an interior camera 175. The steering grip sensor 170 detects whether the driver is gripping the steering wheel, which is a steering control element, and the interior camera 175 captures an image of the driver's head from the front. The autonomous driving control device 160 determines whether the driver is gripping the steering wheel (hands-on) by referring to the output of the steering grip sensor 170, and determines whether the driver is looking at the direction in which the vehicle MA is traveling (eyes-on) by analyzing the image captured by the interior camera 175. The autonomous driving control device 160 allows either or both of the following, based on the surrounding environment of the vehicle MA (the type of road on which the vehicle is traveling) and the speed of the vehicle MA. This allowable state is sometimes referred to as the autonomous driving level. When an impermissible state occurs (for example, when the driver takes his / her hands off the steering wheel in a hands-on state), the automatic driving control device 160 notifies the driver to resolve the state, and if the state is not resolved, performs processing such as switching to manual driving. The automatic driving control device 160 outputs information related to the automatic driving level (automatic driving level information) to the driving assistance device 100A.

[0087] The driving assistance device 100A refers to the autonomous driving level information and automatically stops operation if both hands-off and eyes-off are permitted. In this state, the driver is temporarily no longer required to monitor the periphery of the vehicle MA, and the autonomous driving control device 160 performs control to avoid contact with risk objects. Other functions are the same as those of the first embodiment. In other words, when the driving assistance device 100A determines based on the autonomous driving level information that manual driving will be performed in the autonomously driven vehicle (vehicle MA), it performs notification control of the notification sound based on the state and surrounding circumstances of the vehicle MA, similar to the driving assistance device 100 of the first embodiment.

[0088] According to the second embodiment described above, the driving assistance device 100A of the embodiment can be applied to an autonomous driving vehicle. Therefore, according to the second embodiment, in addition to achieving the same effects as the first embodiment, it is possible to perform control that is highly compatible with autonomous driving.

[0089] <Third embodiment> The third embodiment will be described below. The driving assistance device of the third embodiment is mounted on a motorcycle that is ridden by a rider wearing a helmet. FIG. 12 is a diagram showing devices mounted on a motorcycle MB, focusing on a driving assistance device 100B according to the third embodiment. The driving assistance device 100B differs from the driving assistance device 100 of the first embodiment in that it includes a wireless communication device 80 instead of controlling the speaker 60. The driving assistance device 100B instructs the wireless communication device 80 to transmit instruction information related to a notification sound. The wireless communication device 80 communicates based on a communication standard such as Bluetooth (registered trademark).

[0090] A receiver 310, a left ear speaker 320-L, and a right ear speaker 320-R are attached to a helmet 300 worn by the driver. The helmet 300 may be provided with at least one sub-speaker in front of the head and one on each of the left and right sides. A plurality of sub-speakers may be provided as surround speakers surrounding the head so that surround sound can be provided inside the helmet 300.

[0091] The driving assistance device 100B performs the same processing as described in the first embodiment to determine a notification sound (guidance sound, warning sound, OK notification sound) according to the state of the motorcycle MB and the surrounding circumstances, and transmits instruction information to the wireless communication device 80 to cause the left ear speaker 320-L and / or the right ear speaker 320-R to output the notification sound accordingly. The receiver 310 outputs the notification sound from the left ear speaker 320-L, the right ear speaker 320-R, and / or one or both of the sub-speakers based on the instruction information received from the wireless communication device 80. In the third embodiment, earphones with wireless functionality may be used instead of the speakers provided in the helmet.

[0092] In the third embodiment, the degree of freedom in sound image localization is somewhat reduced compared to the first and second embodiments. Therefore, the notification control unit 140 may output the guidance sound for steering guidance to the left from the left ear speaker 320-L and the guidance sound for steering guidance to the right from the right ear speaker 320-R. In this case, the notification control unit 140 may adjust the volume and cycle according to the degree of deviation between the target steering angle and the actual steering angle. Furthermore, when outputting a warning sound, the notification control unit 140 outputs the warning sound from the left ear speaker 320-L if a target object with a possibility of contact is present on the left side as viewed from the traveling direction (front direction) of the vehicle MB, and outputs the warning sound from the right ear speaker 320-R if the target object is present on the right side. Furthermore, the notification control unit 140 may adjust the volume and cycle of the warning sound according to the contact margin time TTC.

[0093] According to the third embodiment described above, although the degree of freedom in sound image localization is somewhat reduced, in other respects it is possible to achieve the same effects as the first embodiment.

[0094] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: Recognize the surrounding situation of the moving object, determining whether to prompt the driver of the moving body to perform a steering operation and whether there is a possibility of a risk to the moving body based on the surrounding situation; outputting, from a speaker, a guidance sound that prompts the driver of the moving body to steer and a warning sound against the risk based on the determination result by the determination unit; The notification control unit outputs the guiding sound and the warning sound in different manners. Driving assistance device.

[0095] The above-described embodiment can also be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: Recognize the surrounding situation of the moving object, determining a target steering angle with respect to the traveling direction of the moving body based on the surrounding situation; outputting, from a speaker, a guidance sound that prompts a driver of the moving body to perform a steering operation based on the determined target steering angle and the current steering angle of the moving body; varying an output mode of the guiding sound based on a degree of deviation between the target steering angle and a current steering angle of the moving body; Driving assistance device.

[0096] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0097] 10 Vehicle Sensors 20 Navigation devices 30 Object detection device 40 Driving controls 60 speakers 80 Wireless communication equipment 100, 100A, 100B Driving support device 110 Recognition part 120 Target steering angle determination unit 130 Judgment section 132 Steering guidance judgment unit 134 Risk Assessment Department 140 Notification control section 160 Automatic driving control device 170 Steering grip sensor 175 In-car camera 300 helmet

Claims

1. a recognition unit that recognizes the surrounding situation of the moving object; a determination unit that determines whether to prompt the driver of the moving body to perform a steering operation and whether there is a possibility of a risk to the moving body based on the surrounding situation; a notification control unit that outputs, from a speaker, a guidance sound that prompts the driver of the moving body to perform a steering operation and a warning sound regarding the risk based on the determination result by the determination unit; the notification control unit outputs the guiding sound and the warning sound in different manners, a target steering angle determination unit that determines a target steering angle with respect to a traveling direction of the moving body based on the surrounding situation, the notification control unit controls an output mode of the guiding sound based on a degree of deviation between the target steering angle and a current steering angle of the moving body; the notification control unit causes the speaker to output a notification sound indicating that the current steering angle should be maintained when a deviation between the target steering angle and the current steering angle of the moving object is less than a threshold value; Driving assistance device.

2. The notification control unit causes the speaker to output the warning sound having a higher frequency than the guiding sound. The driving assistance device according to claim 1 .

3. The guiding sound has fewer harmonic components than the warning sound. The driving assistance device according to claim 2 .

4. The notification control unit controls the output of the guidance sound so that a sound image moves in a direction in which the driver is desired to steer, based on the driver. The driving assistance device according to claim 1 .

5. The notification control unit localizes a sound image so that the warning sound is output from a direction in which the risk exists relative to the driver. The driving assistance device according to claim 1 .

6. the notification control unit changes at least one of a moving position and a moving speed of the sound image of the guiding sound based on the degree of deviation. The driving assistance device according to claim 1 .

7. the notification control unit determines a start point and an end point when moving the sound image of the guiding sound based on the degree of deviation. The driving assistance device according to claim 1 .

8. the starting point is determined based on a current steering angle of the moving body; The end point is determined based on the target steering angle. The driving assistance device according to claim 7.

9. the notification control unit localizes a sound image of the notification sound at a position corresponding to the target steering angle or the current steering angle of the moving body. The driving assistance device according to claim 1 .

10. When moving a sound image of the guiding sound, the notification control unit moves the sound image in a horizontal direction or in a rotation direction of a driving operator that performs the steering operation. The driving assistance device according to claim 1 .

11. When causing the speaker to output the guiding sound when the moving body is moving backward, the notification control unit generates the guiding sound based on components of a reverse movement sound output from the speaker when the moving body is moving backward. The driving assistance device according to claim 1 .

12. a recognition unit that recognizes the surrounding situation of the moving object; a target steering angle determination unit that determines a target steering angle with respect to the traveling direction of the moving body based on the surrounding situation; a notification control unit that outputs, from a speaker, a guidance sound that prompts a driver of the moving body to perform a steering operation, based on the target steering angle determined by the target steering angle determination unit and a current steering angle of the moving body; the notification control unit varies an output mode of the guiding sound based on a degree of deviation between the target steering angle and a current steering angle of the moving body, the notification control unit causes the speaker to output a notification sound indicating that the current steering angle should be maintained when a deviation between the target steering angle and the current steering angle of the moving object is less than a threshold value; Driving assistance device.

13. a recognition unit that recognizes the surrounding situation of the moving object; a determination unit that determines whether to prompt the driver of the moving body to perform a steering operation and whether there is a possibility of a risk to the moving body based on the surrounding situation; a notification control unit that outputs, from a speaker, a notification sound including at least one of a guidance sound that prompts the driver of the moving body to perform a steering operation and a warning sound for the risk, based on the determination result by the determination unit; the notification control unit outputs the guiding sound and the warning sound in different manners, The notification control unit acquires a change trend of an error between a target steering angle with respect to a traveling direction of the moving body and an actual steering angle of the driver based on a past driving history of the driver, and adjusts a timing to start outputting the notification sound based on the acquired change trend. Driving assistance device.

14. a recognition unit that recognizes the surrounding situation of the moving object; a determination unit that determines whether to prompt the driver of the moving object to accelerate or decelerate the moving object based on the surrounding situation, and whether there is a possibility that a risk will occur to the moving object; a notification control unit that outputs, from a speaker, a guidance sound that prompts the driver of the moving body to either accelerate or decelerate, and a warning sound for the risk, based on the determination result by the determination unit; the notification control unit outputs the guiding sound and the warning sound in different manners, the notification control unit outputs a guidance sound that accelerates or decelerates the moving object according to a degree of deviation between a speed of the moving object and a target speed of the moving object based on the surrounding situation. Driving assistance device.

15. The computer Recognize the surrounding situation of the moving object, determining whether to prompt the driver of the moving body to perform a steering operation and whether there is a possibility of a risk to the moving body based on the surrounding situation; Based on the result of the determination, a guidance sound for prompting the driver of the moving body to perform a steering operation and a warning sound for the risk are output from a speaker; The guiding sound and the warning sound are output in different manners, determining a target steering angle relative to the traveling direction of the moving body based on the surrounding conditions; controlling an output mode of the guiding sound based on a degree of deviation between the target steering angle and a current steering angle of the moving body; When the degree of deviation between the target steering angle and the current steering angle of the moving body is less than a threshold value, a notification sound indicating that the current steering angle should be maintained is output from the speaker. Driving assistance methods.

16. The computer Recognize the surrounding situation of the moving object, determining a target steering angle with respect to the traveling direction of the moving body based on the surrounding situation; outputting, from a speaker, a guidance sound that prompts a driver of the moving body to perform a steering operation based on the determined target steering angle and the current steering angle of the moving body; varying an output mode of the guiding sound based on a degree of deviation between the target steering angle and a current steering angle of the moving body; When the degree of deviation between the target steering angle and the current steering angle of the moving body is less than a threshold value, a notification sound indicating that the current steering angle should be maintained is output from the speaker. Driving assistance methods.

17. On the computer, It allows the moving object to recognize its surroundings, determining whether to prompt the driver of the moving body to perform a steering operation and whether there is a possibility that a risk will occur to the moving body based on the surrounding situation; Based on the result of the determination, a guidance sound for prompting the driver of the moving body to perform a steering operation and a warning sound for the risk are output from a speaker; The guiding sound and the warning sound are output in different manners, determining a target steering angle with respect to the traveling direction of the moving body based on the surrounding situation; controlling an output mode of the guiding sound based on a degree of deviation between the target steering angle and a current steering angle of the moving body; When the degree of deviation between the target steering angle and the current steering angle of the moving body is less than a threshold value, a notification sound indicating that the current steering angle should be maintained is output from the speaker. program.

18. On the computer, It allows the moving object to recognize its surroundings, determining a target steering angle with respect to the traveling direction of the moving body based on the surrounding situation; outputting, from a speaker, a guidance sound that prompts a driver of the moving body to perform a steering operation based on the determined target steering angle and the current steering angle of the moving body; varying an output mode of the guiding sound based on a degree of deviation between the target steering angle and a current steering angle of the moving body; When the degree of deviation between the target steering angle and the current steering angle of the moving body is less than a threshold value, a notification sound indicating that the current steering angle should be maintained is output from the speaker. program.

Images