Information notification device, vehicle system, information notification method, and program

Abstract

To realize information notification allowing a user to hardly feel troublesome while maintaining provision of information required for operation of a vehicle.SOLUTION: An information notification device according to an embodiment includes a controller. The controller selects a notification medium to be used for the information notification based on the driver's tolerance for information notifications in the vehicle and the necessity for recognizing the information to be notified, and performs the information notification using the selected notification medium.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The disclosed embodiments relate to an information notification device, a vehicle system, an information notification method, and a program. [Background technology]

[0002] BACKGROUND ART Conventionally, there is known an information notification device that notifies a driver of a vehicle of information to call his or her attention when making a left turn, which may result in a hit-and-run accident (see, for example, Patent Document 1).

[0003] Such information notification is achieved by combining, for example, an audio notification from a speaker such as "Turning left. Be careful not to get hit by other vehicles," and a visual notification that displays text such as "Be careful not to get hit by other vehicles!" on a display unit (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-102529 [Patent Document 2] Japanese Patent Application Laid-Open No. 2007-122142 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned conventional technology has room for further improvement in terms of realizing information notification that is less bothersome to the user while still providing information necessary for vehicle operation.

[0006] For example, when using the above-described conventional technology, audio and visual notifications are always provided in situations where a driver needs to be alerted, such as when turning left, which can result in excessive information notifications and can be annoying to the user. Furthermore, a user who feels this is annoying may turn off the information notification function. If the user turns off the information notification function, it will no longer be possible to provide information necessary for safe vehicle operation, such as alerts.

[0007] The present invention has been made in consideration of the above, and aims to provide an information notification device, a vehicle system, an information notification method, and a program that can realize information notification that is less bothersome to the user while maintaining the provision of information necessary for vehicle operation. [Means for solving the problem]

[0008] The information notification device according to the present invention includes a controller that selects a notification medium to be used for the information notification in accordance with the driver's tolerance for information notifications in the vehicle and the driver's need to recognize the information to be notified, and notifies the information using the selected notification medium. [Effects of the Invention]

[0009] According to the present invention, a notification medium to be used for information notification is selected depending on the driver's tolerance, including the driver's proficiency, and their cognitive need, including the level of danger. That is, the notification medium can be selected depending on the vehicle operation status and the driver's level of familiarity with information notifications, using the tolerance and the cognitive need as parameters. As a result, according to the present invention, it is possible to realize information notification that is less bothersome to the user while maintaining the provision of information necessary for vehicle operation. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an outline of an information notification method according to an embodiment. [Figure 2] FIG. 2 is an explanatory diagram of a normal notification. [Figure 3] FIG. 3 is an explanatory diagram of the detailed notification. [Figure 4] FIG. 4 is an explanatory diagram of simple notification. [Figure 5] FIG. 5 is a diagram showing an example of an operation for determining a notification method based on the risk level and the proficiency level. [Figure 6] FIG. 6 is an explanatory diagram of a method using a notification method determination data table. [Figure 7] FIG. 7 is an explanatory diagram of a method using a notification method determination condition data table. [Figure 8] FIG. 8 is a diagram illustrating an example of the configuration of a vehicle system according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of user information. [Figure 10] FIG. 10 is a diagram showing an emotion type table. [Figure 11] FIG. 11 is a functional block diagram of the controller. [Figure 12] FIG. 12 is a flowchart (part 1) illustrating a processing procedure executed by the in-vehicle device according to the embodiment. [Figure 13] FIG. 13 is a flowchart (part 2) illustrating the processing procedure executed by the in-vehicle device according to the embodiment. [Figure 14] FIG. 14 is a diagram (part 1) showing an example of updating notification method information. [Figure 15] FIG. 15 is a diagram (part 2) showing an example of updating the notification method information. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of an information notification device, a vehicle system, an information notification method, and a program disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

[0012] In the following, the information notification device according to the embodiment is an in-vehicle device 10 (see FIG. 1) included in the vehicle system 1 according to the embodiment. The information notification method according to the embodiment is executed by a controller 12 (see FIG. 5) provided in the in-vehicle device 10. In other words, a program that realizes the information notification method is executed by a computer including the controller 12. More specifically, the information notification method according to the embodiment is a method for determining a notification medium. In the following, a main example is a situation in which the in-vehicle device 10 issues an information notification as a warning to prevent an accident when the vehicle is turning left.

[0013] First, an overview of an information notification method according to an embodiment will be described with reference to Figs. 1 to 4. Fig. 1 is an explanatory diagram of an overview of an information notification method according to an embodiment. Fig. 2 is an explanatory diagram of a normal notification. Fig. 3 is an explanatory diagram of a detailed notification. Fig. 4 is an explanatory diagram of a simple notification.

[0014] The controller 12 of the in-vehicle device 10 analyzes the conditions inside and outside the vehicle based on sensor information from various sensors mounted on the vehicle while the vehicle is traveling. The controller 12 also constantly determines whether or not a scene requires notification of information to call attention (hereinafter referred to as an "attention-calling scene" as appropriate) based on the analysis results.

[0015] As shown in FIG. 1, when the controller 12 detects a scene requiring attention, it calculates the danger level (need for awareness) of the scene based on the sensor information from various sensors (step S1). The need for awareness corresponds to the degree of need for awareness by the user. The need for awareness has a strong correlation with the danger level, and the higher the danger level, the higher the need for awareness. From this perspective, the need for awareness can be replaced with the danger level in processing (the danger level can be used as the need for awareness), and similarly, the need for awareness can be read as "importance."

[0016] For example, as shown in Figure 1, when a vehicle is turning left and bicycle B1 is approaching from behind, the risk (need for awareness) is higher than when bicycle B1 is not present. Also, when bicycle B1 is present, the risk is higher when the distance between the vehicle and bicycle B1 is close compared to when the distance is far. In this way, the need for awareness is estimated based on various information, such as the positional relationship with a dangerous object such as an obstacle, relative speed, predicted time until an accident such as a collision occurs, and the size and weight of the dangerous object.

[0017] Then, when notifying information for preventing entrapment in this caution scene, the controller 12 switches the notification medium according to the user's tolerance for the caution information notification and the degree of danger (based on the result of comparison with a threshold for determining switching of notification medium) (step S2). The threshold for determining switching of notification medium will be described in detail later.

[0018] Notification media are means used to stimulate a person's five senses for notifying information, primarily through visual, auditory, and tactile stimuli, such as display, sound, and vibration. Stimuli for the senses of smell and taste can also be used as notification media. A combination of display, sound, and vibration is treated as a type of notification media. For example, a notification using display and sound is treated as a type of notification media. A user's tolerance for information notifications is an index of the degree to which the notification is perceived as an annoyance, and corresponds to (is correlated with) the user's "familiarity" with the notification. Familiarity is calculated based on the user's driving distance, the number of times each notification has been received, and the frequency of such notifications. Familiarity may also be interpreted as "learning level." By using this proficiency, which corresponds to the user's past experience, as one of the parameters, it is possible to more accurately determine whether the user is likely to be annoyed.

[0019] The information notification method according to the embodiment switches between three notification methods, "normal notification," "detailed notification," and "simple notification," as appropriate. The notification media differ between these notification methods. As shown in FIG. 2, normal notification is a combination of video notification (display notification) and vibration notification.

[0020] A video notification is an information notification that displays (draws) information on a display unit 4 such as an in-vehicle display, and displays, for example, rearward video captured from the vehicle along with warning text (here, "Watch out for left-hand collisions").

[0021] Vibration notification vibrates a member in contact with the occupant's body, and in this case, is information notification that vibrates the vibration unit 5 provided on the seat surface of the driver's seat or the like. The vibration unit 5 is configured with an electric vibration converter, for example, an exciter. Furthermore, multiple vibration units 5 are provided on each of the left and right sides of the driver's seat. When warning the driver when turning left (to the left), the controller 12 vibrates only the vibration unit 5 provided on the left side of the seat. Furthermore, when warning the driver when turning right (to the right), the controller 12 vibrates only the vibration unit 5 provided on the right side of the seat. In normal notifications, these visual and vibration notifications are given simultaneously.

[0022] As shown in FIG. 3, the detailed notification is a notification that includes not only a visual notification and a vibration notification but also a sound notification.

[0023] The voice notification is output from a voice output device, and in this case, the voice is output from the in-vehicle speaker 6 installed in the vehicle cabin and used to notify various information such as driving assistance information and to play music, etc., and is an information notification that outputs a voice warning, for example (in this case, "Be careful of vehicles being hit by vehicles on the left").

[0024] While detailed notification, which uses all of the visual, audio, and vibration notifications, has the advantage of being able to reliably notify the user of information, it also has the disadvantage of being the most annoying to the user. Also, normal notification, which is a combination of visual and vibration notifications, has similar advantages and disadvantages, though weaker than detailed notification. For example, for users with advanced driving proficiency, even normal notification can be annoying. Note that detailed notification is an example of a "medium for notification at the highest risk level."

[0025] As shown in Figure 4, simple notifications are information notifications that only use vibration. Simple notifications have the advantage that they are the least bothersome to users. On the other hand, they have a weak point in terms of reliably notifying users of information. In particular, when there are vibrations other than those for information notifications, such as when driving on rough roads, there is a disadvantage that it is difficult for users to recognize the information (vibrations).

[0026] The level of detail in information notifications is in the order of simple notification < normal notification < detailed notification, but the more detailed the information notification, the more likely the user feels annoyed. In other words, annoyance, or in other words, negative feelings toward information notifications (hereinafter referred to as "negative feelings"), is in a trade-off relationship with the level of detail in information notifications (number of notification media types, amount of notification information, degree of overlap of notification information). In the following, positive feelings will be referred to as "positive feelings" as appropriate, in contrast to negative feelings.

[0027] Returning to the explanation of Figure 1, therefore, after switching the notification medium and notifying the information in step S2, the controller 12 changes the level of detail of the information notification in accordance with the user's emotional state (the degree of annoyance felt) regarding the notification. Specifically, the controller 12 changes the determination threshold for switching the notification medium (step S3). In other words, the controller 12 updates, for example, the notification method determination data table or the notification method determination condition data table (see Figure 6 or Figure 7) of the notification method information 11c.

[0028] 5 is a diagram showing an example of an operation for determining a notification method based on the risk level and the proficiency level. As shown in FIG. 5, the in-vehicle device 10 has notification method information 11c for switching notification media (switching between simple notification, normal notification, and detailed notification).

[0029] The notification media switching pattern can be expressed as a two-dimensional map with the first axis corresponding to the aforementioned risk level and the second axis corresponding to the aforementioned proficiency level. As shown in Figure 5, the risk level is divided into levels of "low," "medium," "high," and "extremely high" based on a threshold, for example. The proficiency level is also divided into levels of "low," "medium," and "high" based on a threshold, for example. Note that the level divisions here are for convenience and are merely an example.

[0030] A corresponding notification method (simple notification, normal notification, or detailed notification) is set for each area specified by a combination of risk and proficiency levels. In other words, the notification method is determined according to the combination of detected risk and proficiency levels.

[0031] In the example shown in FIG. 5, when the risk level is "low," a normal notification is selected if the proficiency level is "low," and a simple notification is selected if the proficiency level is "medium" or "high." When the risk level is "medium," a detailed notification is selected if the proficiency level is "low," a normal notification is selected if the proficiency level is "medium," and a simple notification is selected if the proficiency level is "high." When the risk level is "high," a detailed notification is selected if the proficiency level is "low" or "medium," and a normal notification is selected if the proficiency level is "high." When the risk level is "extremely high," a detailed notification is selected regardless of whether the proficiency level is "low," "medium," or "high."

[0032] In this way, the higher the risk level and the lower the level of familiarity, the more detailed the information notification (the more types of notification media are used), so it is possible to convey necessary information to the user without making the user feel bothered. Also, by providing detailed notifications when the risk level is extremely high, regardless of the level of familiarity, it is possible to reliably notify any user that the situation is extremely dangerous.

[0033] The operation shown in Fig. 5 can be realized, for example, by a method using a notification method determination data table or a method using a notification method determination condition data table. Fig. 6 is an explanatory diagram of a method using a notification method determination data table. Fig. 7 is an explanatory diagram of a method using a notification method determination condition data table.

[0034] As shown in Figure 6, the notification method determination data table is a data table consisting of two parameters, risk level and proficiency level, and notification method data associated with each of these parameters. Conceptually, the notification method determination data table is a data table with risk level and proficiency level as two axes, and notification method data corresponding to cells in the data table determined by the risk level and proficiency level are stored. The notification method determination data table is included in the notification method information 11c. The detected risk level and proficiency level are then searched for in the notification method determination data table, and the method of notifying the information is determined based on the notification method data stored in the corresponding cell.

[0035] Conceptually, in the notification method determination data table, the value of each parameter corresponding to the boundary where the notification method data changes becomes the threshold for determining the notification method. In other words, the threshold is set (changed) by setting (changing) the data (notification method) of a cell in the data table. That is, as shown in FIG. 6, thresholds T1 to T4, T11, and T12 for determining switching of notification media are set by setting the data of each cell of the notification method determination data table. For example, thresholds T1 to T4 are each determination threshold set for the risk level. Also, thresholds T11 and T12 are each determination threshold set for the proficiency level. Note that threshold T4 is a threshold for determining an extremely high risk level and is a fixed value regardless of the proficiency level (it is not subject to threshold change processing).

[0036] When using this notification method determination data table, the controller 12 selects the notification method stored in the cell corresponding to the risk level and proficiency level. Furthermore, when updating the notification method determination data table, the controller 12 changes the value of each cell to be updated (detailed notification, normal notification, simple notification). Changing the value of each cell is equivalent to changing each threshold value T1 to T4, T11, and T12.

[0037] As shown in FIG. 7, the notification method determination condition data table is provided as a table of risk level and proficiency level conditions for determining whether to use detailed notification, normal notification, or simple notification as the notification method. Conceptually, the risk level and proficiency level conditions in the notification method determination condition data table correspond to thresholds for determining the notification method. When using this notification method determination condition data table, the controller 12 searches the table for conditions satisfied by the detected risk level and proficiency level, and selects the notification method corresponding to the searched condition as the notification method to use for notification. When updating the notification method determination condition data table, the controller 12 changes the conditions in each cell of the table (changes the threshold value in the condition). Note that the risk level range D12 is an extremely high risk range, and detailed notification is always assigned regardless of proficiency level (it is not subject to threshold value change processing).

[0038] The controller 12 uses such a notification method determination data table or notification method determination condition data table (hereinafter referred to as "notification method determination data table" as appropriate) to switch the notification medium and notify the user of information. After the notification, the controller 12 estimates the user's emotional state in response to the information notification based on the sensor signals of the various sensors described above. Then, the controller 12 updates the notification method determination data table according to the estimated emotional state.

[0039] For example, if it is estimated that the user has a negative emotion in response to the information notification, the controller 12 updates the data table for determining the notification method so as to reduce the amount of information in the notification. As an example, if the controller 12 issues a normal notification to a user with a "high" proficiency level when the risk level is "high," and the user expresses discomfort, the controller 12 updates the data table so that a simple notification is used even when the risk level is "high" and the proficiency level is "high."

[0040] Then, the next time a caution scene is detected, the controller 12 switches the notification medium based on this updated notification method determination data table. That is, the controller 12 performs information notification while dynamically adjusting the threshold for determining the notification method (notification method determination data table) according to the user's reaction to the information notification while the vehicle is in operation so that the user does not feel annoyed. This makes it possible to realize appropriate information notification that corresponds to various changes in situations, including unexpected situation types (information notification that balances the annoyance felt by the user due to the information notification and the amount of information notification).

[0041] An example of the configuration of the vehicle system 1 including the on-board device 10 to which the information notification method according to the embodiment described above is applied will be described in more detail below. Fig. 8 is a diagram showing an example of the configuration of the vehicle system 1 according to the embodiment.

[0042] 8 shows only the components necessary to explain the features of this embodiment, and general components are omitted. In the explanation using FIG. 8, the explanation of components that have already been explained may be simplified or omitted.

[0043] As shown in FIG. 8, the vehicle system 1 includes an operation unit 2, an in-vehicle sensor unit 3, a display unit 4, a vibration unit 5, an in-vehicle speaker 6, and an in-vehicle device 10.

[0044] The operation unit 2 receives operations from the user on the in-vehicle device 10 and is configured with a push button switch, a touch panel, etc. The operation unit 2 receives, for example, an ON / OFF operation for the information notification function of the in-vehicle device 10. The operation unit 2 also receives a manual operation for switching the notification medium in the information notification function.

[0045] The on-board sensor unit 3 is a group of various sensors mounted on the vehicle. The on-board sensor unit 3 includes an exterior camera 3a, an interior camera 3b, a microphone 3c, a turn signal sensor 3d, and a radar 3e. The exterior camera 3a captures images outside the vehicle. The exterior camera 3a is attached near the windshield, dashboard, rear window, side mirror, or the like, and captures images of the vehicle's surroundings. The interior camera 3b captures images inside the vehicle. The interior camera 3b is attached near the windshield, dashboard, or the like so that at least the driver is included in the captured image. The microphone 3c collects audio from inside the vehicle. The microphone 3c is attached near the windshield, dashboard, or the like so that at least the driver's speech can be collected. The turn signal sensor 3d detects the direction indicated when the user activates the turn signal to turn left or right. The radar 3e detects obstacles around the vehicle.

[0046] In addition to the sensors shown in FIG. 8, the on-vehicle sensor unit 3 may include various sensors such as a G sensor, a GPS (Global Positioning System) sensor, an accelerator sensor, a brake sensor, and a steering sensor.

[0047] The display unit 4 is an in-vehicle display or the like as described above, and presents a video notification, which is a visual information notification, to the user. The vibration unit 5 is a vibration component provided in the driver's seat as described above, and presents a vibration notification to the user by vibration. The vibration unit 5 vibrates with a signal waveform based on a control signal from the controller 12. The in-vehicle speaker 6 is an in-vehicle speaker or the like as described above, and presents a voice notification, which is an auditory information notification, to the user.

[0048] The in-vehicle device 10 includes a storage unit 11 and a controller 12. The storage unit 11 is realized by a storage device such as a read-only memory (ROM), a random access memory (RAM), a flash memory, or a disk device. In the example of Fig. 8, the storage unit 11 stores user information 11a, an image recognition model 11b, notification method information 11c, and an emotion estimation model 11d.

[0049] The user information 11a includes various information for calculating the proficiency of each user who drives a vehicle. Fig. 9 is a diagram showing an example of the user information 11a, and also shows the database format stored in the storage unit 11. As shown in Fig. 9, the user information 11a includes information such as a driving history and a notification history for each user identified by a user ID, and the user's proficiency with information notifications.

[0050] When user information about a new user is stored in the storage unit 11, a user ID is generated based on predetermined ID generation conditions, and a data record for the generated user ID is generated in the storage unit 11 to store various information about the user. Various information obtained from the user's vehicle driving, etc. is stored in the data record of the user ID corresponding to the user.

[0051] User authentication can be performed using a facial image authentication technology in which a facial image of the user driving the vehicle is stored in advance for each user ID (not shown), and the controller 12 compares the facial image of the user captured by the in-vehicle camera 3b of the user's vehicle with the stored facial image to identify the user.

[0052] Furthermore, for example, when a user drives a company car, rental car, shared car, etc., the user is recognized in an authentication process before boarding, and this recognized user information (e.g., user ID) and various information obtained based on driving the vehicle are provided to the vehicle-mounted device 10 via a server, etc., thereby making it possible to collect and use user information.

[0053] In this example, the driving distance is used as the driving history. The number of times of driving (for example, a drive from the departure point to the destination is counted as one time) or the number of days of driving (the number of days of driving) can also be used as the driving history. The driving distance is the cumulative driving distance for each user, and is updated as the vehicle is driven. The notification history is the number of times the user has received information notifications (detailed notification, normal notification, and simple notification) using each information notification method, and is updated as the information notification is received.

[0054] The proficiency level of each user is calculated from time to time based on the mileage and notification history of each user, and stored as user information 11a. For example, the proficiency level is calculated to be higher the longer the mileage. Also, the proficiency level is calculated to be higher the more times each information notification is received.

[0055] Returning to the description of Fig. 8, the image recognition model 11b is a machine learning model used for image recognition processing of the outside-of-vehicle video captured by the outside-vehicle camera 3a. The image recognition model 11b is, for example, a DNN (Deep Neural Network) model.

[0056] The image recognition model 11b functions as an image recognition AI (Artificial Intelligence) by being loaded into the controller 12 and running as a program. The image recognition model 11b is learned and generated in advance so that the image recognition AI can detect objects around the vehicle, traffic lights, the lighting colors of the traffic lights, etc. when a sensor signal (exterior image of the vehicle) is input from the exterior camera 3a. The image recognition model 11b may be a combination of multiple machine learning models.

[0057] As described above, the notification method information 11c is information used for switching the notification medium (switching between simple notification, normal notification, and detailed notification). Note that the notification method information 11c as an initial value is set when the in-vehicle device 10 is designed, and is stored in the storage unit 11 when the in-vehicle device 10 is manufactured.

[0058] The emotion estimation model 11d is a machine learning model used for estimating the user's emotion, and in this case, it is a machine learning model that performs emotion estimation based on the in-car video captured by the in-car camera 3b and the audio signal collected by the microphone 3c. The emotion estimation model 11d is, for example, a DNN model.

[0059] Emotion estimation can be performed, for example, based on Russell's circumplex model. Russell's circumplex model is a model that estimates emotion types based on the relationship between arousal and comfort level. Arousal and comfort level can be calculated (estimated) based on biosignals such as electroencephalograms and heart rate. Therefore, emotion types can be estimated by applying arousal and comfort level calculated from biosignals to Russell's circumplex model. However, since it is difficult to measure biosignals using an in-vehicle device due to factors such as ease of attachment, it is preferable to use an emotion estimation model that estimates emotions based on the facial expressions and voice of the target of emotion estimation, such as the driver.

[0060] Generating such a model (AI (artificial intelligence) model) requires training data that uses facial expressions and voice as input data and the emotions at that time as ground truth data. For example, to collect such training data, biosignals of a subject are measured, and facial image and voice signals of the subject are acquired. Then, arousal and comfort levels are calculated based on the measured biosignals, and the calculated arousal and comfort levels are applied to Russell's circumplex of emotions model to estimate the emotion type. Training data is then generated that uses the acquired facial image and voice signals as input data and estimates the emotion type as ground truth data. By training a pre-training AI model using a large amount of training data generated in this way, an emotion estimation model can be generated that estimates the emotion type by inputting facial image and voice signals of a user whose emotion is to be estimated.

[0061] Fig. 10 is a diagram showing an emotion type table that converts an emotion type into an impression (too much information, an appropriate amount of information, or not enough information) of an information notification method (notification medium). Since an emotion type does not directly become an impression of an information notification method (notification medium), the data in this emotion type table is used to convert an emotion type into an impression of an information notification method (notification medium). Therefore, the emotion type table is a data table that stores data that associates emotion type data to be estimated by an emotion estimation model with notification impression type data, which is an impression of a notification, and is generated by a designer or developer based on experiments or the like.

[0062] For example, if an emotion of displeasure is detected after an information notification, it is presumed that the user felt the information notification was noisy, annoying, etc., so the emotion type table stores displeasure and an impression of information overload in association with each other. Also, if an emotion of anxiety is detected after an information notification, it is presumed that the user felt that there were unclear points about the information notification, etc., so the emotion type table stores anxiety and an impression of information insufficiency in association with each other. Also, if an emotion of relief is detected after an information notification, it is presumed that the user felt satisfied with the information notification, etc., so the emotion type table stores relief and an impression of an appropriate amount of information in association with each other.

[0063] Returning to the explanation of Fig. 8, the controller 12 corresponds to a so-called processor. The controller 12 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), or the like. The controller 12 executes a program according to an embodiment (not shown) stored in the storage unit 11, using RAM as a work area. The controller 12 can also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0064] The controller 12 executes various information processes shown in Fig. 11. Fig. 11 is a functional block diagram of the controller 12. As shown in Fig. 11, the controller 12 first executes a scene determination process based on each sensor signal from the in-vehicle sensor unit 3 (step S11).

[0065] In the scene determination process, the controller 12 determines whether or not the scene is one requiring attention. Next, the controller 12 executes a risk calculation process based on each sensor signal (step S12). In the risk calculation process, the controller 12 calculates the risk of the scene requiring attention based on each sensor signal. For example, the controller 12 predicts the probability of a collision with an obstacle and the impact situation upon collision based on the driving operation state and running state of the vehicle, the position of surrounding obstacles, the relative speed with respect to the vehicle, the size and type of the obstacle, road conditions, etc., and determines whether or not the scene is one requiring attention and calculates the risk according to the collision probability and the impact situation upon collision. Note that a method of determining that the scene is one requiring attention when the risk exceeds a threshold value for the scene requiring attention (a method using the risk) can also be applied.

[0066] Next, the controller 12 executes a notification medium selection process (step S13). In the notification medium selection process, the controller 12 selects a notification method (notification medium) to be used by referring to the notification method determination data table of the notification method information 11c shown in Fig. 6 based on the risk level of the scene and the proficiency level of the target user (driver) based on the user information 11a. Next, the controller 12 executes a notification process using the selected notification medium (step S14).

[0067] Next, the controller 12 acquires each sensor signal after the notification process and executes emotion estimation process based on the acquired sensor signal (step S15). In the emotion estimation process, the controller 12 estimates the user's emotion (type) regarding the information notification (step S14) as described above, and further converts the estimated emotion (type) into impression data regarding the information notification method (notification medium).

[0068] Next, the controller 12 executes a parameter update process (step S16). In the parameter update process, the controller 12 adjusts the relationship between the risk level and proficiency level and the notification method (notification medium) based on the calculated impression data, and updates the notification method determination data table shown in Fig. 6 or the notification method determination condition data table shown in Fig. 7. After the parameter update process, the controller 12 repeats the process from step S11.

[0069] The processing procedure executed by the controller 12 will be described in more detail. Fig. 12 is a flowchart (part 1) showing the processing procedure executed by the in-vehicle device 10 according to the embodiment. Fig. 13 is a flowchart (part 2) showing the processing procedure executed by the in-vehicle device according to the embodiment. This processing starts when the in-vehicle device 10 is started (such as when the vehicle engine is started).

[0070] 12, the controller 12 of the in-vehicle device 10 executes various initial setting processes for the vehicle system 1 (step S101), and then acquires each sensor signal from the in-vehicle sensor unit 3 (step S102). Note that in step S101, the controller 12 reads the user information 11a and the notification method determination data table (notification method determination condition data table) that were saved when the vehicle system 1 was last shut down.

[0071] The controller 12 then analyzes the vehicle situation based on each sensor signal (step S103) and determines whether or not the situation is one requiring attention (step S104). For example, when the blinker is activated and a sensor signal indicating "left" is input from the blinker sensor 3d, the controller 12 determines that the situation is one requiring attention regarding a left turn. If the in-vehicle device 10 has a car navigation function and has route setting information, the controller 12 may determine whether or not the situation is one requiring attention regarding a change in traveling direction, such as a left turn or a right turn, based on the route setting information. In addition, a risk level, which will be described later, may be used to determine whether or not the situation is one requiring attention (if the risk level is very low (less than a threshold for determining a situation requiring attention), it is determined that the situation is not one requiring attention).

[0072] If the scene is not one requiring attention (step S104, No), the controller 12 repeats the process from step S102. On the other hand, if the scene is one requiring attention (step S104, Yes), the controller 12 calculates the degree of danger based on each sensor signal (step S105). The controller 12 also acquires the user's proficiency level based on the user information 11a (step S106). Then, the controller 12 selects an information notification method (notification medium) by referring to the notification method information 11c stored in the storage unit 11 based on the degree of danger and the proficiency level (step S107), and executes notification processing of the attention information using the selected information notification method (step S108).

[0073] After the notification process, the controller 12 acquires each sensor signal that detects emotion estimation information from the in-vehicle sensor unit 3 (step S109). Then, the controller 12 executes a user emotion estimation process based on each sensor signal (step S110). Next, the controller 12 converts the emotion type estimated in the emotion estimation process into impression data for the information notification method (notification medium) and determines the type of the obtained impression data (step S111). If the impression data is "information excess" (step S111, information excess), the controller 12 updates the notification method information 11c so as to favor a notification method (notification medium) with a small amount of notification information (step S112).

[0074] An example of operation when updating the notification method information 11c so that the notification method (notification medium) tends to have a smaller amount of notification information is shown in Fig. 14. Fig. 14 is a diagram (part 1) showing an example of updating the notification method information. For the sake of convenience, it is assumed that the notification method information 11c before updating is the same as that shown in Fig. 5.

[0075] As shown in FIG. 14, when the controller 12 is designed to favor a notification method (notification medium) with a low amount of information, it increases the threshold for the risk level between "detailed notification," "normal notification," and "simple notification," and decreases the threshold for the proficiency level.

[0076] These updates are performed by changing the value of each cell to be updated, for example, in the case of a method using the notification method determination data table shown in Fig. 6. Also, in the case of a method using the notification method determination condition data table shown in Fig. 7, for example, these updates are performed by changing the condition of each cell of the data table (changing the threshold value in the condition).

[0077] Even when updating the notification method information 11c to make it easier for the notification method (notification medium) to have a smaller amount of information, the controller 12 does not update the setting that selects detailed notification regardless of the level of proficiency when the risk level is "extremely high."

[0078] Returning to the explanation of Fig. 13, if the impression data is "appropriate amount of information" (step S111, appropriate amount of information), the controller 12 does not update the notification method information 11c (step S113). Note that since no particular processing is performed in step S113, there is no need to illustrate it, but it is illustrated so that the overall picture of the update processing of the notification method information 11c can be easily understood.

[0079] Furthermore, if the impression data is "insufficient information" (step S111, insufficient information), the controller 12 updates the notification method information 11c so as to favor a notification method (notification medium) with a large amount of notification information (step S114).

[0080] An example of operation when updating notification method information 11c so that the notification method (notification medium) tends to have a large amount of information is shown in Fig. 15. Fig. 15 is a diagram (part 2) showing an example of updating notification method information 11c. For convenience of explanation, it is assumed that notification method information 11c before updating is the same as that shown in Fig. 5, similar to Fig. 14.

[0081] As shown in Figure 15, when the controller 12 is designed to make it easier for a notification method (notification medium) to contain a large amount of information, it decreases the threshold for the risk level between "detailed notification," "normal notification," and "simple notification," and also increases or decreases the threshold for the proficiency level.

[0082] These updates are performed by changing the value of each cell to be updated, for example, in the case of a method using the notification method determination data table shown in Fig. 6. Also, in the case of a method using the notification method determination condition data table shown in Fig. 7, for example, these updates are performed by changing the condition of each cell of the data table (changing the threshold value in the condition).

[0083] Even when updating the notification method information 11c to a notification method (notification medium) that tends to have a large amount of notification information, the controller 12 does not update the setting that selects detailed notification regardless of the level of proficiency when the risk level is "extremely high."

[0084] Returning to the description of Fig. 13, the controller 12 then determines whether or not the operation of the vehicle system 1 is to be terminated (step S115). If the operation of the vehicle system 1 is to be terminated due to the system power being turned off or the like (step S115, Yes), the controller 12 terminates the processing. On the other hand, if the operation of the vehicle system 1 is not to be terminated (step S115, No), the controller 12 repeats the processing from step S102.

[0085] As described above, the in-vehicle device 10 according to the embodiment (corresponding to an example of an "information notification device") includes the controller 12. The controller 12 selects a notification medium to be used for the information notification in accordance with the driver's tolerance for information notifications in the vehicle and the need to recognize the information to be notified, and performs the information notification using the selected notification medium. Therefore, according to the in-vehicle device 10, the notification medium to be used for the information notification is appropriately selected in accordance with the driver's tolerance for information notifications, which affects the driver's feelings about the information notification, and the need to recognize the notification information. As a result, the in-vehicle device 10 can realize information notification that is less bothersome to the user while maintaining the provision of information necessary for vehicle operation.

[0086] In the above-described embodiment, the main example of information notification to prevent collisions is when a vehicle turns left, but this embodiment can also be applied when turning right on a one-way street or a road without a center line.

[0087] Furthermore, the controller 12 notifies the information using the notification medium selected based on the notification method information 11c, but may change the notification medium as appropriate depending on the situation of the vehicle. For example, when notifying the information using simple notification, if the vehicle is traveling on a rough road, the user may not be able to perceive the vibration, so the controller 12 may notify the information using normal notification or detailed notification.

[0088] Furthermore, the controller 12 may change the vibration mode for vibration notification when the level of danger is extremely high or when an obstacle is present. In this case, the controller 12 may vibrate the vibration unit 5 at a frequency of around 140 Hz, which is considered to be unpleasant, for example. Alternatively, the controller 12 may increase the vibration pattern speed of the vibration unit 5. By changing the frequency or the vibration pattern speed, the vibration stimulus given to the user can be changed, making it easier for the user to pay attention to the information notification. Furthermore, vibration at a frequency considered to be unpleasant can increase the user's sense of tension. This makes it easier for the user to reliably recognize the information notification in a dangerous situation.

[0089] In a similar example, for visual notification, the controller 12 may, for example, display an approaching obstacle by surrounding it with a detection frame in a conspicuous color. The controller 12 may also cause the detection frame to blink. In a similar example, for audio notification, the controller 12 may, for example, increase the volume of the audio as the distance to the approaching obstacle decreases.

[0090] Furthermore, if the voice notification when detailed notification is performed overlaps with the voice guidance of the car navigation function, the controller 12 may change to a normal notification that does not perform voice notification.

[0091] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0092] 1 Vehicle Systems 2 Control section 3. In-vehicle sensor section 4 Display section 5 Vibration unit 6. Car speakers 10 Onboard equipment 11 Storage section 11a User Information 11b Image Recognition Model 11c Notification method information 11d Emotion estimation model 12 Controllers

Claims

1. selecting a notification medium to be used for the information notification in accordance with the driver's tolerance for the information notification in the vehicle and the driver's need to recognize the information to be notified; a controller that notifies the information using the selected notification medium; An information notification device comprising:

2. The need for recognition is a degree of risk regarding an event corresponding to the information to be notified, The information notification device according to claim 1 .

3. The tolerance is a degree of familiarity with the information notification. The information notification device according to claim 2 .

4. The controller When the risk level exceeds a maximum threshold, a notification medium for a maximum risk level is selected as the notification medium. The information notification device according to claim 2 .

5. The controller When the risk level exceeds a maximum threshold, a notification medium for a maximum risk level is selected as the notification medium. The information notification device according to claim 3 .

6. The controller Estimating an emotional state of the driver in response to the notified information notification; adjusting a condition for selecting the notification medium based on the estimated emotional state; 5. The information notification device according to claim 1, 2 or 4.

7. The notification medium is Detailed notifications that use video notifications, vibration notifications, and audio notifications; Or, among the video notification, vibration notification, and audio notification, a normal notification is a notification that uses only the video notification and the vibration notification; Alternatively, among the video notification, vibration notification, and audio notification, it is a simple notification that uses only the vibration notification.

5. The information notification device according to claim 1, 2 or 4.

8. The vehicle-mounted device includes a plurality of sensors, a display unit, a speaker, and a vibration unit, The in-vehicle device generating information for a driver of the vehicle based on sensor signals from the plurality of sensors; selecting a notification medium to be used for the information notification from the display unit, the speaker, and the vibration unit according to the driver's tolerance for information notification and the need to recognize the information to be notified; notifying the generated information using the selected notification medium; Vehicle systems.

9. selecting a notification medium to be used for the information notification in accordance with the driver's tolerance for information notification in the vehicle and the driver's need to recognize the information to be notified; The information notification method performed by the controller.

10. Selecting a notification medium to be used for the information notification in accordance with the driver's tolerance for information notification in the vehicle and the driver's need to recognize the information to be notified; performing the information notification using the selected notification medium; A program that causes a computer to execute the following.

Images