Information providing method and information providing system

Abstract

Provided is an information providing method for providing information to a user, which is an occupant of a vehicle, using an output device provided in the vehicle. In the information providing method, a topic to be provided to a user and first information relating to the topic are determined, the first information is output by an output device, a reaction of the user when the first information is output by the output device is detected, and a degree of interest of the user with respect to the topic is estimated on the basis of the reaction of the user. Furthermore, when the degree of interest is higher than a predetermined first threshold value, the control device causes the output device to output second information relating to the topic, the second information being information that is more detailed than the first information.

Description

Technical Field

[0001] The invention relates to an information providing method and an information providing system. Background Art

[0002] JP2018-134902A discloses the following information presentation system: driving-related information is displayed by changing the manner of multiple display bodies, so that the driver (user) of the vehicle can perceive the driving-related information. Summary of the invention

[0003] Furthermore, in general, in a conversation between people, a speaker of a certain topic may speculate on the other party's interest (degree of interest), and when the other party's interest is speculated to be high, the speaker may add more detailed information on the topic before speaking.

[0004] In the information presentation system described in JP2018-134902A, the user's interest is not estimated, so it may not be possible to provide the user with the information they want for topics that the user is interested in. In addition, it is possible that detailed information is unnecessarily provided for objects that the user is not interested in, thereby annoying the user.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide an information providing method and an information providing system capable of providing information desired by a user.

[0006] According to one embodiment of the present invention, there is provided an information providing method for providing information to a user who is a passenger of the vehicle using an output device provided in a vehicle. In the information providing method, a topic to be provided to the user and first information related to the topic are determined, the output device is caused to output the first information, the user's reaction when the output device outputs the first information is detected, and the user's interest in the topic is estimated based on the user's reaction. Furthermore, when the interest is higher than a predetermined first threshold, the output device is caused to output second information related to the topic, the second information being more detailed than the first information. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 This is a block diagram showing a schematic configuration of an information providing system that employs the information providing method according to the first embodiment of the present invention.

[0008] Figure 2 This is a flowchart illustrating the information providing method according to the first embodiment.

[0009] Figure 3 This is a block diagram showing a schematic configuration of an information providing system using the information providing method of the second embodiment.

[0010] Figure 4 This is a flowchart illustrating the information providing method according to the second embodiment. DETAILED DESCRIPTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings and the like.

[0012] (First Embodiment)

[0013] Figure 1 1 is a block diagram showing a schematic structure of an information providing system 100 using an information providing method according to a first embodiment of the present invention. The information providing system 100 is mainly mounted on a vehicle or the like. In addition, in this embodiment, a vehicle equipped with the information providing system 100 is simply referred to as a vehicle. In addition, a user of the information providing system 100 is a passenger of the vehicle, that is, a driver and / or a fellow passenger, but in this embodiment, for simplicity, the driver of the vehicle is set as the user.

[0014] The information providing system 100 includes a user status information acquiring device 10 , a vehicle information acquiring device 20 , an output device 30 , and a controller 40 .

[0015] The user status information acquisition device 10 is a device group for detecting the status of the user (driver), including an in-car camera, an in-car microphone, a passenger sensor, an eye tracker, a body temperature sensor, a heart rate sensor, a sweat sensor, a brain wave sensor, etc. The in-car camera is one or more photographic devices that shoot the interior of the vehicle to obtain a still image or a moving image thereof, and in particular, by continuously or intermittently shooting a range including the driver (user), an image (including a moving image) for detecting the user's actions is obtained. In the following, the image obtained by the in-car camera is set as the in-car image. The in-car microphone is one or more sound collecting devices that obtain voices or sounds generated inside the vehicle. The in-car microphone at least obtains the sound of the user's speech. The passenger sensor detects whether there are passengers in the vehicle. The eye tracker detects the direction of the user's line of sight. The body temperature sensor, the heart rate sensor, the sweat sensor, and the brain wave sensor respectively detect the user's body temperature, heart rate, sweat volume, and brain wave. Information such as images, sounds, fellow passengers, the user's line of sight, the user's body temperature, heart rate, sweat volume, and brain waves acquired by the user status information acquisition device 10 is sent to a controller 40 described later as user status information.

[0016] In addition, the device (sensor) included in the user state information acquisition device 10 is not limited to the above-mentioned devices. For example, an infrared sensor can be used to detect the user's movements, and a camera inside the vehicle can be used to detect the user's sight direction. In addition, the user state information acquisition device 10 only needs to include at least a device that enables the controller 40 described later to detect the user's movements, sounds, and sight direction, and other devices can be added arbitrarily.

[0017] The vehicle information acquisition device 20 is a device group that acquires information related to the current position and surrounding conditions of the vehicle, and includes a current position information acquisition device 21 and a surrounding condition acquisition device 22 .

[0018] The current location information acquisition device 21 is a device for acquiring location information related to the current location of the vehicle, and includes a GPS (Global Positioning System) receiving device, an azimuth sensor, a distance sensor, a steering angle sensor, etc. The GPS receiving device uses artificial satellites to detect the location of the vehicle, and the azimuth sensor, the distance sensor, and the steering angle sensor respectively detect the azimuth (direction) of the vehicle, the distance between the vehicle and the objects around the vehicle, and the steering angle (travel direction) of the vehicle. The current location information acquisition device 21 combines the detection information of the GPS receiving device, the azimuth sensor, the distance sensor, the steering angle sensor, etc. to acquire the location information of the vehicle. In addition, the device (sensor) included in the current location information acquisition device 21 is not limited to the above-mentioned devices, and any device can be used as long as it can acquire the location information related to the current location of the vehicle. For example, the location information can also be acquired using a GNN (Global Navigation Satellite System) sensor, and the location information can also be acquired only from the GPS receiving device. The location information acquired by the current location information acquisition device 21 is sent to the controller 40 described later.

[0019] The surrounding condition acquisition device 22 includes an external camera. The external camera is one or more photographic devices that capture the exterior of the vehicle to obtain a still image or a moving image thereof. The image (including a moving image) used to detect the conditions around the vehicle is acquired by continuously or intermittently capturing the surroundings of the vehicle. The image outside the vehicle acquired by the surrounding condition acquisition device 22 is sent to the controller 40 described later as surrounding information. In addition, the surrounding condition acquisition device 22 may also include a front vehicle distance sensor, a rear vehicle distance sensor, etc. that acquire vehicle distance information between the vehicle in front and the vehicle behind.

[0020] The output device 30 includes a sound output device 31 that outputs sound and a display unit 32 that outputs an image.

[0021] The sound output device 31 is a device that outputs sound data to the user, and includes, for example, a speaker that can output sound. The sound output device 31 outputs sound data received from a controller 40 to be provided to the user as sound.

[0022] The display unit 32 is a device that outputs image data to the user, such as a display capable of outputting images. The display unit 32 outputs image data received from the controller 40 described later to provide to the user, and displays images (including moving images).

[0023] The controller 40 is a unit that controls the information providing system 100 , determines information to be provided to the user, and causes the output device 30 to output the information.

[0024] The controller 40 is constituted by a computer including a central processing unit (CPU), a read-only memory (ROM), a random access memory (RAM), and an input / output interface (I / O interface), and executes a process for providing information to a user by executing a specific program.

[0025] The controller 40 includes a map information acquisition unit 41, a feature information acquisition unit 42, a function database 43, a confusion cause detection unit 44, a confusion estimation unit 45, an information accumulation database 46, an interest estimation unit 47, and an information determination unit 48 as a structure for determining information to be provided to the user. In addition, the controller 40 includes a role database 49 and an information output unit 50 as a structure for causing the output device 30 to output information.

[0026] The map information acquisition unit 41 determines the current position of the vehicle on the map. Position information related to the current position of the vehicle is input to the map information acquisition unit 41 from the current position information acquisition device 21. In addition, the map information acquisition unit 41 acquires map information from a map database (not shown) inside or outside the controller 40, and compares the position information with the map information, thereby determining the current position of the vehicle on the map. The map information acquisition unit 41 sends information on the current position of the vehicle on the map (current position information on the map) to the feature information acquisition unit 42. In addition, the map database is updated with the latest information at predetermined intervals.

[0027] The feature information acquisition unit 42 acquires feature information related to features around the current position of the vehicle. Here, the feature refers to characteristic facilities, historical buildings, etc. The current location information on the map is input to the feature information acquisition unit 42 from the map information acquisition unit 41, and the surrounding information is input to the feature information acquisition unit 42 from the surrounding condition acquisition device 22. In addition, the feature information acquisition unit 42 acquires map information from a map database (not shown) inside or outside the controller 40. The map database has information on features at various locations registered, and the map information includes the feature information. The feature information acquisition unit 42 acquires information related to features visible to the user and information related to other features around the vehicle (hereinafter, both are collectively referred to as feature information) based on the current location information on the map, the surrounding information, and the map information. In addition, the feature information acquisition unit 42 registers the feature information in the information accumulation database 46, and when there is a feature visible to the user, sends a signal indicating this (feature existence signal) to the information determination unit 48 described later. The feature information acquired by the feature information acquisition unit 42 is a candidate for information provided to the user. The above-mentioned method of acquiring feature information is an example, and the method of acquiring feature information is not limited thereto.

[0028] The function database 43 is a database storing data related to the functions of devices in the vehicle. Information related to the functions of each device installed in the vehicle and information such as the frequency of use of each device and the functions installed in each device are registered in the function database 43. The function database 43 is updated at predetermined intervals.

[0029] The confusion cause detection unit 44 detects the function that confuses the user among the functions of the equipment in the vehicle as the confusion cause. The function that confuses the user (confusion cause) here refers to the function that is recommended for the user to use, but the user does not know the existence of the function or does not understand the use method and therefore does not use it. Specifically, it refers to the function of the equipment when a new equipment is added to the vehicle, when the specified equipment in the vehicle is updated, or when the specified equipment in the vehicle has not been used for a long time or a long period (hereinafter referred to as a long time). In addition, the situation where the specified equipment has not been used for a long time includes the situation where the equipment itself is used but part of the function installed in the equipment has not been used for a long time. The confusion cause detection unit 44 queries the function database 43 every specified time (period) whether there is a new equipment addition, equipment update, or equipment that has not been used for a long time. When it is detected that the equipment is added, updated, or the specified equipment has not been used for a long time, the confusion cause detection unit 44 sends information related to the function of the equipment that has been added or updated or the function of the equipment that has not been used for a long time as the confusion cause information to the confusion estimation unit 45, and registers it in the information accumulation database 46.

[0030] The confusion estimation unit 45 estimates whether the user is confused about the operation of a specified device (or function). The user state information is input to the confusion estimation unit 45 from the user state information acquisition device 10, and the confusion cause information is input to the confusion estimation unit 45 from the confusion cause detection unit 44. The confusion estimation unit 45 estimates whether the user is confused about the operation of the specified device based on the detected user's actions contained in the user state information and the confusion cause information. Here, the user's actions refer to the user's actions, the sounds made by the user, the changes in the user's sight direction, etc. For example, it is possible to detect the situation in which the user wants to use a device equipped with a function that is a cause of confusion based on the user's actions captured by the in-vehicle camera, and when it is detected that the user operates the device incorrectly, it is estimated that the user is confused. When it is estimated that the user is confused about the operation of the specified device, the confusion estimation unit 45 registers the information of the device as confusing device information in the information accumulation database 46. In addition, the estimation method of estimating whether the user is confused is not limited to the above method. For example, it is also possible to make an estimation based only on the user's actions detected by the user state information acquisition device 10.

[0031] The information accumulation database 46 stores information that is a candidate for information provided to the user. For example, the information accumulation database 46 stores information related to the operation of various devices in the vehicle. In addition, the information accumulation database 46 stores feature information from the feature information acquisition unit 42, confusion cause information from the confusion cause detection unit 44, and confusion device information from the confusion estimation unit 45. The information accumulation database 46 is updated at predetermined intervals.

[0032] The concern estimation unit 47 estimates the user's concern for the information (first information) provided to the user by the information providing system 100. The user state information is input to the concern estimation unit 47 from the user state information acquisition device 10, and the user's concern is estimated based on the user state information when the information (first information) is provided to the user. The estimated user's concern is sent to the information determination unit 48. In addition, the details of the concern estimation unit 47 are described later.

[0033] The information determination unit 48 determines the topic and information to be provided to the user from the information accumulation database 46. Specifically, the information determination unit 48 inquires whether there is confusing device information in the information accumulation database 46, and if confusing device information is registered, determines the device that the user is confused about operating as a topic, and determines information related to the device as information to be provided to the user (first information). In addition, when the information determination unit 48 receives a signal (feature existence signal) indicating that a feature visible to the user exists from the feature information acquisition unit 42, the feature is determined as a topic, and information related to the feature visible to the user among the feature information registered in the information accumulation database 46 is determined as information to be provided to the user (first information). Furthermore, the information determination unit 48 inquires whether there is confusion cause information in the information accumulation database 46, and if there is a confusion cause, determines a device equipped with a function that is a confusion cause as a topic, and determines information related to the device as information to be provided to the user (first information). Here, the first information to be provided to the user is, for example, an outline of the object determined as a topic. There is no particular limitation on which of the confusing device, feature information, and confusion cause is prioritized when determining the topic and the first information, but in the present embodiment, the priority is set in descending order of confusing device, feature information, and confusion cause. In addition, when neither confusing device information nor confusion cause information is registered in the information accumulation database 46, and the information determination unit 48 does not receive a feature presence signal, a topic provided to the user and information related to the topic (first information) are randomly selected from the information stored in the information accumulation database 46.

[0034] In addition, the user's interest level is input from the interest level estimation unit 47 to the information determination unit 48. When the user is provided with the first information, if the user's interest level in the first information is higher than a predetermined threshold value (first threshold value), the information determination unit 48 determines information (second information, detailed information) related to the subject of the topic that is more detailed than the first information as information to be provided to the user. The information determined to be provided to the user is sent to the information output unit 50. In addition, if the user's interest level in the first information is lower than a second threshold value set to be lower than the first threshold value, the information determination unit 48 changes the topic and the first information.

[0035] The character database 49 stores character-specific information (character-specific information) such as the appearance, speech method, movement, and personality of the character, which is displayed on the display unit 32 of the output device 30 by the information output unit 50 described later. A plurality of characters are registered in the character database 49, and character-specific information related to each character is stored. In addition, the character is not limited to the form of a human, and may be expressed in the form of an animal, a plant, an object, or other symbols.

[0036] The information output unit 50 determines how to output the information (first information and second information) provided to the user determined by the information determination unit 48. In the present embodiment, two characters are displayed on the display unit 32, and the first information and the second information are output as a dialogue between the characters. In addition, in the present embodiment, the first information and the second information are information that can be output by sound.

[0037] The information output unit 50 refers to the role-specific information stored in the role database 49 to determine which role is to be spoken according to what script to provide information to the user. Specifically, the information output unit 50 determines the role to be displayed on the display unit 32 from the role database 49, and obtains the inherent information of the role from the role database 49. In addition, the information output unit 50 divides the first information into a first part to be spoken by one role (the first role) and a second part to be spoken by the other role (the second role). Then, the display unit 32 is made to output (display) the image (including moving images) data of the determined role, and the sound data of the first part and the sound data of the second part are output from the sound output device 31 in coordination with the action of the role displayed on the display unit 32. Thus, the first information is provided to the user. In this way, by dividing the first information into two parts and making two roles speak, the first information can be provided to the user in the form of a dialogue between the roles. Therefore, the user's responsibility for responding to the speech issued from the information providing system 100 is reduced, and the user's comfort is improved.

[0038] When the first information is output via the output device 30 , the information output unit 50 transmits a signal indicating that the first information has been output to the interest estimation unit 47 .

[0039] In addition, the information output unit 50 determines which character is to speak the second information (detailed information) provided to the user determined by the information determination unit 48. Then, the display unit 32 is caused to display the image (including a moving image) of the determined character, and the sound data of the second information is output from the sound output device 31 in coordination with the action of the character displayed on the display unit 32. In this way, the second information is provided to the user.

[0040] Furthermore, the information output unit 50 outputs voice data set in a speech method (for example, wording and sentence formation, etc.) that matches the personality of the character from the voice output device 31 based on the character-specific information of the character being used.

[0041] In addition, the first information is output as a conversation between the first character and the second character, and the second information is output as a speech of the first character or the second character. However, the output method of the first information and the second information is not limited to this.

[0042] The method for determining the character to be displayed on the display unit 32 is not particularly limited, and for example, a character having an appearance and a personality that matches the topic provided to the user can be selected.

[0043] Next, the details of the interest degree estimation unit 47 will be described.

[0044] When the first information provided to the user is output via the output device 30, a signal indicating that the first information is output is input from the information output unit 50 to the concern estimation unit 47. When the signal indicating that the first information is output is input, the concern estimation unit 47 estimates the user's concern for the first information (hereinafter also referred to as the user's concern) based on the user's reaction when the first information is output. Here, the user's reaction is the state of the user detected by the user state information acquisition device 10 when the output device 30 outputs the first information, and the concern estimation unit 47 estimates the user's concern based on the user state information from the user state information acquisition device 10.

[0045] More specifically, the user's reaction includes the user's action, and the interest estimation unit 47 estimates the user's interest in the first information based on the user's action included in the user state information. Here, the user's action includes the user's movement detected from the in-car image, the user's speech sound acquired by the in-car microphone, the movement of the user's sight direction detected by the eye tracker, etc. For example, when the output device 30 outputs the first information, the user state information acquisition device 10 detects that the user agrees, nods, makes a sound such as "ah", continues to look at the object that has become a topic for a predetermined time, and visually confirms at least one of the first character and the second character who said the first information, etc., the interest estimation unit 47 estimates that the user's interest is high. Specifically, for example, the eye tracker acquires the user's image information, detects the direction of the user's face sight line included in the image information, and determines whether the detected sight line direction is consistent with (or approximately consistent with) a specific direction, wherein the specific direction is a direction connecting the user's eyes to the output device 30 that displays the first character and / or the second character. Furthermore, when it is determined that the direction of the detected line of sight is consistent with (or approximately consistent with) a specific direction, it is determined that the user has viewed the first character and / or the second character, thereby estimating the user's interest. On the other hand, when it is determined that the direction of the detected line of sight is not consistent with (or approximately consistent with) a specific direction, it is determined that the user has not viewed the first character and / or the second character. In addition, known image recognition technology can be used for face detection processing and line of sight detection processing. In this way, when the user performs an action such as being interested in the first information, the user's interest is estimated to be higher than a predetermined threshold value (first threshold value), and more detailed information (second information) related to the object of the topic is provided to the user.

[0046] The above is the main structure of the information providing system 100. As described above, in the information providing system 100, information to be provided to the user is determined from the information accumulation database 46, and the information is provided to the user via the output device 30.

[0047] In addition, if the information provided to the user is determined without estimating the user's interest, the information the user wants may not be provided for topics that the user is most interested in. In addition, unnecessary detailed information may be provided for objects that the user is less interested in, which may annoy the user.

[0048] On the other hand, it is difficult to estimate the user's interest before providing information, and there is a high risk of misestimating the interest.

[0049] In contrast, in the information providing system 100 that adopts the information providing method of the present embodiment, the topic to be provided to the user and the first information related to the topic (for example, an overview related to the object of the topic) are determined, and the user's interest in the first information is estimated based on the user's reaction when the first information is output. Furthermore, when the interest is higher than the first threshold, the second information related to the topic is output, and the second information is more detailed than the first information. In this way, more detailed information, that is, the second information, is output only when the user's interest is high, so that the user can obtain detailed information only on topics that are of interest to the user. That is, the information that the user wants can be provided. In addition, it is possible to prevent the user from being annoyed by providing detailed information on topics that the user does not care about. In addition, since the user's interest is estimated based on the reaction to the first information, the accuracy of the estimation of the interest is improved compared to the case where the interest is estimated before the information is provided.

[0050] Figure 2 1 is a flowchart for explaining the information providing method of the first embodiment. The following controls are all repeatedly executed by the controller 40 at a predetermined time interval. In addition, the following controls can be started when the information providing system 100 is started, or can be started when some information has been output to the user. In addition, the controller 40 always obtains the user status information from the user status information acquisition device 10 at a predetermined time interval, and always obtains the position information and surrounding information from the vehicle information acquisition device 20 at a predetermined time interval.

[0051] In step S101, the controller 40 estimates whether the user is confused about the operation of a specified device or function. As described above, the estimation of whether the user is confused is estimated based on the detected user's actions contained in the user state information. In the case where it is estimated that the user is confused about the operation of the specified device based on the user's actions, the controller 40 performs the processing of step S102. On the other hand, in the case where it is not estimated that the user is confused, the controller 40 performs the processing of step S111.

[0052] In step S102, the controller 40 determines the device (confused device) whose operation the user is confused about as a topic, and determines a summary related to the operation of the device (confused device) as the first information. In this way, when the user's confusion is estimated based on the user's behavior detected before the first information is determined, the summary related to the operation of the confusing device is determined as the first information.

[0053] In step S111, the controller 40 determines whether there are any features visible to the user. The controller 40 determines whether there are any features visible to the user based on the current location information, surrounding information, and map information on the map. If there are any features visible to the user, the controller 40 performs the processing of step S112. On the other hand, if there are no features visible to the user, the controller 40 performs the processing of step S121.

[0054] In step S112, the controller 40 determines a feature that the user can see as a topic, searches the information accumulation database 46 for feature information of the feature, and determines a summary related to the feature as the first information.

[0055] In step S121, the controller 40 determines whether there is a cause of confusion. As described above, when a new device is added to the vehicle, when a prescribed device in the vehicle is updated, or when it is detected that a prescribed device in the vehicle has not been used for a long time, it is determined that there is a cause of confusion. If there is a cause of confusion, the controller 40 performs the processing of step S122. On the other hand, if there is no cause of confusion, the controller 40 performs the processing of step S131.

[0056] In step S122 , the controller 40 determines the function of a device that has been added or updated or the function of a device that has not been used for a long time as a topic, and determines a summary related to the function as the first information.

[0057] In step S131 , the controller 40 randomly selects information stored in the information accumulation database 46 , determines an object related to the selected information as a topic, and determines a summary of the object as the first information.

[0058] When the first information is determined in step S102 , S112 , S122 , or S131 , the controller 40 causes the output device 30 to output the first information in step S103 .

[0059] In step S104, the controller 40 estimates the user's interest in the topic. As described above, the user's interest is estimated based on the user's reaction, particularly the user's action, when the first information is output. For example, when the user is detected to have visually confirmed an action of at least one of the first character and the second character, the interest is estimated to be high.

[0060] In step S105, the controller 40 determines whether the user's interest in the first information estimated in step S104 is higher than a predetermined first threshold. The first threshold can be set to, for example, the following value: when the output device 30 outputs the first information, if it is detected that the user agrees, nods, makes a sound such as "ah", continues to watch the object of the topic for a predetermined time, visually confirms at least one of the first character and the second character, etc., the user's interest exceeds the first threshold. If the user's interest in the first information is higher than the first threshold, the controller 40 performs the processing of step S106. On the other hand, if the user's interest in the first information is less than the first threshold, the controller 40 performs the processing of step S115.

[0061] In step S106 , the controller 40 causes the output device 30 to output second information which is information related to the topic in more detail than the first information.

[0062] In this way, the controller 40 estimates the user's interest based on the user's action when the first information is output (step S104), and outputs the second information when the interest is higher than the first threshold (steps S105, S106). Therefore, the user can obtain detailed information related to the topic by performing a simple action such as visually confirming the character. In other words, in order to obtain detailed information, there is no need to perform special instructions, thereby reducing the complexity of the operation and improving the user's comfort.

[0063] In step S105, when the user's interest in the first information is below the first threshold, in step S115, the controller 40 determines whether the user's interest in the first information estimated in step S104 is above a predetermined second threshold. The second threshold can be set, for example, such that the user's interest is below the second threshold when the user's state is almost unchanged when the first information is output, or when the user issues a negative keyword (for example, "Change to another topic"), etc. In the case where the user's interest in the first information is above the second threshold, the controller 40 continues to output the first information in step S116. On the other hand, in the case where the user's interest in the first information is below the second threshold, the controller 40 performs the processing of step S125.

[0064] In step S125, the controller 40 changes the topic and the first information, and causes the output device 30 to output the changed first information. As with step S131, the topic and the first information can be changed by randomly selecting from the information stored in the information accumulation database 46. When the changed first information is output in step S125, the controller 40 returns to the process of step S104 to estimate the user's interest in the changed topic.

[0065] In this way, when the user's interest in the first information is lower than the second threshold, the information provided to the user (first information) is changed (steps S115 and S125). This can prevent the user from being annoyed by the continuous provision of information that the user does not care about.

[0066] In addition, in this embodiment, for convenience, the steps of determining the topic and the first information are set separately, but in fact, by providing the user with the first information related to the specified object, the object automatically becomes the topic. Therefore, it can be interpreted that the controller 40 determines the topic by determining the first information.

[0067] According to the information providing method of the first embodiment described above, the following effects can be obtained.

[0068] In the information providing method of this embodiment, the user's interest in the first information is estimated based on the user's reaction when the first information related to the topic provided to the user is output, and when the interest is higher than a first threshold, more detailed information related to the topic, that is, the second information, is output. As a result, the user can obtain detailed information only on the topic that the user is interested in. In other words, the information the user wants can be provided.

[0069] In addition, since the user's interest level is estimated based on the reaction to the first information, the estimation accuracy of the interest level is improved compared to the case where the interest level is estimated before the information is provided.

[0070] In the information providing method of this embodiment, the user's interest in the first information is estimated based on the user's action when the first information is output, and when the interest is higher than the first threshold, more detailed information related to the topic, that is, the second information, is output. As a result, the user can obtain detailed information related to the topic by performing simple actions without making special instructions. As a result, the user's comfort level is improved.

[0071] In the information providing method of this embodiment, the first information is divided into a first part and a second part, the first part is output as the speech of the first character displayed on the display unit 32, and the second part is output as the speech of the second character displayed on the display unit 32. In this way, by dividing the first information into two parts and making two characters speak, the first information can be provided to the user in the form of a dialogue between the characters. Therefore, the user's responsibility for responding to the speech issued from the information providing system 100 is reduced, and the user's comfort is improved.

[0072] In the information providing method of this embodiment, when the user is detected to have visually confirmed at least one of the first character and the second character when the first information is output, the user's interest in the first information is estimated to be high. Therefore, by performing a simple action such as visually confirming the character, the interest can be increased above the first threshold and detailed information (second information) related to the topic can be obtained. That is, in order to obtain detailed information, no special instructions are required, thereby reducing the complexity of the operation and improving the user's comfort.

[0073] In the information providing method of this embodiment, when it is detected that a new device is added to the vehicle, a prescribed device in the vehicle is updated, or a prescribed device in the vehicle has not been used for a long time, information related to the operation of the device detected to be added, updated, or not used for a long time is determined as the first information to be provided to the user. In this way, the user can be prompted to use a function that is recommended to the user but the user does not know the existence of the function or does not understand how to use it and therefore does not use it.

[0074] In the information providing method of this embodiment, when it is estimated from the user's behavior that the user is confused about the operation of a specified device, information related to the operation of the specified device is determined as the first information. Thus, when the user is confused, information related to the operation of the device that the user wants can be provided.

[0075] In the information providing method of this embodiment, when there is a feature visible to the user, the feature is determined as a topic, and feature information related to the feature is determined as the first information provided to the user. In this way, by using an object within the user's field of vision as a topic, the user can guess where the provided information comes from, thereby giving the user a sense of security.

[0076] In the information providing method of this embodiment, when the user's interest in the first information is lower than a predetermined second threshold value that is smaller than the first threshold value, the first information is changed. This can prevent the user from being annoyed by the continuous provision of information that the user does not care about.

[0077] In addition, in the present embodiment, the information is provided to the user by outputting sound and images, but the present invention is not limited to this. For example, the information may be provided only by sound.

[0078] In addition, in the present embodiment, information is provided to the user through the speech of two characters displayed in the display unit 32, but it is not necessarily limited to this. For example, the displayed character can be one person, or three or more. In addition, for example, the appearance of the output device 30 can also be set to the appearance of the character. In this case, the character is not displayed in the display unit 32, but the output device 30 itself becomes the character. Therefore, for example, the first information can also be provided through the dialogue between the characterized output device 30 and the character displayed in the display unit 32. Here, the characterized output device 30, for example, includes an agent device as a small robot set on the dashboard of the vehicle. In addition, the agent device is, for example, a robot that imitates an animal (such as a rabbit), a robot that imitates a virtual creature (such as the face of a character in an animation), and a robot that imitates other objects (such as a TV-type device, a radio-type device), but it is not limited to this. Other devices with user interfaces such as display units (e.g., smartphones, tablet terminals, car navigation devices, IVI (In-Vehicle Infotainment)) can also be used as intelligent devices. In this case, the display unit can display the robot, and the robot's action mode can be changed by changing the display mode of the robot.

[0079] In addition, in this embodiment, the first information is changed when the user's interest in the first information is lower than the second threshold, but the first information may be changed when the user's interest is lower than the second threshold for a predetermined time. Thus, the first information is changed after it is more clearly known that the user is not interested in the first information, thereby preventing the first information from being changed when the user is interested in the currently provided first information.

[0080] In addition, in the present embodiment, when there is a feature visible to the user, the feature is determined as a topic and feature information related to the feature is determined as the first information, but the present invention is not necessarily limited to this. Alternatively, when there is a feature around the vehicle, even if the user cannot see the feature, the feature around the vehicle is determined as a topic and feature information related to the feature determined as the topic is determined as the first information.

[0081] In addition, in this embodiment, when the confusing device, the ground object visible to the user, and the confusion cause do not exist, and when the user's interest in the first information is lower than the second threshold, the first information provided to the user is randomly selected, but it is not necessarily limited to this. For example, instead of randomly selecting the first information, it is also possible to set the importance of each information and preferentially select the information with high importance as the first information.

[0082] In addition, in the present embodiment, the second information is output as the speech of the first character or the second character, but it is not necessarily limited to this. For example, when the second information is information that can be output through an image, the display unit 32 can also display the second information as an image. In this way, the second information can be provided to the user through the image displayed by the display unit 32 independently of the sound output device 31. In this case, the image includes not only photos and pictures, but also text, etc.

[0083] In addition, in the present embodiment, when there is a cause of confusion, information related to a device equipped with a function that is a cause of confusion is determined as the first information, and when there is a confusing device, information related to the device is determined as the first information, but the present invention is not necessarily limited to this. For example, when there is no cause of confusion or confusing device, information related to the operation of any device in the vehicle that the user can use may also be determined as the first information provided to the user. As a result, the user's understanding of the functions of the devices in the vehicle is improved, and the various devices in the vehicle can be used more effectively.

[0084] (Second Embodiment)

[0085] Reference Figure 3 and Figure 4 The information providing method of the second embodiment will be described. Note that the same reference numerals are used for the same elements as those of the first embodiment, and description thereof will be omitted.

[0086] Figure 3 1 is a block diagram showing a schematic configuration of an information providing system 200 that adopts the information providing method of the second embodiment. The second embodiment is different from the first embodiment in the method of determining the first information and the form of outputting the first information.

[0087] like Figure 3 As shown, the information providing system 200 includes a user status information acquiring device 10 , a vehicle information acquiring device 20 , an output device 30 and a controller 40 .

[0088] The user state information acquisition device 10 and the output device 30 are the same as those in the first embodiment, and thus their description is omitted.

[0089] The vehicle information acquisition device 20 includes a current location information acquisition device 21 including a GPS receiving device for acquiring vehicle location information and a surrounding situation acquisition device 22 including an external camera for acquiring surrounding information. In this embodiment, the vehicle information acquisition device 20 further includes a load detection device 23.

[0090] The load detection device 23 includes a steering angle sensor of a steering wheel, an accelerator opening sensor, etc. The steering angle and accelerator opening detected by the load detection device 23 are sent to the controller 40 and used for estimation of driving load described later.

[0091] In addition to the configuration described in the first embodiment, the controller 40 further includes a user behavior information acquisition unit 61, a preference information database 62, a visual confirmation area extraction unit 63, an attention estimation unit 64, an attention object determination unit 65, a confidence estimation unit 66, and an output feature generation unit 67. Figure 3 In the figure, descriptions about the feature information acquisition unit 42, the function database 43, the confusion cause detection unit 44, the confusion estimation unit 45, and the interest degree estimation unit 47 are omitted.

[0092] The user action information acquisition unit 61 includes a driving load detection unit 61A, a voice feature detection unit 61B, and a sight line detection unit 61C to acquire the user's action information. The user action information acquisition unit 61 receives the user state information from the user state information acquisition device 10, the position information, the surrounding information, the steering angle, the accelerator opening information from the vehicle information acquisition device 20, and the current location information from the map information acquisition unit 41.

[0093] The driving load detection unit 61A estimates the driving load of the driver as the user based on the user state information, position information, surrounding information, steering angle, throttle opening information, and current location information. The driving load is estimated, for example, according to the steering entropy method. In addition, it is preferred that the driving load detection unit 61A obtains the driver's skill data based on the current location information, surrounding information, time series data of the steering angle, time series data of the throttle opening, etc., and uses it to estimate the driving load. In this way, the estimation accuracy of the driving load can be improved.

[0094] The voice feature detection unit 61B detects the voice features of the user's speech based on the voice of the user's speech acquired by the in-vehicle microphone. Here, the voice features include the content of the speech (for example, whether it contains specific keywords, etc.), the volume of the voice, the pitch of the voice, the accent, etc. The voice feature detection unit 61B obtains the voice feature amount obtained by digitizing these voice features obtained by the user's speech. In addition, for example, it is possible to determine whether the sound is obtained by the user's speech by pre-storing the user's voice features.

[0095] The sight line detection unit 61C detects the time when the user's sight line is directed in the sight line direction based on the in-vehicle image acquired by the in-vehicle camera (user state information acquisition device 10) and the user's sight line direction detected by the eye tracker (user state information acquisition device 10). The sight line detection unit 61C acquires the user's sight line direction and the time when the sight line is directed in the direction as sight line information. In addition, the method for acquiring sight line information is not limited to the above method. For example, the sight line information can also be acquired only based on the in-vehicle image.

[0096] The information of the driving load estimated by the driving load detection unit 61A, the voice feature value acquired by the voice feature detection unit 61B, and the line of sight information acquired by the line of sight detection unit 61C are sent as user action information to the attention estimation unit 64 and the confidence estimation unit 66 described later. In addition, the line of sight information acquired by the line of sight detection unit 61C is also sent to the visual confirmation area extraction unit 63.

[0097] The preference information database 62 is a database that accumulates the user's preference information. The user's preference information is obtained based on the user's past actions, etc. For example, the more frequently a specific type of restaurant or facility is visited, the more the dish or facility is registered as being preferred by the user. However, the method of obtaining the preference information is not limited to the above method.

[0098] The visual confirmation area extraction unit 63 determines the visual confirmation area that the user can visually confirm and the objects (objects) contained in the visual confirmation area. The visual confirmation area extraction unit 63 is input with the line of sight information acquired in the line of sight detection unit 61C, the current location information on the map from the map information acquisition unit 41, and the surrounding information (the image outside the vehicle captured by the vehicle exterior camera) from the surrounding condition acquisition device 22. The visual confirmation area extraction unit 63 compares the user's line of sight direction contained in the line of sight information with the current location information and surrounding information on the map to determine the visual confirmation area that the user can visually confirm, and determines the objects contained in the visual confirmation area based on the current location information and surrounding information on the map. However, when the time when the user directs the line of sight in the direction of the user's line of sight is shorter than the prescribed time (the fourth threshold value), it is assumed that the visual confirmation cannot be made, and the visual confirmation area extraction unit 63 does not determine the object. The prescribed time here can be set, for example, to a time at which the user cannot recognize what the object in front of the line of sight direction is. In addition, the determination of the visual confirmation area is not limited to the above-mentioned method, but is at least determined based on the user's line of sight direction. The information of the identified visual confirmation area and the object is sent to the attention degree estimation unit 64 and the confidence degree estimation unit 66 to be described later.

[0099] The attention estimation unit 64 estimates the user's attention to each object located in the visual confirmation area. The attention estimation unit 64 receives the user action information from the user action information acquisition unit 61, and the information of the visual confirmation area and the objects contained in the visual confirmation area from the visual confirmation area extraction unit 63. In addition, the attention estimation unit 64 queries the preference information database 62 to obtain the user's preference information related to the objects in the visual confirmation area. The attention estimation unit 64 estimates the user's attention to each object located in the visual confirmation area based on the user action information and the preference information (hereinafter referred to as user information). Specifically, the closer the object is to the center of the visual confirmation area among the objects in the visual confirmation area, the higher the user's attention is estimated (added points), and the more the user likes the object, the higher the user's attention is estimated (added points). If the preference information about the object in the visual confirmation area is not registered in the preference information database 62, the attention is not added. In addition, when the time when the user directs his / her gaze toward the central object closest to the center of the visual confirmation area exceeds a predetermined time (fifth threshold), when the user's voice is detected while the user directs his / her gaze toward the central object and the sound feature amount of the voice exceeds a predetermined value (sixth threshold), the user's attention to the central object is estimated (added points) to be higher. In this way, the user's attention to the object is estimated by taking into account preference information, the time when the gaze is directed toward the central object, sound features, etc., rather than simply estimating the highest degree of attention to the central object. Therefore, the estimation accuracy of the attention is further improved.

[0100] In addition, when the user's driving load is high, the user's attention to the object is estimated to be low. Specifically, in the present embodiment, when the user's driving load is above a prescribed value (the seventh threshold), the user's attention to all objects located in the visual confirmation area is estimated to be 0. That is, when the user's driving load is high, it is estimated that the user is in a state where he wants to concentrate on driving, so the attention is not actually estimated, and the driving load is waited to become less than the prescribed value before the attention is estimated. Therefore, for example, when the user wants to concentrate on driving, it is possible to prevent the attention of the objects in the visual confirmation area from being mistakenly estimated to be high. In this way, the user's driving load is also considered in the estimation of the attention, and when the driving load is high, the user's attention to the object is estimated to be low, so the estimation accuracy of the attention is further improved.

[0101] Information on the degree of attention to each object (attention degree information) estimated by the attention degree estimation unit 64 is sent to the attention object identification unit 65 and the certainty estimation unit 66 .

[0102] The object of interest determination unit 65 determines which object in the visual confirmation area becomes the object of interest. Here, the object of interest is an object that becomes a topic provided to the user. The attention information is input from the attention estimation unit 64 to the object of interest determination unit 65. The object of interest determination unit 65 determines the object with the highest attention among the objects in the visual confirmation area based on the attention information, and determines the object as the object of interest when the user's attention to the object is higher than a predetermined threshold (third threshold). By setting a threshold (third threshold) for determining the object of interest, it is possible to prevent, for example, the case where an object that the user does not pay attention to but has only entered the field of vision is determined as the object of interest. In the object of interest determination unit 65, the central object closest to the center of the visual confirmation area is usually determined as the object of interest. In particular, in the present embodiment, when a specific keyword is spoken while the user is looking toward the central object, the central object is set as the object of interest (the attention estimation unit 64 is set in advance so that the central object has the highest attention when the specific keyword is spoken). For example, when the user says a specific question word (keyword) such as "What is that?" while directing his or her gaze toward a building that serves as the central object, the sound feature quantity increases, and the central object (the building) is determined as the object of interest. In addition, depending on the situation, objects other than the central object can also become the object of interest. For example, in a case where there are a cat and a park as symmetrical objects within the visual confirmation area, the park is more likely to be located in the center of the visual confirmation area because the range of the park is wider than that of the cat. However, if the user's liking for cats is registered in the preference information database 62, the degree of interest in cats is estimated to be high, and cats can be determined as the object of interest. The information on the object of interest determined in the object of interest determination unit 65 (object of interest information) is sent to the confidence estimation unit 66 and the information determination unit 48.

[0103] The confidence estimation unit 66 estimates the degree of accuracy of the determination of the object of interest, that is, the confidence. The confidence estimation unit 66 receives inputs of the user action information from the user action information acquisition unit 61, the visual confirmation area and the information of the object contained in the visual confirmation area from the visual confirmation area extraction unit 63, the attention information from the attention estimation unit 64, and the attention object information from the attention object determination unit 65. In addition, the confidence estimation unit 66 queries the preference information database 62 to obtain the user's preference information related to the object in the visual confirmation area. The confidence is estimated based on the information of the object contained in the visual confirmation area and the user information (user action information and preference information). Specifically, the more objects are contained in the visual confirmation area, the lower the confidence is estimated, the more objects that the user likes are included in the visual confirmation area, the lower the confidence is estimated, and the larger the sound feature amount when the user speaks, the higher the confidence is estimated. The more objects are contained in the visual confirmation area, the more difficult it is to judge which object the user is actually paying attention to, and the lower the accuracy of the determination of the object of interest is. On the contrary, when the number of objects included in the visual confirmation area is small, the possibility that the user is actually paying attention to the determined object of interest is high, and the certainty is estimated to be high. In addition, when the visual confirmation area contains many objects that the user likes highly, it is difficult to determine which object the user is actually paying attention to, so the accuracy of determining the object of interest becomes low. On the contrary, when there are few objects that the user likes highly among the objects included in the visual confirmation area, the accuracy of determining the object of interest becomes high, and the certainty is estimated to be high. For example, in the case where there is only one object that the user likes highly in the visual confirmation area, the object is determined as the object of interest, and the possibility that the user is actually paying attention to the object is high. In addition, when the sound feature amount is large when the user speaks, the possibility that the user is actually paying attention to the determined object of interest is high, and the certainty is estimated to be high. For example, as listed as an example in the description of the object of interest determination unit 65, when there is a building in front of the user's line of sight and the user says a specific question word (keyword) such as "What is that?", the sound feature amount becomes large. In this case, there is a high possibility that the user is actually paying attention to the building (the object of interest, the central object), and therefore the certainty is also estimated to be high.

[0104] As described above, the degree of accuracy of identification of the object of interest, namely, the degree of certainty is quantitatively estimated in the certainty estimation unit 66. Information on the degree of certainty estimated in the certainty estimation unit 66 (certainty information) is sent to the information determination unit 48, the information output unit 50, and the output feature generation unit 67.

[0105] When the object of interest information and the confidence information are input to the information determination unit 48, the information determination unit 48 determines the topic and information to be provided to the user in the same manner as in the first embodiment. In this embodiment, for simplicity, it is assumed that there are no confusing devices, feature presence signals, and confusion causes. The information determination unit 48 determines the object of interest as the topic and determines the information (summary) related to the object of interest as the first information. If the information related to the object of interest (including not only the summary (first information) but also detailed information (second information)) is registered in the information accumulation database 46, the information is obtained from the information accumulation database 46, and if it is not registered, the information is obtained from an external database (not shown). In addition, when there are confusing devices, feature presence signals, confusion causes, etc., the information determination unit 48 selects any one of the confusing devices, features, devices equipped with functions that serve as confusion causes, and objects of interest as the topic. In this case, the priority is not particularly limited, but for example, the priority is set in descending order of confusing devices, feature information, confusion causes, and objects of interest. The topic to be provided to the user and the first information determined by the information determination unit 48 are sent to the information output unit 50 .

[0106] The confidence information is input from the confidence estimation unit 66 to the output feature generation unit 67. The output feature generation unit 67 is configured to be able to mutually receive and send information with the information output unit 50, and the output feature generation unit 67 adds features to the output mode of the first information determined by the information determination unit 48 according to the confidence level. Specifically, when the first information is spoken by a character (including the case where the output device 30 itself is a character), the appearance, action, voice, etc. of the character are given features corresponding to the confidence level. Here, the features of the appearance include the expression, size, color, shape, etc. of the character, and the features of the action include rotation, vibration, speed of movement, slowness and urgency of movement, direction of sight of the character, etc. In addition, the features of the voice include intonation, volume, expression of the end of the sentence, etc. Moreover, when the confidence level is low, these features are used to indicate a lack of confidence. For example, when the confidence level is low, features can be added to the voice data and image data in such a way as to set the expression of the character to an expression without confidence, to reduce the size of the character, and to set the voice of the character to a weak voice. To give a specific example, for example, suppose that a character says the first information about a cat. When the degree of certainty is low, features are added to the sound data and image data in the following manner: the character's eyebrows are lowered, and a speech is made in a weak voice, such as "So, there appears a motion picture of a cat that ○○ (user's name) likes. ... Eh? I think you are looking at the cat, aren't you?" In addition, when the first information is provided by displaying text on the display unit 32, the text is given features according to the degree of certainty. Here, the features of the text include the thickness, size, font, color, expression at the end of the sentence, etc. of the text. For example, when the degree of certainty is low, features are added to the image (text) data in a manner that expresses a lack of confidence, such as by making the text thinner or smaller, or by adding a question word at the end of the sentence.

[0107] The information output unit 50 outputs the audio data and the image data of the first information to which the features are added by the output feature generation unit 67 via the output device 30. Thus, information is provided to the user.

[0108] As described above, in the information providing method of the second embodiment, the first information is output in an output form corresponding to the level of certainty. Thus, even when information that is not of interest to the user is provided, it is possible to alleviate the deterioration of the user's impression.

[0109] In addition, since the confidence is estimated quantitatively, it is easy to determine how information should be provided to the user. That is, since the confidence is quantified, it is easy to determine, for example, what degree of affirmative expression should be used, what degree of volume, what degree of text should be used, etc.

[0110] Furthermore, a correction value acquisition unit may be provided, which acquires a necessary correction value for the estimation of the degree of attention and the degree of certainty when it is estimated from the user's reaction when the first information is output that the user's degree of attention is low (for example, when the user's degree of attention is lower than the second threshold value in the first embodiment, etc.). Thus, the estimation of the degree of attention and the degree of certainty is more suitable for the user.

[0111] Figure 4 It is a flowchart illustrating the information providing method of the second embodiment. The following controls are all repeatedly executed by the controller 40 at prescribed intervals. In addition, the following controls can be started when the information providing system 100 is started, or can be started when some information has been output to the user. In addition, the controller 40 always obtains the user status information from the user status information acquisition device 10 at prescribed intervals, and always obtains the position information, surrounding information, and the information of the steering angle and the throttle opening from the vehicle information acquisition device 20 at prescribed intervals. In addition, as mentioned above, for the sake of simplicity, in this embodiment, it is assumed that there is no confusing device, no signal of the presence of ground objects, and no confusion cause.

[0112] In step S201, the controller 40 estimates the driving load of the user (driver) and determines whether the driving load is less than a prescribed value (seventh threshold value). If the driving load is greater than the seventh threshold value, the controller 40 temporarily interrupts the processing related to information provision until the driving load is less than the seventh threshold value. On the other hand, if the driving load is less than the seventh threshold value, the controller 40 performs the processing of step S202.

[0113] In step S202, the controller 40 obtains the user's line of sight direction and the time the line of sight is directed in that direction (line of sight information), and determines whether the time the user's line of sight is directed forward in the line of sight direction (hereinafter also referred to as the gaze time) is longer than a prescribed time (fourth threshold). If the gaze time is shorter than the fourth threshold, the controller 40 returns to the process of step S201 and does not perform the process after step S203 until the driving load is less than the seventh threshold and the gaze time is longer than the fourth threshold. On the other hand, if the gaze time is longer than the fourth threshold, the controller 40 performs the process of step S203.

[0114] In step S203 , the controller 40 determines a visual confirmation area based on the line of sight information, and determines an object included in the visual confirmation area based on the line of sight information, current location information on the map, and surrounding information.

[0115] In step S204, the controller 40 determines whether the user has spoken a specific keyword. In addition to the aforementioned example ("What is that?"), the specific keyword can also be set to, for example, words such as "What is that?" and "Hmm?". In the case where it is detected that the user has spoken a specific keyword, the controller 40 executes the processing of step S214. On the other hand, in the case where it is not detected that the user has spoken a specific keyword, the controller 40 executes the processing of step S205.

[0116] In step S214 , the controller 40 determines the central object as the object of interest, and executes the process of step S208 .

[0117] In the case where the user is not detected to have spoken a specific keyword, in step S205, the controller 40 estimates the user's attention for each object included in the visual confirmation area. As described above, the closer the object is to the center of the visual confirmation area, the higher the attention is estimated to be. In addition, the more the user likes the object, the higher the attention is estimated to be. In addition, when the time when the user's line of sight is directed to the central object closest to the center of the visual confirmation area exceeds a specified time (fifth threshold), the sound emitted by the user during the period when the user's line of sight is directed to the central object is detected, and when the sound feature value of the sound exceeds a specified value (sixth threshold), the attention of the central object is estimated to be higher.

[0118] In step S206, the controller 40 determines whether the attention degree (maximum attention degree) of the object with the highest attention degree is higher than a predetermined threshold value (third threshold value). The third threshold value can be set, for example, to the following value: if the user may want to obtain some information about the object, the attention degree exceeds the third threshold value. If the attention degree of the object with the highest attention degree is higher than the third threshold value, the controller 40 performs the processing of step S207. On the other hand, if the attention degree of all objects is below the third threshold value, the controller 40 returns to the processing of step S201.

[0119] In step S207 , the controller 40 determines the object with the highest degree of attention as the object of attention.

[0120] When the object of interest is determined, the controller 40 estimates the confidence in step S208. As described above, the confidence is estimated based on the information of the objects included in the visual confirmation area and the user information. The more objects included in the visual confirmation area, the lower the confidence is estimated to be. The more objects that are highly preferred by the user are included in the objects included in the visual confirmation area, the lower the confidence is estimated to be. The greater the sound feature amount when the user speaks, the higher the confidence is estimated to be.

[0121] When the confidence level is estimated, in step S209, the controller 40 outputs information (first information) in an output mode corresponding to the confidence level. For example, when the confidence level is low, features are added to the sound data and image data to express lack of confidence and output them. On the other hand, when the confidence level is high, features are added to the sound data and image data, such as making the text and sound louder, or making the character express a generous expression, to output them.

[0122] When the first information is output, the controller 40 executes the processes shown in steps S104 to S106 and S115, S116, and S125. These processes are the same as those in the first embodiment, and thus description thereof will be omitted.

[0123] According to the information providing method of the second embodiment described above, the following effects can be obtained.

[0124] In the information providing method of the present embodiment, the object of interest that the user pays the most attention to among the objects located in the visual confirmation area that the user can visually confirm is determined, and the correctness of the determination of the object of interest, that is, the degree of confidence, is estimated based on the information of the object in the visual confirmation area and the user information (the preference information related to the user's preference and the user action information related to the user's action). In addition, the object of interest is determined as a topic, and the information related to the object of interest is determined as the first information provided to the user, and the first information is output in an output mode corresponding to the degree of confidence. In this way, the first information is output in an output mode corresponding to the degree of confidence, so that even if information that the user does not care about is provided, the user's impression can be alleviated.

[0125] In the information providing method of this embodiment, the user's attention to each object located in the visual confirmation area is estimated based on the preference information and the user's action information including the user's line of sight direction, the voice feature quantity when the user speaks, and the driving load of the vehicle driven by the user. In addition, the information related to the object of interest to which the user pays the highest attention is determined as the first information to be provided to the user. In this way, the user's attention to each object in the visual confirmation area is estimated based on the preference information and the user's action information, so even if the visual confirmation area contains multiple objects, it is possible to determine which object information should be provided to the user.

[0126] Furthermore, since the user's attention is estimated based not only on the user's line of sight and speech (voice characteristics) but also on the user's driving load, it is possible to prevent the attention from being mistakenly estimated to be high when the user wants to concentrate on driving. As a result, the accuracy of the estimation of the attention is further improved.

[0127] In the information providing method of the present embodiment, the closer the object is to the center of the visual confirmation area, the higher the user's attention to the object is estimated to be, and the more the user likes the object, the higher the user's attention to the object is estimated to be. In addition, when the time when the user directs his / her sight toward the object closest to the center of the visual confirmation area, i.e., the central object, exceeds a prescribed time (fifth threshold), and when the user's voice is detected while the user's sight is directed toward the central object and the sound feature of the voice exceeds a prescribed value (sixth threshold), the user's attention to the central object is estimated to be higher. In this way, the user's attention to the object is not simply estimated to be the highest, but the preference information, the time when the sight is directed toward the central object, the sound feature, etc. are also considered to estimate the user's attention to the object. As a result, the accuracy of the estimation of the attention is further improved.

[0128] In the information providing method of this embodiment, when the driving load of the user while driving the vehicle is above a predetermined value, the user's attention to the object is estimated to be low. Thus, when the user wants to concentrate on driving, it is possible to prevent the attention to the object from being mistakenly estimated to be high and information related to the object from being provided to the user.

[0129] In the information providing method of this embodiment, when the user has the highest degree of attention to an object and the degree of attention is higher than a predetermined third threshold, the object is determined as the object of interest. By setting the threshold (third threshold) for determining the object of interest in this way, it is possible to prevent, for example, an object that the user has not paid attention to but has only entered the field of vision from being determined as the object of interest. That is, it is possible to prevent the user from being annoyed by the provision of information that the user does not particularly want.

[0130] In the information providing method of this embodiment, the greater the number of objects in the visual confirmation area, the lower the confidence is estimated, the greater the number of objects that the user likes among the objects in the gaze area, the lower the confidence is estimated, and the greater the voice feature amount when the user speaks, the higher the confidence is estimated. Since the confidence is quantitatively estimated in this way, it is easy to determine how to provide information to the user.

[0131] In addition, in the present embodiment, the confidence is estimated based on the information of the object and the user information, and the first information is output in an output mode corresponding to the confidence level. However, for example, a timing determination unit may be provided to estimate whether the timing of providing information is appropriate, and output the information in an output mode corresponding to the appropriateness of the timing of providing information. In this case, whether the timing is appropriate can be determined based on the user's driving condition (driving load, etc.), data on the user's reaction when the information was provided in the past, and the like.

[0132] In addition, the information may be output in an output mode corresponding to the newness of the information provided to the user (the freshness of the information), the reliability of the information source, etc. That is, the output mode may be determined not only based on the certainty of the present embodiment, but also based on a comprehensive judgment of the information.

[0133] In any of the embodiments, the information determination unit 48 acquires information to be provided to the user from the information accumulation database 46 , but the present invention is not limited thereto, and the information to be provided to the user may be acquired from an external database.

[0134] in addition, Figure 2 , Figure 4 Each of the processing steps shown shows an example for implementing the present embodiment. The order of a part of each processing step may be exchanged, a part of each processing step may be omitted, or other processing steps may be added within the scope of implementing the present embodiment.

[0135] Although the embodiments of the present invention have been described above, the above embodiments merely illustrate a part of application examples of the present invention, and the technical scope of the present invention is not intended to be limited to the specific structures of the above embodiments.

[0136] Although the above-mentioned embodiments have been described as separate embodiments, they may be appropriately combined.

[0137] In addition, the present application claims priority based on Japanese Patent Application No. 2022-078464 filed with the Japan Patent Office on May 11, 2022, the entire contents of which are incorporated herein by reference.

Claims

1. An information providing method for providing information to a user who is a passenger of the vehicle using an output device provided in the vehicle, wherein: determining a topic to be provided to the user and first information related to the topic, causing the output device to output the first information, detecting a reaction of the user when the output device outputs the first information, estimating the user's interest in the topic based on the user's response, When the interest level is higher than a predetermined first threshold, the output device is caused to output second information related to the topic, the second information being more detailed than the first information.

2. The information providing method according to claim 1, wherein: The user's reaction includes the user's action, The user's interest in the first information is estimated based on the user's action when the output device outputs the first information.

3. The information providing method according to claim 1 or 2, wherein: The output device includes a display unit for outputting images and a sound output device for outputting sounds. The first information is information that can be output at least by sound, dividing the first information into a first part and a second part, causing the display unit to display at least one of the first character and the second character, causing the sound output device to output the first part as a speech made by the first character, The audio output device is caused to output the second portion as speech made by the second character.

4. The information providing method according to claim 3, wherein: When it is detected that the user has performed an action of visually checking at least one of the first character and the second character when the output device outputs the first information, the interest level is estimated to be high.

5. The information providing method according to claim 4, wherein: The second information is information that can be output at least by sound, The audio output device is caused to output the second information as a speech made by the first character or the second character.

6. The information providing method according to claim 1 or 2, wherein: The output device includes a display unit for outputting an image. The second information is information that can be output at least through an image, The display unit is caused to output the second information as an image.

7. The information providing method according to claim 1 or 2, wherein: The first information is information related to operation of a device in the vehicle that can be used by the user.

8. The information providing method according to claim 7, wherein: When it is detected that a new device has been added to the vehicle, a prescribed device in the vehicle has been updated, or a prescribed device in the vehicle has not been used for a long time, the device that has been detected to have been added, updated, or not been used for a long time is determined as a topic, and information related to the operation of the device is determined to be output as the first information.

9. The information providing method according to claim 7, wherein: detecting an action of the user before determining the first information, When it is estimated based on the detected behavior of the user that the user is confused about the operation of a predetermined device, the predetermined device is determined as a topic, and information related to the operation of the predetermined device is determined as the first information to be output.

10. The information providing method according to claim 1 or 2, wherein: obtaining position information related to the current position of the vehicle, Based on the position information and the map information, acquiring feature information related to features around the current position of the vehicle, The feature is determined as a topic, and the feature information is determined as the first information to be output.

11. The information providing method according to claim 1, wherein: detecting the user's sight direction, Based on the detected sight line direction of the user, a visual confirmation area that the user can visually confirm and an object located in the visual confirmation area are determined; detecting the user's actions and estimating the user's attention to each object located in the visual confirmation area based on user information, wherein the user information includes preference information related to the user's preferences accumulated in a database and user action information related to the detected user's actions, determining the object of highest interest to the user, namely the object of interest, estimating the degree of accuracy, i.e., the degree of certainty, of identifying the object of interest based on the information of the object in the visual confirmation area and the user information, The object of interest is determined as a topic, information related to the object of interest is determined as first information, and the first information is output in an output method corresponding to the level of the certainty.

12. The information providing method according to claim 11, wherein: The user action information includes the direction of the user's line of sight, a feature amount of a voice when the user speaks, and a driving load of the vehicle driven by the user.

13. The information providing method according to claim 11 or 12, wherein: The closer the object is to the center of the visual confirmation area, the higher the user's interest in the object is estimated to be, and the more the user likes the object, the higher the user's interest in the object is estimated to be, Regarding the user's attention to the object closest to the center of the visual confirmation area, namely the central object, if the time when the user directs his / her gaze toward the central object exceeds a specified time, and if the user's voice is detected while the user directs his / her gaze toward the central object and the sound feature value of the voice exceeds a specified value, the user's attention to the central object is estimated to be higher.

14. The information providing method according to claim 11 or 12, wherein: When the driving load of the user when driving the vehicle is greater than or equal to a predetermined value, The user's interest in the object is estimated to be low.

15. The information providing method according to claim 11 or 12, wherein: If the object has the highest degree of interest of the user and the degree of interest is higher than a predetermined third threshold, the object is determined as the object of interest.

16. The information providing method according to claim 11 or 12, wherein: The greater the number of objects in the visual confirmation area, the lower the estimated certainty is; the more objects that the user likes among the objects in the gaze area, the lower the estimated certainty is; and the greater the sound feature value when the user speaks, the higher the estimated certainty is.

17. The information providing method according to claim 1 or 11, wherein: When the estimated interest of the user in the first information is lower than a predetermined second threshold value that is smaller than the first threshold value, the first information is changed.

18. The information providing method according to claim 1 or 11, wherein: When the estimated user's interest in the first information is lower than a predetermined second threshold value that is smaller than the first threshold value for a predetermined time period, the first information is changed.

19. An information providing system for providing information to a user who is an occupant of a vehicle, the information providing system comprising: an output device for outputting said information; an information determination unit that determines information to be provided to the user; an information output unit configured to cause the output device to output the information determined by the information determination unit to be provided to the user; A user status information acquisition device, which detects the user's reaction to the output information; as well as an interest estimation unit for estimating the user's interest in the output information based on the user's reaction acquired by the user status information acquisition device, When the user's interest in the output first information is estimated to be higher than a predetermined first threshold, the information output unit causes the output device to output second information related to the first information, the second information being more detailed information.

Images