Information provision method and information provision system

Abstract

Provided is an information provision method in which an output device provided to a vehicle is used to provide information to a user who is an occupant of the vehicle. The information provision method involves determining a topic to be provided to the user and first information pertaining to the topic, causing the output device to output the first information, detecting the user's reaction when the output device outputs the first information, and inferring the user's level of interest in the topic on the basis of the user's reaction. If the level of interest is higher than a prescribed first threshold, second information, which is more detailed than the first information, pertaining to the topic is caused to be output by the output device.

Description

Information provision method and information provision system

[0001] The present invention relates to an information providing method and an information providing system.

[0002] JP2018-134902A discloses an information presentation system that displays driving-related information by changing the shape of a plurality of display bodies and makes the driver (user) of the vehicle aware of the driving-related information.

[0003] In a typical conversation between people, a speaker on a certain topic will guess the interest (level of interest) of the other person, and if they guess that the other person is highly interested, they may add more detailed information about the topic.

[0004] The information presentation system described in JP2018-134902A does not estimate the user's level of interest, and therefore may not be able to provide the user with the information they want about topics that they are highly interested in. Furthermore, the system may provide unnecessarily detailed information about subjects that the user is less interested in, which may annoy the user.

[0005] The present invention has been made in view of the above-mentioned problems, and has as its object to provide an information providing method and an information providing system that can provide information that a user desires.

[0006] According to one aspect of the present invention, there is provided an information providing method for providing information to a user who is an occupant of a vehicle using an output device provided in the vehicle. The information providing method includes: determining a topic to be provided to the user and first information related to the topic; causing the output device to output the first information; detecting a reaction of the user when the output device outputs the first information; and estimating the user's level of interest in the topic based on the reaction. If the level of interest is higher than a predetermined first threshold, causing the output device to output second information related to the topic, which is more detailed information than the first information.

[0007] Fig. 1 is a block diagram showing a schematic configuration of an information provision system employing an information provision method according to a first embodiment of the present invention. Fig. 2 is a flowchart explaining the information provision method according to the first embodiment. Fig. 3 is a block diagram showing a schematic configuration of an information provision system employing an information provision method according to a second embodiment. Fig. 4 is a flowchart explaining the information provision method according to the second embodiment.

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0009] (First Embodiment) Fig. 1 is a block diagram showing a schematic configuration of an information provision system 100 that employs an information provision method according to a first embodiment of the present invention. The information provision system 100 is mainly mounted on a vehicle or the like. In this embodiment, the vehicle on which the information provision system 100 is mounted is simply referred to as a vehicle. Furthermore, although the user of the information provision system 100 is the vehicle's occupant, i.e., the driver and / or passenger, in this embodiment, for simplicity, the driver of the vehicle is referred to as the user.

[0010] The information providing system 100 includes a user state information acquisition device 10 , a vehicle information acquisition device 20 , an output device 30 , and a controller 40 .

[0011] The user state information acquisition device 10 is a group of devices that detect the state of the user (driver), and includes an in-vehicle camera, an in-vehicle microphone, a passenger sensor, an eye tracker, a body temperature sensor, a heart rate sensor, a sweat sensor, an EEG sensor, etc. The in-vehicle camera is one or more imaging devices that capture images of the interior of the vehicle and obtain still or video images. In particular, the in-vehicle camera continuously or intermittently captures an area including the driver (user) to obtain images (including video) for detecting the user's movements. Hereinafter, the images captured by the in-vehicle camera are referred to as in-vehicle images. The in-vehicle microphone is one or more sound collection devices that capture sounds or voices generated inside the vehicle. The in-vehicle microphone captures at least the voice uttered by the user. The passenger sensor detects the presence or absence of a passenger in the vehicle. The eye tracker detects the user's gaze direction. The body temperature sensor, heart rate sensor, sweat sensor, and EEG sensor detect the user's body temperature, heart rate, sweat rate, and brain waves, respectively. Information such as images, audio, passengers, the user's gaze direction, the user's body temperature, heart rate, amount of sweat, and brain waves acquired by the user state information acquisition device 10 is transmitted as user state information to the controller 40 described below.

[0012] The devices (sensors) included in the user state information acquisition device 10 are not limited to those described above. For example, the user's movements may be detected by an infrared sensor, and the user's gaze direction may be detected by an in-vehicle camera. Furthermore, the user state information acquisition device 10 only needs to include devices that enable the controller 40 (described below) to detect the user's movements, voice, and gaze direction, and other devices may be added as desired.

[0013] The vehicle information acquisition device 20 is a group of devices that acquire information about the current position of the vehicle and the surrounding conditions, and includes a current location information acquisition device 21 and a surrounding conditions acquisition device 22 .

[0014] The current location information acquisition device 21 is a device that acquires location information related to the current location of the vehicle and includes a GPS (Global Positioning System) receiver, a direction sensor, a distance sensor, a steering angle sensor, etc. The GPS receiver detects the vehicle's location using artificial satellites, and the direction sensor, distance sensor, and steering angle sensor respectively detect the vehicle's direction (orientation), the distance between the vehicle and objects around the vehicle, and the vehicle's steering angle (direction of travel). The current location information acquisition device 21 acquires vehicle location information by combining information detected by the GPS receiver, direction sensor, distance sensor, steering angle sensor, etc. Note that the devices (sensors) included in the current location information acquisition device 21 are not limited to those described above, and any device that can acquire location information related to the vehicle's current location may be used. For example, location information may be acquired using a GNN (Global Navigation Satellite System) sensor, or location information may be acquired only from the GPS receiver. The location information acquired by the current location information acquisition device 21 is transmitted to the controller 40, which will be described later.

[0015] The surrounding condition acquisition device 22 includes an exterior camera. The exterior camera is one or more imaging devices that capture images of the outside of the vehicle and obtain still or video images. By continuously or intermittently capturing images of the vehicle's surroundings, the exterior camera acquires images (including video images) for detecting the conditions around the vehicle. The images of the outside of the vehicle acquired by the surrounding condition acquisition device 22 are transmitted to a controller 40 (described later) as surrounding information. The surrounding condition acquisition device 22 may also include a front vehicle distance sensor and a rear vehicle distance sensor that acquire information on the distance between the vehicle and the front vehicle and the rear vehicle.

[0016] The output device 30 includes an audio output device 31 that outputs audio, and a display unit 32 that outputs images.

[0017] The audio output device 31 is a device that outputs audio data to the user, and includes, for example, a speaker that can output audio. The audio output device 31 outputs audio data to be provided to the user that is received from the controller 40 (described later) as audio.

[0018] The display unit 32 is a device that outputs image data to the user, such as a display that can output images. The display unit 32 outputs image data to be provided to the user that is received from the controller 40 (described later), and displays images (including videos).

[0019] The controller 40 is a unit that controls the information providing system 100, determines the information to be provided to the user, and causes the output device 30 to output the information.

[0020] The controller 40 is composed of a computer equipped with a central processing unit (CPU), read-only memory (ROM), random access memory (RAM), and an input / output interface (I / O interface), and executes specific programs to perform processes for providing information to the user.

[0021] The controller 40 includes, as components for determining information to be provided to the user, a map information acquisition unit 41, a feature information acquisition unit 42, a function database 43, a confusion factor detection unit 44, a confusion estimation unit 45, an information accumulation database 46, an interest level estimation unit 47, and an information determination unit 48. The controller 40 also includes, as components for causing the output device 30 to output information, a character database 49 and an information output unit 50.

[0022] The map information acquisition unit 41 identifies the current position of the vehicle on a map. Position information regarding the current position of the vehicle is input to the map information acquisition unit 41 from the current position information acquisition device 21. The map information acquisition unit 41 also acquires map information from a map database (not shown) inside or outside the controller 40, and identifies the current position of the vehicle on the map by comparing the position information with the map information. The map information acquisition unit 41 transmits information about the current position of the vehicle on the map (current position information on the map) to the feature information acquisition unit 42. The map database is updated with the latest information at predetermined intervals.

[0023] The feature information acquisition unit 42 acquires feature information related to features around the vehicle's current location. Here, features refer to distinctive facilities, historical buildings, and the like. The feature information acquisition unit 42 receives current location information on the map from the map information acquisition unit 41 and surrounding area information from the surroundings acquisition device 22. The feature information acquisition unit 42 also acquires map information from a map database (not shown) internal or external to the controller 40. The map database stores information about features at each location, and the map information includes information about the features. The feature information acquisition unit 42 acquires information about features visible to the user and information about other features around the vehicle (hereinafter, both of these are collectively referred to as feature information) from the current location information, surrounding area information, and map information on the map. The feature information acquisition unit 42 also registers the feature information in the information storage database 46. If a feature visible to the user is present, the unit 42 transmits a signal indicating this (a feature presence signal) to the information determination unit 48 (described later). The feature information acquired by the feature information acquisition unit 42 is a candidate for information to be provided to the user. Note that the above-described method for acquiring feature information is one example, and the method for acquiring feature information is not limited to this.

[0024] The function database 43 is a database that stores data related to the functions of devices in the vehicle. Information on the functions installed in each device in the vehicle, the frequency of use of each device and the functions installed in each device, etc. are registered in the function database 43. The information in the function database 43 is updated at predetermined intervals.

[0025] The confusion factor detection unit 44 detects, as a confusion factor, a function that confuses the user among the functions of the devices in the vehicle. Here, a function that confuses the user (confuse factor) refers to a function that is recommended for use by the user but is not used because the user is unaware of the function or does not know how to use it. Specifically, this refers to a function of a device that has been added to the vehicle, a specific device that has been updated, or a device that has not been used for a long period of time (hereinafter simply referred to as a long period of time). Note that a device that has not been used for a long period of time includes a case where the device itself is being used but some of its functions have not been used for a long period of time. The confusion factor detection unit 44 queries the function database 43 at predetermined intervals (periods) to determine whether a new device has been added, a device has been updated, or a device that has not been used for a long period of time. When it is detected that a device has been added or updated, or that a specific device has not been used for a long period of time, the confusion factor detection unit 44 transmits information about the added or updated function of the device, or the function of the device that has not been used for a long period of time, as confusion factor information to the confusion estimation unit 45 and registers it in the information storage database 46.

[0026] The confusion estimation unit 45 estimates whether the user is confused about how to use a specific device (or function). The confusion estimation unit 45 receives user state information from the user state information acquisition device 10 and confusion factor information from the confusion factor detection unit 44. The confusion estimation unit 45 estimates whether the user is confused about how to use a specific device based on the detected user behavior included in the user state information and the confusion factor information. Here, user behavior refers to the user's movements, the voice emitted by the user, changes in the user's line of sight, etc. For example, if it is detected from the user's behavior captured by an in-vehicle camera that the user is attempting to use a device equipped with a function that causes confusion, and if it is detected that the user is using the device incorrectly, it can be estimated that the user is confused. If it is estimated that the user is confused about how to use a specific device, the confusion estimation unit 45 registers information about the device in the information storage database 46 as confusion device information. Note that the method for estimating whether the user is confused is not limited to the above method. For example, the estimation may be made only from the user's behavior detected by the user state information acquisition device 10 .

[0027] The information storage database 46 stores candidate information for information to be provided to the user. For example, the information storage database 46 stores information regarding the handling of various devices in the vehicle. The information storage database 46 also stores feature information from the feature information acquisition unit 42, confusion factor information from the confusion factor detection unit 44, confusing device information from the confusion estimation unit 45, etc. The information storage database 46 is updated at predetermined intervals.

[0028] The interest level estimation unit 47 estimates the user's level of interest in information (first information) provided to the user by the information providing system 100. User state information is input to the interest level estimation unit 47 from the user state information acquisition device 10, and the user's level of interest is estimated based on the user state information when the information (first information) is provided to the user. The estimated user's level of interest is transmitted to the information determination unit 48. Details of the interest level estimation unit 47 will be described later.

[0029] The information determination unit 48 determines a topic and information to be provided to the user from the information storage database 46. Specifically, the information determination unit 48 queries the information storage database 46 for the presence or absence of confusing device information. If the confusing device information is registered, the information determination unit 48 determines the device that the user is having difficulty using as the topic and determines information about the device as information (first information) to be provided to the user. Furthermore, if the information determination unit 48 receives a signal indicating that a user-visible feature exists (a feature presence signal) from the feature information acquisition unit 42, the information determination unit 48 determines the feature as the topic and determines information about the user-visible feature, among the feature information registered in the information storage database 46, as information (first information) to be provided to the user. Furthermore, the information determination unit 48 queries the information storage database 46 for the presence or absence of confusion factor information. If a confusion factor exists, the information determination unit 48 determines the device equipped with the confusing function as the topic and determines information about the device as information (first information) to be provided to the user. Here, the first information to be provided to the user is, for example, an overview of the object determined as the topic. When determining the topic and the first information, there is no particular limitation as to which of the confusing device, the feature information, and the confusion factor is given priority, but in this embodiment, the confusing device has the highest priority, followed by the feature information and the confusion factor. Note that if neither the confusing device information nor the confusion factor information is registered in the information accumulation database 46 and the information determination unit 48 has not received a feature presence signal, the topic and information related to the topic (first information) to be provided to the user are randomly selected from the information stored in the information accumulation database 46.

[0030] The information determination unit 48 also receives an input of the user's level of interest from the interest level estimation unit 47. When the first information is provided to the user, if the user's level of interest in the first information is higher than a predetermined threshold (first threshold), the information determination unit 48 determines that information to be provided to the user is more detailed than the first information (second information, detailed information) regarding the object being discussed. The information determined to be provided to the user is transmitted to the information output unit 50. If the user's level of interest in the first information is lower than a second threshold set lower than the first threshold, the information determination unit 48 changes the topic and the first information.

[0031] The character database 49 stores character-specific information (character-specific information), such as the appearance, speech style, movements, and personality of the characters to be displayed on the display unit 32 of the output device 30 by the information output unit 50 (described later). A plurality of characters are registered in the character database 49, and character-specific information for each character is stored. Note that characters are not limited to human forms, and may be represented in the form of animals, plants, objects, or other symbols.

[0032] The information output unit 50 determines in what form the information (first information and second information) to be provided to the user determined by the information determination unit 48 will be output. In this embodiment, two characters are displayed on the display unit 32, and the first information and second information are output as a conversation between the characters. In this embodiment, the first information and second information are information that can be output by voice.

[0033] The information output unit 50 references the character-specific information stored in the character database 49 and determines which character will speak the information to be provided to the user and in what scenario. Specifically, the information output unit 50 determines the character to be displayed on the display unit 32 from the character database 49 and acquires the character-specific information from the character database 49. The information output unit 50 also divides the first information into a first part spoken by one character (first character) and a second part spoken by the other character (second character). The information output unit 50 then outputs (displays) image (including video) data of the determined character on the display unit 32, and outputs audio data of the first part and audio data of the second part from the audio output device 31 in accordance with the movements of the character displayed on the display unit 32. In this way, the first information is provided to the user. By dividing the first information into two parts and having two characters speak the first information, the first information can be provided to the user in the form of a conversation between the characters. Therefore, the user's responsibility for responding to utterances from the information providing system 100 is reduced, and the user's comfort level is improved.

[0034] When the first information is output via the output device 30, the information output unit 50 transmits a signal indicating that the first information has been output to the interest level estimation unit 47.

[0035] Furthermore, the information output unit 50 determines which character will speak the second information (detailed information) to be provided to the user determined by the information determination unit 48. Then, an image (including a moving image) of the determined character is displayed on the display unit 32, and audio data of the second information is output from the audio output device 31 in accordance with the movement of the character displayed on the display unit 32. In this way, the second information is provided to the user.

[0036] The information output unit 50 outputs, from the audio output device 31, audio data that has been adapted to a speech style (e.g., wording) that matches the character's personality, etc., based on the character-specific information of the character being used.

[0037] The first information is output as a conversation between the first character and the second character, and the second information is output as an utterance by the first character or the second character, but the output form of the first information and the second information is not limited to this.

[0038] The method for determining the character to be displayed on the display unit 32 is not particularly limited, and for example, a character with an appearance and personality that matches the topic to be presented to the user can be selected.

[0039] Next, the interest level estimation unit 47 will be described in detail.

[0040] When first information to be provided to a user is output via the output device 30, a signal indicating that the first information has been output is input from the information output unit 50 to the interest level estimation unit 47. When the signal indicating that the first information has been output is input, the interest level estimation unit 47 estimates the user's level of interest in the first information (hereinafter simply referred to as the user's interest level) based on the user's reaction when the first information is output. Here, the user's reaction refers to the user's state detected by the user state information acquisition device 10 when the output device 30 outputs the first information, and the interest level estimation unit 47 estimates the user's level of interest based on the user state information from the user state information acquisition device 10.

[0041] More specifically, the user's reaction includes the user's movements, and the interest level estimation unit 47 estimates the user's level of interest in the first information based on the user's movements included in the user state information. Here, the user's movements include the user's movements detected from an in-vehicle image, the user's speech captured by an in-vehicle microphone, and the user's gaze direction detected by an eye tracker. For example, when the output device 30 outputs the first information, if the user state information acquisition device 10 detects that the user has responded with a response, nodded, uttered a sound such as "Ah," continued to look at the object being discussed for a predetermined period of time, or visually recognized at least one of the first character and the second character who spoke the first information, the interest level estimation unit 47 estimates the user's level of interest as high. Specifically, for example, the eye tracker acquires image information of the user and detects the gaze direction of the user's face included in the image information, and determines whether the detected gaze direction coincides (or approximately coincides) with a specific direction connecting the user's eyes and the output device 30 displaying the first and / or second characters. If the detected gaze direction is determined to match (or approximately match) the specific direction, it is determined that the user has looked at the first and / or second character, and the user's level of interest is estimated to be high. On the other hand, if the detected gaze direction is determined to not match (or approximately match) the specific direction, it is determined that the user has not looked at the first and / or second character. Note that known image recognition technology can be used for the face detection process and gaze detection process. In this way, if the user performs an action that indicates interest in the first information, the user's level of interest is estimated to be higher than a predetermined threshold (first threshold), and more detailed information (second information) about the object being discussed is provided to the user.

[0042] The above is the main configuration of the information providing system 100. As described above, in the information providing system 100, information to be provided to the user is determined from the information storage database 46, and the information is provided to the user via the output device 30.

[0043] However, if information to be provided to a user is determined without estimating the user's level of interest, there is a risk that the information desired by the user about a topic that the user is highly interested in may not be provided. Also, there is a risk that unnecessarily detailed information about a subject that the user is not highly interested in may be provided, causing the user to feel annoyed.

[0044] On the other hand, it is difficult to estimate the user's level of interest before providing information, and there is a high risk of erroneous estimation of the level of interest.

[0045] In contrast, the information provision system 100 employing the information provision method of this embodiment determines a topic to be provided to a user and first information related to the topic (e.g., an outline of the topic's subject), and estimates the user's level of interest in the first information based on the user's reaction when the first information is output. If the level of interest is higher than a first threshold, second information related to the topic, which is more detailed than the first information, is output. In this way, the second information, which is more detailed information, is output only when the user's level of interest is high, allowing the user to obtain detailed information only about topics that interest the user. In other words, the information desired by the user can be provided. Furthermore, providing detailed information about topics that the user is not interested in, which would annoy the user, is prevented. Furthermore, because the user's level of interest is estimated based on the user's reaction to the first information, the accuracy of the interest level estimation is improved compared to when the interest level is estimated before the information is provided.

[0046] 2 is a flowchart illustrating the information providing method according to the first embodiment. The following control is repeatedly executed by the controller 40 at predetermined time intervals. The following control may be started when the information providing system 100 is started, or may be started when some information has already been output to the user. The controller 40 constantly acquires user state information from the user state information acquisition device 10 and location information and surrounding area information from the vehicle information acquisition device 20 at predetermined time intervals.

[0047] In step S101, the controller 40 estimates whether the user is confused about how to use a specific device or function. As described above, whether the user is confused is estimated based on the detected user behavior included in the user state information. If it is estimated from the user behavior that the user is confused about how to use a specific device, the controller 40 executes the process of step S102. On the other hand, if it is not estimated that the user is confused, the controller 40 executes the process of step S111.

[0048] In step S102, the controller 40 determines the device about which the user is confused about how to use (the confusing device) as the topic, and determines a summary of how to use the device (the confusing device) as the first information. In this way, if the user's confusion is estimated from the user's behavior detected before determining the first information, the summary of how to use the confusing device is set as the first information.

[0049] In step S111, the controller 40 determines whether or not there is a feature that the user can see. The presence or absence of a feature is determined based on current location information, surrounding information, and map information on the map. If there is a feature that the user can see, the controller 40 executes the process of step S112. On the other hand, if there is no feature that the user can see, the controller 40 executes the process of step S121.

[0050] In step S112, the controller 40 determines a feature that can be seen by the user as a topic, queries the information storage database 46 for feature information about the feature, and determines a summary of the feature as first information.

[0051] In step S121, the controller 40 determines whether or not there is a confusion factor. As described above, if a new device is added to the vehicle, if a specific device in the vehicle is updated, or if it is detected that a specific device in the vehicle has not been used for a long period of time, it is determined that there is a confusion factor. If there is a confusion factor, the controller 40 executes the process of step S122. On the other hand, if there is no confusion factor, the controller 40 executes the process of step S131.

[0052] In step S122, the controller 40 determines the topic to be an added or updated function of a device, or a function of a device that has not been used for a long time, and determines a summary of the function as the first information.

[0053] In step S131, the controller 40 randomly selects information from among the information stored in the information storage database 46, determines the object related to the selected information as the topic, and determines the outline of the object as the first information.

[0054] When the first information is determined in step S102, S112, S122, or S131, the controller 40 causes the output device 30 to output the first information in step S103.

[0055] In step S104, the controller 40 estimates the user's level of interest in the topic. As described above, the user's level of interest is estimated based on the user's reaction when the first information is output, particularly the user's actions. For example, if it is detected that the user has made an action to visually recognize at least one of the first character and the second character, the level of interest is estimated to be high.

[0056] In step S105, the controller 40 determines whether the user's level of interest in the first information estimated in step S104 is higher than a predetermined first threshold. The first threshold can be set to a value such that the user's level of interest exceeds the first threshold when, for example, the output device 30 outputs the first information and the user responds by nodding, uttering a sound such as "ah," looking at the object being discussed for a predetermined period of time, or visually recognizing at least one of the first character and the second character is detected. If the user's level of interest in the first information is higher than the first threshold, the controller 40 executes the process of step S106. On the other hand, if the user's level of interest in the first information is equal to or lower than the first threshold, the controller 40 executes the process of step S115.

[0057] In step S106, the controller 40 causes the output device 30 to output second information, which is more detailed information about the topic than the first information.

[0058] In this way, the controller 40 estimates the user's level of interest based on the user's actions when the first information is output (step S104), and outputs the second information if the level of interest is higher than the first threshold (steps S105 and S106). Therefore, the user can obtain detailed information about the topic by simply performing the action of visually checking the character, for example. In other words, since there is no need to issue a special instruction to obtain the detailed information, the user's operation is less cumbersome and the user's comfort level is improved.

[0059] If the user's level of interest in the first information is equal to or less than the first threshold in step S105, the controller 40 determines in step S115 whether the user's level of interest in the first information estimated in step S104 is equal to or greater than a predetermined second threshold. The second threshold can be set so that the user's level of interest is lower than the second threshold, for example, when there is little change in the user's state or when the user utters a negative keyword (e.g., "Let's change the topic") when the first information is output. If the user's level of interest in the first information is equal to or greater than the second threshold, the controller 40 continues outputting the first information in step S116. On the other hand, if the user's level of interest in the first information is lower than the second threshold, the controller 40 executes the process of step S125.

[0060] In step S125, the controller 40 changes the topic and the first information, and causes the output device 30 to output the changed first information. As in step S131, the topic and the first information can be changed by randomly selecting information from among the information stored in the information storage database 46. After outputting the changed first information in step S125, the controller 40 returns to the processing of step S104 and estimates the user's level of interest in the changed topic.

[0061] In this way, when the user's interest in the first information is lower than the second threshold, the information (first information) to be provided to the user is changed (steps S115 and S125). This prevents the user from being annoyed by continuously being provided with information that the user is not interested in.

[0062] In this embodiment, for convenience, the steps are set to determine the topic and the first information, but in reality, by providing the user with the first information about a predetermined object, the object automatically becomes the topic. Therefore, it can be interpreted that the controller 40 determines the topic at the same time as determining the first information.

[0063] According to the information providing method of the first embodiment described above, the following effects can be obtained.

[0064] The information providing method of this embodiment estimates the user's level of interest in the first information based on the user's reaction when first information related to a topic to be provided to the user is output, and if the level of interest is higher than a first threshold, outputs second information, which is more detailed information related to the topic. This allows the user to obtain detailed information only about topics that interest the user. In other words, it is possible to provide the user with the information they want.

[0065] Furthermore, since the user's level of interest is estimated based on the reaction to the first information, the accuracy of the estimation of the level of interest is improved compared to when the level of interest is estimated before the information is provided.

[0066] The information providing method of this embodiment estimates the user's level of interest in the first information based on the user's behavior when the first information is output, and if the level of interest is higher than a first threshold, outputs second information, which is more detailed information about the topic. This allows the user to obtain detailed information about the topic by performing a simple behavior without any special instructions, thereby improving the user's comfort level.

[0067] In the information providing method of this embodiment, the first information is divided into a first part and a second part, and the first part is output as an utterance by a first character displayed on the display unit 32, and the second part is output as an utterance by a second character displayed on the display unit 32. In this way, by dividing the first information into two parts and having two characters utter the first information, it is possible to provide the user with the first information in the form of a conversation between the characters. This reduces the user's responsibility for responding to utterances from the information providing system 100, improving the user's comfort level.

[0068] In the information providing method of this embodiment, if it is detected that the user has performed an action of visually checking at least one of the first character and the second character when the first information is output, the user's level of interest in the first information is estimated to be high. Therefore, by simply performing the action of visually checking the character, the user can increase the level of interest above the first threshold and obtain detailed information (second information) related to the topic. In other words, since no special instructions are required to obtain the detailed information, the user's operational hassle is reduced and the user's comfort level is improved.

[0069] In the information providing method of this embodiment, when it is detected that a new device has been added to the vehicle, that a specific device has been updated, or that a specific device has not been used for a long time, information regarding the handling of the device that has been added, updated, or detected as not being used for a long time is determined as the first information to be provided to the user. This makes it possible to prompt the user to use a function that is recommended for use by the user but is not being used because the user is unaware of the function or does not know how to use it.

[0070] In the information providing method of this embodiment, when it is estimated from the user's behavior that the user is confused about how to use a specific device, information about how to use the specific device is determined to be the first information. This makes it possible to provide information about how to use a device that the user wants in a situation where the user is confused.

[0071] In the information providing method of this embodiment, if there is a feature that the user can see, the feature is determined to be the topic, and feature information related to the feature is determined to be the first information to be provided to the user. In this way, by discussing something within the user's field of vision, the user can guess where the provided information is coming from, which gives the user a sense of security.

[0072] In the information providing method of this embodiment, when the user's interest in the first information is lower than a predetermined second threshold value that is lower than the first threshold value, the first information is changed, thereby preventing the user from feeling annoyed by continuing to receive information that is not of interest to the user.

[0073] In the present embodiment, information is provided to the user by outputting audio and images, but this is not limiting. For example, information may be provided by audio only.

[0074] In the present embodiment, information is provided to the user through the speech of two characters displayed on the display unit 32. However, this is not necessarily limited to this. For example, the number of characters displayed may be one, or three or more. Furthermore, for example, the appearance of the character may be the appearance of the output device 30. In this case, instead of displaying a character on the display unit 32, the output device 30 itself becomes the character. Therefore, for example, the first information may be provided through a conversation between the animated output device 30 and the character displayed on the display unit 32. Here, the animated output device 30 includes, for example, an agent device that is a small robot installed on the dashboard of a vehicle. Note that the agent device may be, for example, a robot modeled after an animal (e.g., a rabbit), a robot modeled after a virtual creature (e.g., the face of an anime character), or a robot modeled after another object (e.g., a television-type device, a radio-type device), but is not limited to these. Other devices equipped with a user interface such as a display unit (e.g., a smartphone, a tablet terminal, a car navigation device, or an IVI (In-Vehicle Infotainment)) may also be used as the agent device. In this case, the robot described above is displayed on the display unit, and the manner in which the robot is moved can be changed by changing the manner in which the robot is displayed.

[0075] In addition, in this embodiment, the first information is changed when the user's level of interest in the first information is lower than the second threshold, but the first information may be changed when the user's level of interest remains lower than the second threshold for a predetermined period of time. This prevents the first information from being changed when the user is interested in the currently provided first information, as the first information is changed only after it becomes clearer that the user is not interested in the first information.

[0076] In addition, in this embodiment, if there is a feature that the user can see, the feature is determined to be the topic and feature information related to the feature is determined to be the first information, but this is not necessarily limited to this. If there is a feature around the vehicle, even if the user cannot see it, the feature around the vehicle may be determined to be the topic and feature information related to the feature determined to be the topic may be determined to be the first information.

[0077] Furthermore, in this embodiment, when there is no confusing device, no feature visible to the user, or no confusing factor, and when the user's level of interest in the first information is lower than the second threshold, the first information to be provided to the user is randomly selected, but this is not necessarily limited to this. For example, instead of randomly selecting the first information, an importance level may be set for each piece of information, and information with a higher importance level may be preferentially selected as the first information.

[0078] Furthermore, in this embodiment, the second information is output as speech by the first character or the second character, but this is not necessarily limited to this. For example, if the second information is information that can be output as an image, the second information may be displayed as an image on the display unit 32. This makes it possible to provide the second information to the user as an image displayed on the display unit 32, separate from the audio output device 31. In this case, the image may include not only a photograph or picture, but also text, etc.

[0079] In addition, in this embodiment, if there is a confusion factor, information about a device equipped with the function that causes the confusion is determined as the first information, and if there is a confusing device, information about that device is determined as the first information, but this is not necessarily limited to this. For example, even if there is no confusion factor or confusing device, information about how to use any device that the user can use in the vehicle may be determined as the first information. This improves the user's understanding of the functions of the devices in the vehicle, allowing the user to use various devices in the vehicle more effectively.

[0080] Second Embodiment An information providing method according to a second embodiment will be described with reference to Figures 3 and 4. Elements similar to those in the first embodiment are given the same reference numerals, and descriptions thereof will be omitted.

[0081] 3 is a block diagram showing a schematic configuration of an information providing system 200 that employs an information providing method according to the second embodiment. The second embodiment differs from the first embodiment in the method of determining the first information and the form of output of the first information.

[0082] As shown in FIG. 3, the information providing system 200 includes a user state information acquisition device 10 , a vehicle information acquisition device 20 , an output device 30 , and a controller 40 .

[0083] The user state information acquisition device 10 and the output device 30 are the same as those in the first embodiment, and therefore a description thereof will be omitted.

[0084] The vehicle information acquisition device 20 includes a current location information acquisition device 21 including a GPS receiver or the like that acquires vehicle position information, and a surrounding situation acquisition device 22 including an external camera or the like that acquires surrounding information, as well as a load detection device 23 in this embodiment.

[0085] The load detection device 23 includes a steering angle sensor, an accelerator opening sensor, etc. The steering angle and accelerator opening detected by the load detection device 23 are transmitted to the controller 40 and used to estimate the driving load, which will be described later.

[0086] In addition to the components described in the first embodiment, the controller 40 includes a user behavior information acquisition unit 61, a preference information database 62, a visual recognition area extraction unit 63, an attention level estimation unit 64, an attention object identification unit 65, a confidence level estimation unit 66, and an output feature generation unit 67. Note that in Fig. 3, the feature information acquisition unit 42, the function database 43, the confusion factor detection unit 44, the confusion estimation unit 45, and the interest level estimation unit 47 are omitted.

[0087] The user behavior information acquisition unit 61 includes a driving load detection unit 61A, a voice feature detection unit 61B, and a gaze detection unit 61C, and acquires user behavior information. User status information from the user status information acquisition device 10, position information, surrounding information, steering angle and accelerator opening information from the vehicle information acquisition device 20, and current location information from the map information acquisition unit 41 are input to the user behavior information acquisition unit 61.

[0088] The driving load detection unit 61A estimates the driving load of the user (driver) from user state information, location information, surrounding information, steering angle and accelerator opening information, and current location information. The driving load is estimated, for example, according to the steering entropy method. Preferably, the driving load detection unit 61A acquires driver skill data based on the current location information, surrounding information, time-series data of the steering angle, time-series data of the accelerator opening, etc., and uses this data to estimate the driving load. This can improve the accuracy of the driving load estimation.

[0089] The voice feature detection unit 61B detects voice features of the user's speech based on the user's speech captured by the in-vehicle microphone. Here, the voice features include the content of the speech (e.g., whether a specific keyword is included), the volume of the voice, the pitch of the voice, accent, etc. The voice feature detection unit 61B acquires voice feature quantities that quantify these voice features of the user's speech. Note that whether the voice is a user's speech can be determined, for example, by storing the user's voice features in advance.

[0090] The gaze detection unit 61C detects the time period during which the gaze is directed in the user's gaze direction based on the in-vehicle image acquired by the in-vehicle camera (user state information acquisition device 10) and the user's gaze direction detected by the eye tracker (user state information acquisition device 10). The gaze detection unit 61C acquires the user's gaze direction and the time period during which the gaze is directed as gaze information. Note that the method of acquiring gaze information is not limited to the above. For example, gaze information may be acquired only from the in-vehicle image.

[0091] The driving load information estimated by the driving load detection unit 61A, the audio feature amount acquired by the audio feature detection unit 61B, and the gaze information acquired by the gaze detection unit 61C are transmitted as user behavior information to an attention level estimation unit 64 and a confidence level estimation unit 66, which will be described later. The gaze information acquired by the gaze detection unit 61C is also transmitted to a visual recognition area extraction unit 63.

[0092] The preference information database 62 is a database that stores user preference information. The user preference information is acquired based on the user's past behavior, etc. For example, the more frequently a particular type of restaurant or facility is visited, the more the user registers that food or facility as being more preferred by the user. However, the method of acquiring preference information is not limited to the above method.

[0093] The visual recognition area extraction unit 63 identifies a visual recognition area visible to the user and objects (objects) contained within the visual recognition area. The visual recognition area extraction unit 63 receives the gaze information acquired by the gaze detection unit 61C, current location information on the map from the map information acquisition unit 41, and surrounding information (images of the exterior of the vehicle captured by the exterior camera) from the surroundings acquisition device 22. The visual recognition area extraction unit 63 compares the user's gaze direction contained in the gaze information with the current location information and surrounding information on the map to identify a visual recognition area visible to the user, and identifies objects contained within the visual recognition area based on the current location information and surrounding information on the map. However, if the user's gaze direction is directed for a period of time shorter than a predetermined time (fourth threshold), the visual recognition area extraction unit 63 determines that the object is not visible and does not identify the object. This predetermined time can be set, for example, to a period of time that prevents the user from recognizing the object in the gaze direction. The method for identifying the visual recognition area is not limited to the above method, but is at least determined based on the user's gaze direction. Information on the identified visual recognition area and object is sent to the attention level estimation unit 64 and the certainty level estimation unit 66, which will be described later.

[0094] The attention level estimation unit 64 estimates the user's level of attention to each object in the visual recognition area. The attention level estimation unit 64 receives user behavior information from the user behavior information acquisition unit 61 and information on the visual recognition area and the objects contained in the visual recognition area from the visual recognition area extraction unit 63. The attention level estimation unit 64 also queries the preference information database 62 to acquire user preference information related to the objects in the visual recognition area. The attention level estimation unit 64 estimates the user's level of attention to each object in the visual recognition area based on the user behavior information and preference information (hereinafter referred to as user information). Specifically, the closer an object in the visual recognition area is to the center of the visual recognition area, the higher the estimated user attention level (added points), and the higher the user's preference for the object. If preference information for an object in the visual recognition area is not registered in the preference information database 62, the attention level is not added. Furthermore, if the time that the user directs their gaze toward the central object closest to the center of the visual recognition area exceeds a predetermined time (a fifth threshold), the system detects the user's voice while the user is directing their gaze toward the central object, and if the voice feature of the voice exceeds a predetermined value (a sixth threshold), the system estimates the user's level of attention to the central object higher (adds points).In this way, rather than simply estimating the level of attention to the central object as the highest, the system also takes into account preference information, the time spent directing their gaze toward the central object, the voice feature, and the like, thereby further improving the accuracy of the estimation of the level of attention.

[0095] Note that, when the user's driving load is high, the user's attention level to the object is estimated to be low. Specifically, in this embodiment, when the user's driving load is equal to or greater than a predetermined value (seventh threshold), the user's attention level to all objects in the visible area is estimated to be 0. In other words, when the user's driving load is high, it is estimated that the user wants to concentrate on driving, so the attention level is not actually estimated, and the attention level is estimated only after the driving load becomes smaller than a predetermined value. Therefore, for example, when the user wants to concentrate on driving, the attention level of objects in the visible area is prevented from being erroneously estimated to be high. In this way, the user's driving load is also taken into account in the attention level estimation, and the user's attention level to objects is estimated to be low when the driving load is high, thereby further improving the accuracy of the attention level estimation.

[0096] Information on the degree of attention for each object (attention level information) estimated by the attention level estimation unit 64 is transmitted to the attention object identification unit 65 and the certainty level estimation unit 66 .

[0097] The attention object identification unit 65 identifies which object in the visual recognition area will be the attention object. Here, the attention object is an object that will be a topic of conversation for the user. The attention object identification unit 65 receives attention level information from the attention level estimation unit 64. Based on the attention level information, the attention object identification unit 65 identifies the object with the highest attention level among the objects in the visual recognition area, and identifies the object as the attention object if the user's attention level for the object is higher than a predetermined threshold (third threshold). By setting the threshold (third threshold) for identifying an object as the attention object, it is possible to prevent, for example, an object that simply comes into view without the user paying attention to it from being identified as the attention object. The attention object identification unit 65 typically identifies the central object closest to the center of the visual recognition area as the attention object. In particular, in this embodiment, if the user utters a specific keyword while directing their gaze toward the central object, the central object is designated as the object of attention (the attention level estimation unit 64 is configured so that the central object receives the highest attention level when the specific keyword is uttered). For example, if the user utters a specific interrogative word (keyword) such as "What's that?" while directing their gaze toward a building that is the central object, the speech feature value increases, and the central object (the building) is designated as the object of attention. Note that, in some cases, an object other than the central object may be designated as the object of attention. For example, if a cat and a park are located in the visual recognition area, the park is likely to be located in the center of the visual recognition area because it has a larger area than the cat. However, if the preference information database 62 registers that the user likes cats, the attention level toward the cat may be estimated to be high, and the cat may be designated as the object of attention. Information about the object of attention designated by the object of attention designation unit 65 (object of attention information) is transmitted to the confidence level estimation unit 66 and the information determination unit 48.

[0098] The certainty factor estimation unit 66 estimates a certainty factor, which is the degree of accuracy of the identification of the target of interest. The certainty factor estimation unit 66 receives as input user behavior information from the user behavior information acquisition unit 61, information on the target of interest and the target contained within the target of interest from the target of interest extraction unit 63, attention level information from the attention level estimation unit 64, and target of interest information from the target of interest identification unit 65. The certainty factor estimation unit 66 also queries the preference information database 62 to acquire user preference information regarding the target of interest within the target of interest. The certainty factor is estimated based on the information on the target of interest within the target of interest and the user information (user behavior information and preference information). Specifically, the certainty factor is estimated to be lower as the number of targets contained within the target of interest increases, lower as the target of interest within the target of interest includes more targets that are highly preferred by the user, and higher as the speech feature of the user when the user speaks increases. The greater the number of objects included in the visual recognition area, the more difficult it is to determine which object the user is actually paying attention to, and the lower the accuracy of identifying the object of attention. Conversely, when the number of objects included in the visual recognition area is small, it is highly likely that the user is actually paying attention to the identified object of attention, and the confidence level is estimated to be high. Furthermore, when the visual recognition area contains many objects that are highly preferred by the user, it is also difficult to determine which object the user is actually paying attention to, and the accuracy of identifying the object of attention is low. Conversely, when there are few objects that are highly preferred by the user among the objects included in the visual recognition area, the accuracy of identifying the object of attention is high, and the confidence level is estimated to be high. For example, when there is only one object that is highly preferred by the user in the visual recognition area, that object is identified as the object of attention, but the likelihood that the user is actually paying attention to that object is high. Furthermore, when the audio feature value when the user speaks is large, it is highly likely that the user is actually paying attention to the identified object of attention, and the confidence level is estimated to be high. For example, as was given as an example in the explanation of the attention object identification unit 65, when a building is in the direction of the user's gaze and the user utters a specific interrogative word (keyword) such as "What is that?", the speech feature value becomes large. In this case, it is highly likely that the user is actually paying attention to the building (attention object, central object), and therefore the certainty is estimated to be high.

[0099] As described above, the certainty, which is the degree of accuracy in identifying the target object, is quantitatively estimated in the certainty estimation unit 66. Information on the certainty estimated in the certainty estimation unit 66 (certainty information) is transmitted to the information determination unit 48, the information output unit 50, and the output feature generation unit 67.

[0100] When the attention object information and the confidence factor information are input to the information determination unit 48, the information determination unit 48 determines the topic and information to be provided to the user, as in the first embodiment. In this embodiment, for simplicity, it is assumed that there are no confusing devices, feature presence signals, or confusion factors. The information determination unit 48 determines the attention object as the topic and determines information (summary) about the attention object as the first information. Information about the attention object (including not only the summary (first information) but also detailed information (second information)) is acquired from the information accumulation database 46 if it is registered in the information accumulation database 46, or from an external database (not shown) if it is not registered. Note that, if there are confusing devices, feature presence signals, confusion factors, etc., the information determination unit 48 selects one of the confusing devices, feature, devices equipped with a function that is a confusion factor, or the attention object as the topic. In this case, the priority is not particularly limited, but, for example, the priority is set in the order of confusing devices, feature information, confusion factors, and attention object. The topic and first information to be provided to the user determined by the information determination unit 48 are transmitted to the information output unit 50 .

[0101] The output feature generation unit 67 receives confidence information from the confidence estimation unit 66. The output feature generation unit 67 is configured to be able to transmit and receive information to and from the information output unit 50, and characterizes the output form of the first information determined by the information determination unit 48 according to the confidence level. Specifically, when the first information is uttered using a character (including when the output device 30 itself is the character), the character's appearance, movements, voice, etc. are given characteristics according to the confidence level. Here, the appearance characteristics include the character's facial expression, size, color, shape, etc., and the movement characteristics include rotation, vibration, movement speed, movement speed, and gaze direction, etc. Furthermore, the voice characteristics include intonation, volume, and ending expressions, etc. When the confidence level is low, these characteristics are used to express a lack of confidence. For example, when the confidence level is low, the voice data and image data can be characterized by making the character's facial expression look unconfident, making the character smaller, or making the character's voice sound weak. To cite one specific example, suppose a character utters first information about a cat. If the confidence level is low, the voice data and image data are characterized so that the character lowers its eyebrows and utters something like, "By the way, there was a video of a cat that XX (user's name) likes up. ... Huh? I thought I was looking at a cat just now, but maybe I was wrong?" Furthermore, if the first information is provided by displaying text on the display unit 32, the text is given characteristics depending on the confidence level. Here, character characteristics include the thickness, size, font, color, and ending of the sentence. For example, if the confidence level is low, the image (text) data is characterized so as to express a lack of confidence by making the text thinner or smaller, adding a question mark to the end of the sentence, etc.

[0102] The information output unit 50 outputs the audio data and image data of the first information characterized by the output feature generation unit 67 via the output device 30. In this way, the information is provided to the user.

[0103] As described above, in the information providing method according to the second embodiment, the first information is output in an output format according to the degree of certainty, thereby making it possible to mitigate a negative impression on the user even when information that is not of interest to the user is provided.

[0104] Furthermore, since the confidence level is quantitatively estimated, it is easy to determine what form of information should be provided to the user. That is, since the confidence level is quantified, it is easy to determine, for example, how assertive an expression should be, how loud the volume or characters should be, etc.

[0105] If it is estimated from the user's reaction when the first information is output that the user's level of interest is low (for example, if the user's level of interest is lower than the second threshold in the first embodiment), a correction value acquisition unit may be provided that acquires necessary correction values ​​for the estimation of the attention level and the certainty level, thereby making the estimation of the attention level and the certainty level more suited to the user.

[0106] FIG. 4 is a flowchart illustrating an information provision method according to the second embodiment. The following control is repeatedly executed by the controller 40 at predetermined time intervals. The following control may be initiated when the information provision system 100 is started, or may be initiated when some information has already been output to the user. The controller 40 constantly acquires user state information from the user state information acquisition device 10, and location information, surrounding information, and steering angle and accelerator opening information from the vehicle information acquisition device 20 at predetermined time intervals. As mentioned above, for simplicity, it is assumed in this embodiment that there are no confusing devices, feature presence signals, or confusing factors.

[0107] In step S201, the controller 40 estimates the driving load of the user (driver) and determines whether the driving load is less than a predetermined value (a seventh threshold). If the driving load is equal to or greater than the seventh threshold, the controller 40 temporarily suspends the process related to information provision until the driving load falls below the seventh threshold. On the other hand, if the driving load is less than the seventh threshold, the controller 40 executes the process of step S202.

[0108] In step S202, controller 40 acquires the user's gaze direction and the time the gaze is directed (gaze information), and determines whether the time the gaze is directed in the user's gaze direction (hereinafter also referred to as gaze time) is equal to or longer than a predetermined time (fourth threshold). If the gaze time is shorter than the fourth threshold, controller 40 returns to the process of step S201 and does not execute the processes of step S203 and subsequent steps until the conditions that the driving load is smaller than the seventh threshold and the gaze time is equal to or longer than the fourth threshold are met. On the other hand, if the gaze time is equal to or longer than the fourth threshold, controller 40 executes the process of step S203.

[0109] In step S203, the controller 40 identifies the visual recognition area based on the line-of-sight information, and identifies objects included in the visual recognition area based on the line-of-sight information, current location information on the map, and surrounding area information.

[0110] In step S204, the controller 40 determines whether or not the user has uttered a specific keyword. In addition to the above-mentioned example ("What's that?"), the specific keyword can also be set to, for example, words such as "What's that?" or "Hmm?". If it is detected that the user has uttered a specific keyword, the controller 40 executes the process of step S214. On the other hand, if it is not detected that the user has uttered a specific keyword, the controller 40 executes the process of step S205.

[0111] In step S214, the controller 40 specifies the central object as the target object, and executes the process of step S208.

[0112] If it is not detected that the user has uttered a specific keyword, in step S205, the controller 40 estimates the user's attention level for each object included in the visual recognition area. As described above, the closer an object is to the center of the visual recognition area, or the more the user prefers an object, the higher the attention level is estimated to be. Furthermore, if the time the user spends directing their gaze at the central object closest to the center of the visual recognition area exceeds a predetermined time (fifth threshold), and if voice by the user is detected while the user is directing their gaze at the central object, and if the voice feature value of the voice exceeds a predetermined value (sixth threshold), the attention level of the central object is estimated to be higher.

[0113] In step S206, the controller 40 determines whether the attention level (maximum attention level) of the object with the highest attention level is higher than a predetermined threshold (third threshold). The third threshold can be set to a value that exceeds the third threshold, for example, when there is a possibility that the user desires at least some information about the object. If the attention level of the object with the highest attention level is higher than the third threshold, the controller 40 executes the process of step S207. On the other hand, if the attention levels of all objects are equal to or lower than the third threshold, the controller 40 returns to the process of step S201.

[0114] In step S207, the controller 40 identifies the object with the highest attention level as the attention object.

[0115] When the target object of interest is identified, controller 40 estimates a certainty factor in step S208. As described above, the certainty factor is estimated based on information about the objects included in the visual recognition area and user information, and the certainty factor is estimated to be lower the more objects included in the visual recognition area, lower the estimated certainty factor is when more of the objects included in the visual recognition area are highly preferred by the user, and higher the estimated certainty factor is when the voice feature value when the user speaks is larger.

[0116] When the confidence level is estimated, the controller 40 outputs information (first information) in an output format according to the confidence level in step S209. For example, if the confidence level is low, the voice data and image data are output with characteristics that express a lack of confidence. On the other hand, if the confidence level is high, the voice data and image data are output with characteristics such as making the text or voice louder or giving the character a more dignified expression.

[0117] After outputting the first information, the controller 40 executes the processes shown in steps S104 to S106, S115, S116, and S125. These processes are the same as those in the first embodiment, and therefore will not be described further.

[0118] According to the information providing method of the second embodiment described above, the following effects can be obtained.

[0119] The information providing method of this embodiment identifies an object of interest, which is the object that the user pays the most attention to, among objects within a visible area that the user can see. Based on information about the objects within the visible area and user information (preference information about the user's preferences and user behavior information about the user's behavior), the method estimates a certainty factor, which is the degree of accuracy of identifying the object of interest. The method then determines the object of interest as a topic, determines information about the object of interest as first information to be provided to the user, and outputs the first information in an output format corresponding to the degree of certainty. Because the first information is output in an output format corresponding to the degree of certainty, it is possible to mitigate a negative impression on the user even if information that the user is not interested in is provided.

[0120] The information providing method of this embodiment estimates the user's level of attention to each object in the visual field based on preference information and user behavior information including the user's gaze direction, voice features when the user speaks, and the driving load of the vehicle driven by the user. Information about the object of interest that has received the highest level of attention from the user is then determined as the first information to be provided to the user. Because the user's level of attention to each object in the visual field is estimated from the preference information and the user behavior information in this way, it is possible to determine which object information should be provided to the user even when multiple objects are included in the visual field.

[0121] Furthermore, since the user's attention level is estimated based not only on the user's line of sight and speech (voice characteristics) but also on the user's driving load, it is possible to prevent the attention level from being erroneously estimated to be high in situations where the user wants to concentrate on driving, thereby further improving the accuracy of the attention level estimation.

[0122] In the information provision method of this embodiment, the user's attention level to an object is estimated higher for objects closer to the center of the visual field, and higher for objects with higher user preference. Furthermore, the user's attention level to a central object, which is the object closest to the center of the visual field, is estimated higher if the user's gaze is directed toward the central object for a predetermined period of time (a fifth threshold), or if the user's voice is detected while the user is directing their gaze toward the central object and the voice feature of the voice exceeds a predetermined value (a sixth threshold). In this way, rather than simply estimating the central object as the highest level of attention, the user's attention level to the object is estimated taking into account preference information, the time spent directing their gaze toward the central object, voice features, and the like. This further improves the accuracy of attention level estimation.

[0123] The information providing method of this embodiment estimates the user's attention level to an object as low when the driving load of the user while driving a vehicle is equal to or greater than a predetermined value. This prevents the user from erroneously estimating the attention level to the object as high and providing information about the object to the user when the user wants to concentrate on driving.

[0124] In the information providing method of this embodiment, an object that has the highest level of user attention and whose level of attention is higher than a predetermined third threshold is identified as the object of attention. By setting a threshold (third threshold) for identifying an object as an object of attention in this manner, it is possible to prevent, for example, an object that has simply come into the user's field of view from being identified as the object of attention. In other words, it is possible to prevent the user from being annoyed by providing information that the user does not particularly want.

[0125] In the information providing method of this embodiment, the confidence level is estimated to be lower as the number of objects in the visual recognition area increases, lower as the objects in the gaze area include more objects that the user has a high preference for, and higher as the voice feature value when the user speaks increases. Because the confidence level is estimated quantitatively in this way, it is easy to determine what form should be used to provide information to the user.

[0126] In this embodiment, the degree of certainty is estimated based on the information on the object and the user information, and the first information is output in an output format corresponding to the degree of certainty. However, for example, a timing determination unit may be provided that estimates whether the timing of providing the information was appropriate, and the information may be output in an output format corresponding to the appropriateness of the timing of providing the information. In this case, whether the timing is appropriate can be determined based on the user's driving situation (driving load, etc.) and data on the user's reaction when information was provided in the past.

[0127] Furthermore, information may be output in an output format according to the newness of the information provided to the user (freshness of the information), the degree of reliability of the information source, etc. In other words, the output format may be determined based on a comprehensive assessment of the information, rather than just the certainty level as in this embodiment.

[0128] Furthermore, in all of the embodiments, the information determination unit 48 acquires the information to be provided to the user from the information storage database 46, but this is not limited to this, and the information to be provided to the user may also be acquired from an external database.

[0129] Furthermore, the processing procedures shown in Figures 2 and 4 are examples for realizing this embodiment, and the order of some of the processing procedures may be changed, some of the processing procedures may be omitted, or other processing procedures may be added, within the scope that allows this embodiment to be realized.

[0130] Although the embodiments of the present invention have been described above, the above embodiments merely illustrate some of the application examples of the present invention, and it is not intended that the technical scope of the present invention be limited to the specific configurations of the above embodiments.

[0131] Although the above-described embodiments have been described as separate embodiments, they may be combined as appropriate.

[0132] This application claims priority based on Japanese Patent Application No. 2022-078464, filed with the Japan Patent Office on May 11, 2022, the entire contents of which are incorporated herein by reference.

Claims

1. 1. An information providing method for providing information to a user who is an occupant of a vehicle by using an output device provided in the vehicle, comprising: determining a topic and first information related to the topic to provide to the user; causing the output device to output the first information; detecting a reaction of the user when the output device outputs the first information; Estimating the user's level of interest in the topic based on the user's response; When the degree of interest is higher than a predetermined first threshold, the output device is caused to output second information, which is information related to the topic and is more detailed than the first information. Information provision method.

2. The information providing method according to claim 1, The user's response includes a motion of the user; estimating a degree of interest of the user in the first information based on a behavior of the user when the output device outputs the first information; Information provision method.

3. 3. The information providing method according to claim 1 or 2, the output device includes a display unit for outputting an image and an audio output device for outputting an audio; The first information is information that can be output at least by voice, Dividing the first information into a first part and a second part; displaying at least one of a first character and a second character on the display unit; outputting the first part to the audio output device as an utterance by the first character; outputting the second part to the audio output device as an utterance by the second character; Information provision method.

4. The information providing method according to claim 3, when it is detected that the user has performed an action of visually recognizing at least one of the first character and the second character when the output device outputs the first information, the interest level is estimated to be high. Information provision method.

5. The information providing method according to claim 4, The second information is information that can be output at least by voice, causing the audio output device to output the second information as an utterance by the first character or the second character; Information provision method.

6. The information providing method according to claim 1, the output device includes a display unit for outputting an image and an audio output device for outputting an audio; The first information is information that can be output at least by voice, Dividing the first information into a first part and a second part; Displaying at least one character on the display unit; outputting the first part and the second part to the voice output device as speech by the character; When it is detected that the user has performed an action of viewing the character when the output device outputs the first information, the interest level is estimated to be high. Information provision method.

7. 3. The information providing method according to claim 1 or 2, the output device includes a display unit that outputs an image; the second information is information that can be outputted at least as an image, outputting the second information to the display unit as an image; Information provision method.

8. 3. The information providing method according to claim 1 or 2, The first information is information regarding the handling of equipment in the vehicle that can be used by the user. Information provision method.

9. The information providing method according to claim 8, when it is detected that a new device has been added to the vehicle, that a specific device in the vehicle has been updated, or that a specific device in the vehicle has not been used for a long time, the device that has been added, updated, or has not been used for a long time is determined to be a topic, and information regarding the handling of the device is determined to be the first information and output. Information provision method.

10. The information providing method according to claim 8, Detecting a behavior of the user before the first information is determined; When it is estimated from the detected behavior of the user that the user is confused about how to handle a specific device, the specific device is determined to be a topic of conversation, and information regarding how to handle the specific device is determined to be the first information and output. Information provision method.

11. 3. The information providing method according to claim 1 or 2, obtaining location information regarding a current location of the vehicle; acquiring feature information regarding features around the current position of the vehicle based on the position information and map information; determining the feature as the topic and the feature information as the first information, and outputting the information; Information provision method.

12. The information providing method according to claim 1, Detecting the user's gaze direction; Identifying a visual recognition area that is visible to the user and an object within the visual recognition area based on the detected gaze direction of the user; Detecting the behavior of the user, and estimating the degree of attention of the user to each of the objects in the visual recognition area based on user information including preference information regarding the preferences of the user stored in a database and user behavior information regarding the detected behavior of the user; Identifying an object of interest that is the object that has received the highest attention from the user; Estimating a confidence level, which is a degree of accuracy of identifying the target object of interest, based on information of the target object in the visual recognition area and the user information; determining the object of attention as a topic, determining information relating to the object of attention as first information, and outputting the first information in an output form corresponding to the degree of certainty; Information provision method.

13. The information providing method according to claim 12, The user behavior information includes a gaze direction of the user, a voice feature when the user speaks, and a driving load of the vehicle driven by the user. Information provision method.

14. The information providing method according to claim 12 or 13, The degree of attention of the user to the object is estimated to be higher for an object closer to a center of the visual recognition area and higher for an object having a higher preference of the user; the level of attention of the user to a central object, which is an object closest to the center of the visual recognition area, is estimated to be higher when the time during which the user directs his / her gaze to the central object exceeds a predetermined time, or when the user's voice is detected while the user directs his / her gaze to the central object and a voice feature value of the voice exceeds a predetermined value; Information provision method.

15. The information providing method according to claim 12 or 13, When the driving load when the user is driving the vehicle is equal to or greater than a predetermined value, Estimating the user's attention to the object to be low; Information provision method.

16. The information providing method according to claim 12 or 13, Identifying an object as an object of interest when the object has the highest degree of attention from the user and the degree of attention is higher than a predetermined third threshold value. Information provision method.

17. The information providing method according to claim 12 or 13, The certainty is estimated to be lower as the number of objects in the visual recognition area increases, lower as the objects in the gaze area include more objects that are highly preferred by the user, and higher as the voice feature value when the user speaks increases. Information provision method.

18. The information providing method according to claim 1 or 12, modifying the first information when the estimated degree of interest of the user in the first information is lower than a predetermined second threshold value that is lower than the first threshold value; Information provision method.

19. The information providing method according to claim 1 or 12, changing the first information when a state in which the estimated degree of interest of the user in the first information is lower than a predetermined second threshold value that is lower than the first threshold value continues for a predetermined period of time; Information provision method.

20. An information providing system that provides information to a user who is an occupant of a vehicle, an output device for outputting the information; an information determination unit that determines information to be provided to the user; an information output unit that causes the output device to output the information to be provided to the user determined by the information determination unit; a user state information acquisition device that detects a reaction of the user to the output information; an interest level estimation unit that estimates an interest level of the user in output information based on a reaction of the user acquired by the user state information acquisition device; Equipped with when the degree of interest of the user in the outputted first information is estimated to be higher than a predetermined first threshold, the information output unit causes the output device to output second information which is more detailed information regarding the first information; Information provision system.