Information provision methods and information provision systems

Abstract

Provided is an information provision method in which an output device provided to a vehicle is used to provide information to a user who is an occupant of the vehicle. The information provision method involves determining a topic to be provided to the user and first information pertaining to the topic, causing the output device to output the first information, detecting the user's reaction when the output device outputs the first information, and inferring the user's level of interest in the topic on the basis of the user's reaction. If the level of interest is higher than a prescribed first threshold, second information, which is more detailed than the first information, pertaining to the topic is caused to be output by the output device.

Description

Technical Field

[0001] The present invention relates to an information providing method and an information providing system.

Background Art

[0002] JP2018-134902A discloses an information presentation system that displays travel-related information by changing the form of a plurality of displays and makes a vehicle driver (user) aware of the travel-related information.

Summary of the Invention

[0003] By the way, usually in conversations between people, the speaker of a certain topic guesses the interest (degree of interest) of the other party, and when it is guessed that the interest is high, the speaker may add more detailed information about the topic and continue the conversation.

[0004] In the information presentation system described in JP2018-134902A, since the degree of interest of the user is not estimated, there is a possibility that the information desired by the user about a topic with a high degree of interest of the user cannot be provided. In addition, there is a possibility that unnecessary detailed information is provided about an object with a low degree of interest of the user, causing annoyance to the user.

[0005] The present invention is made in view of the above problems, and an object thereof is to provide an information providing method and an information providing system capable of providing information desired by a user.

[0006] According to one aspect of the present invention, there is provided an information providing method for providing information to a user who is a passenger of a vehicle using an output device provided in the vehicle. This information providing method determines a topic to be provided to the user and first information related to the topic, outputs the first information to the output device, detects the reaction of the user when the output device outputs the first information, and estimates the degree of interest of the user in the topic based on the reaction of the user. Then, when the degree of interest is higher than a predetermined first threshold value, the output device is caused to output second information which is more detailed information about the topic than the first information.

Brief Description of the Drawings

[0007] [Figure 1] Figure 1 is a block diagram showing a schematic configuration of an information provision system employing the information provision method according to the first embodiment of the present invention. [Figure 2] Figure 2 is a flowchart illustrating the information provision method according to the first embodiment. [Figure 3] Figure 3 is a block diagram showing the schematic configuration of an information provision system employing the information provision method according to the second embodiment. [Figure 4] Figure 4 is a flowchart illustrating the information provision method according to the second embodiment. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below with reference to the drawings and other figures.

[0009] (First Embodiment) Figure 1 is a block diagram showing the schematic configuration of an information provision system 100 employing the information provision method according to the first embodiment of the present invention. The information provision system 100 is mainly installed in vehicles, etc. In this embodiment, the vehicle on which the information provision system 100 is installed is simply referred to as a vehicle. The user of the information provision system 100 is the occupant of the vehicle, i.e., the driver and / or passengers, but in this embodiment, for simplicity, the driver of the vehicle is referred to as the user.

[0010] The information provision system 100 includes a user status information acquisition device 10, a vehicle information acquisition device 20, an output device 30, and a controller 40.

[0011] The user status information acquisition device 10 is a group of devices that detect the status of the user (driver), and includes an in-vehicle camera, an in-vehicle microphone, a passenger sensor, an eye tracker, a body temperature sensor, a heart rate sensor, a sweat sensor, an electroencephalogram (EEG) sensor, etc. The in-vehicle camera is one or more cameras that capture images of the interior of the vehicle and obtain still images or videos, and in particular, it continuously or intermittently captures an area including the driver (user) to acquire images (including videos) for detecting the user's movements. Hereinafter, images acquired by the in-vehicle camera will be referred to as in-vehicle images. The in-vehicle microphone is one or more sound-collecting devices that acquire sounds or voices generated inside the vehicle. The in-vehicle microphone acquires at least the voices spoken by the user. The passenger sensor detects the presence or absence of passengers in the vehicle. The eye tracker detects the direction of the user's gaze. The body temperature sensor, heart rate sensor, sweat sensor, and EEG sensor detect the user's body temperature, heart rate, amount of sweat, and electroencephalogram, respectively. Information such as images, audio, passengers, user's gaze direction, user's body temperature, heart rate, sweating amount, and brain waves acquired by the user status information acquisition device 10 is transmitted to the controller 40, described later, as user status information.

[0012] The devices (sensors) included in the user status information acquisition device 10 are not limited to those described above. For example, user movements may be detected by an infrared sensor, and the user's gaze direction may be detected by an in-vehicle camera. Furthermore, the user status information acquisition device 10 only needs to include devices that can detect user movements, voice, and gaze direction, as described later in the controller 40, and other devices may be added as needed.

[0013] The vehicle information acquisition device 20 is a group of devices that acquire information about the vehicle's current location and surrounding conditions, and includes a current location information acquisition device 21 and a surrounding conditions acquisition device 22.

[0014] The current location information acquisition device 21 is a device that acquires location information regarding the current position of the vehicle, and includes a GPS (Global Positioning System) receiver, a compass sensor, a distance sensor, a steering angle sensor, etc. The GPS receiver detects the vehicle's position using artificial satellites, and the compass sensor, distance sensor, and steering angle sensor detect the vehicle's orientation, the distance between the vehicle and surrounding objects, and the vehicle's steering angle (direction of travel), respectively. The current location information acquisition device 21 acquires the vehicle's location information by combining the detection information from the GPS receiver, compass sensor, distance sensor, steering angle sensor, etc. Note that the devices (sensors) included in the current location information acquisition device 21 are not limited to those described above, and any device that can acquire location information regarding the vehicle's current position may be used. For example, location information may be acquired using a GNN (Global Navigation Satellite System) sensor, or location information may be acquired from the GPS receiver alone. The location information acquired by the current location information acquisition device 21 is transmitted to the controller 40 described later.

[0015] The surrounding conditions acquisition device 22 includes an external camera. The external camera is one or more cameras that capture images of the outside of the vehicle and obtain still images or videos. By continuously or intermittently capturing images of the area around the vehicle, it acquires images (including videos) for detecting the conditions around the vehicle. The images of the outside of the vehicle acquired by the surrounding conditions acquisition device 22 are transmitted as surrounding information to the controller 40, which will be described later. The surrounding conditions acquisition device 22 may also include a front distance sensor and a rear distance sensor, etc., which acquire distance information between the vehicle and the vehicle in front and behind.

[0016] The output device 30 includes an audio output device 31 that outputs sound and a display unit 32 that outputs images.

[0017] The audio output device 31 is a device that outputs audio data to the user, and includes, for example, a speaker capable of outputting sound. The audio output device 31 outputs the audio data received from the controller 40 (described later) as audio for the user.

[0018] The display unit 32 consists of a device that outputs image data to the user, and is, for example, a display capable of outputting images. The display unit 32 outputs image data received from the controller 40 (described later) for the user and displays images (including videos).

[0019] The controller 40 is a unit that controls the information provision system 100, determines the information to be provided to the user, and outputs that information to the output device 30.

[0020] The controller 40 consists of a computer equipped with a central processing unit (CPU), read-only memory (ROM), random access memory (RAM), and input / output interface (I / O interface), and performs processing to provide information to the user by executing a specific program.

[0021] The controller 40 includes a map information acquisition unit 41, a feature information acquisition unit 42, a function database 43, a confusion factor detection unit 44, a confusion estimation unit 45, an information storage database 46, an interest level estimation unit 47, and an information determination unit 48, as configurations for determining the information to be provided to the user. The controller 40 also includes a character database 49 and an information output unit 50, as configurations for outputting information to the output device 30.

[0022] The map information acquisition unit 41 identifies the vehicle's current location on the map. The map information acquisition unit 41 receives location information regarding the vehicle's current position from the current location information acquisition device 21. The map information acquisition unit 41 also acquires map information from an internal or external map database (not shown) of the controller 40, and identifies the vehicle's current location on the map by comparing the location information with the map information. The map information acquisition unit 41 transmits the information about the vehicle's current location on the map (current location information on the map) to the feature information acquisition unit 42. The map database is updated with the latest information at predetermined intervals.

[0023] The feature information acquisition unit 42 acquires feature information regarding features around the current position of the vehicle. Here, a feature refers to a characteristic facility, a historical building, or the like. The current position information on the map is input to the feature information acquisition unit 42 from the map information acquisition unit 41, and the surrounding information is input from the surrounding situation acquisition device 22. Further, the feature information acquisition unit 42 acquires map information from a map database (not shown) inside or outside the controller 40. Information on features at each location is registered in the map database, and the map information includes information on features. The feature information acquisition unit 42 acquires information regarding features visible to the user and information regarding other features around the vehicle (hereinafter, both are collectively referred to as feature information) from the current position information on the map, the surrounding information, and the map information. Further, the feature information acquisition unit 42 registers the feature information in the information storage database 46, and when there is a feature visible to the user, it transmits a signal (feature present signal) to that effect to the information determination unit 48 described later. The feature information acquired by the feature information acquisition unit 42 serves as a candidate for information to be provided to the user. Note that the above-described method for acquiring feature information is an example, and the method for acquiring feature information is not limited thereto.

[0024] The function database 43 is a database in which data regarding the functions of devices in the vehicle is stored. In the function database 43, information regarding the functions mounted on each device in the vehicle, information such as the usage frequency of each device and the functions mounted on each device is registered. The information in the function database 43 is updated at predetermined intervals.

[0025] The confusion factor detection unit 44 detects functions of the equipment in the vehicle that may confuse the user as confusion factors. Functions that confuse the user (confusion factors) here refer to functions that are recommended for use by the user but are not used because the user is unaware of their existence or does not know how to use them. Specifically, this refers to functions of equipment when new equipment is added to the vehicle, when certain equipment in the vehicle is updated, or when certain equipment in the vehicle has not been used for a long period of time or a long time (hereinafter simply referred to as "a long period of time"). Note that "a certain piece of equipment has not been used for a long period of time" includes cases where the equipment itself is used, but some of the functions installed in that equipment have not been used for a long period of time. The confusion factor detection unit 44 queries the function database 43 at predetermined intervals (periods) for the presence of new equipment additions, equipment updates, and equipment that has not been used for a long period of time. If the addition or update of equipment, or the detection that a specified piece of equipment has not been used for a long period of time, the confusion factor detection unit 44 transmits information regarding the functions of the added or updated equipment, or the functions of the equipment that has not been used for a long period of time, to the confusion factor estimation unit 45 as confusion factor information, and also registers it in the information storage database 46.

[0026] The confusion estimation unit 45 estimates whether the user is confused about the handling of a predetermined device (or function). The user state information is input to the confusion estimation unit 45 from the user state information acquisition device 10, and the confusion factor information is input from the confusion factor detection unit 44. The confusion estimation unit 45 estimates whether the user is confused about the handling of a predetermined device based on the detected user actions included in the user state information and the confusion factor information. Here, the user actions refer to the user's movements, the voices uttered by the user, changes in the user's line of sight direction, etc. For example, when it is detected from the user's actions captured by the in-vehicle camera that the user is trying to use a device equipped with a function that is a confusion factor, and it is detected that the user's method of handling the device is incorrect, it can be estimated that the user is confused. When it is estimated that the user is confused about the handling of a predetermined device, the confusion estimation unit 45 registers the information of the device as confusion device information in the information storage database 46. Note that the method for estimating whether the user is confused is not limited to the above method. For example, it may be estimated only from the user actions detected by the user state information acquisition device 10.

[0027] The information storage database 46 stores information that is a candidate for the information provided to the user. For example, the information storage database 46 stores information regarding the handling of various devices in the vehicle. In addition, the information storage database 46 stores the ground feature information from the ground feature information acquisition unit 42, the confusion factor information from the confusion factor detection unit 44, the confusion device information from the confusion estimation unit 45, etc. Note that the information storage database 46 is updated at regular intervals.

[0028] The interest degree estimation unit 47 estimates the user's interest degree in the information (first information) provided by the information providing system 100 to the user. The user state information is input to the interest degree estimation unit 47 from the user state information acquisition device 10, and the user's interest degree is estimated based on the user state information when the information (first information) is provided to the user. The estimated user interest degree is transmitted to the information determination unit 48. Note that the details of the interest degree estimation unit 47 will be described later.

[0029] The information determination unit 48 determines the topic and information to be provided to the user from the information storage database 46. Specifically, the information determination unit 48 queries the information storage database 46 for the presence or absence of information about troublesome devices. If troublesome device information is registered, it determines the device that the user is having trouble handling as the topic and determines that information about the device will be provided to the user as the information (first information). Furthermore, if the information determination unit 48 receives a signal from the feature information acquisition unit 42 indicating that there is a feature that the user can see (feature presence signal), it determines that the feature will be the topic and determines that information about the feature that the user can see from the feature information registered in the information storage database 46 will be provided to the user as the information (first information). In addition, the information determination unit 48 queries the information storage database 46 for the presence or absence of information about troublesome factors. If troublesome factors exist, it determines the device equipped with the function that is the troublesome factor as the topic and determines that information about the device will be provided to the user as the information (first information). Here, the first information provided to the user is, for example, an overview of the object that was determined to be the topic. When determining the topic and first information, there is no particular limit to which of the confusing devices, feature information, and confusing factors takes priority; however, in this embodiment, the order of priority is given to confusing devices, feature information, and confusing factors. If neither confusing device information nor confusing factor information is registered in the information storage database 46, and the information determination unit 48 has not received a feature presence signal, the topic and information related to the topic (first information) to be provided to the user will be randomly selected from the information stored in the information storage database 46.

[0030] Furthermore, the information determination unit 48 receives the user's level of interest from the interest estimation unit 47. When the information determination unit 48 provides the user with the first information, if the user's level of interest in the first information is higher than a predetermined threshold (first threshold), it decides to provide the user with more detailed information (second information, detailed information) about the subject of the discussion than the first information. The information decided to provide to the user is transmitted to the information output unit 50. If the user's level of interest in the first information is lower than the second threshold, which is set lower than the first threshold, the information determination unit 48 changes the topic and the first information.

[0031] The character database 49 stores character-specific information (character-specific information) such as the appearance, speech pattern, movements, and personality of the character, which the information output unit 50 (described later) will display on the display unit 32 of the output device 30. Multiple characters are registered in the character database 49, and character-specific information for each character is stored therein. Characters are not limited to human form; they may also be represented in the form of animals, plants, objects, or other symbols.

[0032] The information output unit 50 determines in what form the information (first information and second information) to be provided to the user, as determined by the information determination unit 48, will be output. In this embodiment, two characters are displayed on the display unit 32, and the first information and second information are output as a conversation between the characters. In this embodiment, the first information and second information are information that can be output as voice.

[0033] The information output unit 50 refers to character-specific information stored in the character database 49 and determines which character will speak and in what scenario to provide information to the user. Specifically, the information output unit 50 determines which character to display on the display unit 32 from the character database 49 and obtains the character-specific information for that character from the character database 49. The information output unit 50 also divides the first information into a first part, which is spoken by one character (the first character), and a second part, which is spoken by the other character (the second character). The image (including video) data of the determined character is then output (displayed) on the display unit 32, and the audio data of the first part and the audio data of the second part are output from the audio output device 31 in accordance with the movements of the character displayed on the display unit 32. In this way, the first information is provided to the user. By dividing the first information into two parts and having two characters speak it in this way, the first information can be provided to the user in the form of a conversation between the characters. Therefore, the user's responsibility to respond to utterances from the information provision system 100 is reduced, and the user's comfort level is improved.

[0034] When the first information is output via the output device 30, the information output unit 50 transmits a signal to the interest level estimation unit 47 indicating that the first information has been output.

[0035] Furthermore, the information output unit 50 determines which character will speak the second information (detailed information) to be provided to the user, as determined by the information determination unit 48. The output unit 50 then displays an image (including video) of the determined character on the display unit 32, and outputs the audio data of the second information from the audio output device 31 in accordance with the movements of the character displayed on the display unit 32. In this way, the second information is provided to the user.

[0036] Furthermore, the information output unit 50 outputs voice data from the voice output device 31 that is formatted to match the character's personality and other characteristics, based on the character-specific information of the character being used.

[0037] Furthermore, the first information is output as a conversation between the first and second characters, and the second information is output as an utterance by either the first or second character. However, the output format of the first and second information is not limited to these.

[0038] Furthermore, the method for determining the character to be displayed on the display unit 32 is not particularly limited; for example, a character with an appearance and personality that matches the topic being presented to the user can be selected.

[0039] Next, we will explain the details of the interest level estimation unit 47.

[0040] When the first information to be provided to the user is output via the output device 30, the information output unit 50 inputs a signal to the interest level estimation unit 47 indicating that the first information has been output. Upon receiving the signal indicating that the first information has been output, the interest level estimation unit 47 estimates the user's level of interest in the first information (hereinafter also simply referred to as the user's level of interest) based on the user's reaction when the first information was output. Here, the user's reaction is the user's state detected by the user state information acquisition device 10 when the output device 30 outputs the first information, and the interest level estimation unit 47 estimates the user's level of interest based on the user state information from the user state information acquisition device 10.

[0041] More specifically, user reactions include user actions, and the interest level estimation unit 47 estimates the user's level of interest in the first information based on the user actions included in the user state information. Here, user actions include user movements detected from in-car images, voices spoken by the user acquired by the in-car microphone, and movements of the user's gaze direction detected by the eye tracker. For example, when the output device 30 outputs the first information, if the user state information acquisition device 10 detects that the user has nodded in agreement, made a sound such as "ah," continued to look at the object being discussed for a predetermined time, or made eye contact with at least one of the first character and second character that spoke the first information, the interest level estimation unit 47 estimates the user's level of interest to be high. Specifically, for example, the eye tracker acquires user image information and detects the direction of the user's gaze on the user's face included in the image information, and determines whether the detected direction of the gaze matches (or nearly matches) a specific direction connecting the user's eyes and the output device 30 that displays the first and / or second characters. If the detected gaze direction is determined to match (or nearly match) a specific direction, it is determined that the user has looked at the first and / or second character, and the user's level of interest is estimated to be high. On the other hand, if the detected gaze direction is determined not to match (or nearly match) a specific direction, it is determined that the user has not looked at the first and / or second character. For face detection and gaze detection, known image recognition technologies can be used. In this way, if the user performs an action that indicates interest in the first information, the user's level of interest is estimated to be higher than a predetermined threshold (first threshold), and more detailed information (second information) about the object being discussed is provided to the user.

[0042] The above describes the main components of the information provision system 100. As described above, in the information provision system 100, information to be provided to the user is determined from the information storage database 46, and this information is provided to the user via the output device 30.

[0043] By the way, if we decide what information to provide to users without estimating their level of interest, there is a risk that we will not be able to provide the information that users want on topics that are of high interest to them. Also, there is a risk that we may provide unnecessarily detailed information on topics that are of low interest to users, causing them to feel annoyed.

[0044] On the other hand, it is difficult to estimate a user's level of interest before providing information, and there is a high risk of misestimating their level of interest.

[0045] In contrast, the information provision system 100 employing the information provision method of this embodiment determines the topic to be provided to the user and the first information related to that topic (for example, an overview of the subject of the topic), and estimates the user's level of interest in the first information based on the user's reaction when the first information is output. If the level of interest is higher than a first threshold, the system outputs second information, which is more detailed than the first information, related to that topic. In this way, since the second information, which is more detailed, is output only when the user's level of interest is high, the user can obtain detailed information only on topics that interest them. In other words, the system can provide the information the user wants. Furthermore, it prevents the system from providing detailed information on topics that the user is not interested in, thereby preventing the user from feeling annoyed. In addition, since the system estimates the user's level of interest based on the reaction to the first information, the accuracy of the interest estimation is improved compared to when the level of interest is estimated before providing the information.

[0046] Figure 2 is a flowchart illustrating the information provision method according to the first embodiment. The following controls are all repeatedly executed by the controller 40 at predetermined intervals. Furthermore, the following controls may be started when the information provision system 100 is started, or they may be started when some information has already been output to the user. The controller 40 continuously acquires user status information from the user status information acquisition device 10 and location information and surrounding information from the vehicle information acquisition device 20 at predetermined intervals.

[0047] In step S101, the controller 40 estimates whether the user is confused about how to use a given device or function. As mentioned above, the estimation of whether the user is confused is based on the detected user behavior included in the user state information. If the user behavior suggests that the user is confused about how to use the given device, the controller 40 executes the process in step S102. On the other hand, if the user is not estimated to be confused, the controller 40 executes the process in step S111.

[0048] In step S102, the controller 40 determines the device that the user is having trouble handling (the troublesome device) to be the topic of discussion and determines an overview of how to handle the device (the troublesome device) as the first information. Thus, if user confusion can be inferred from the user's behavior detected before determining the first information, the overview of how to handle the troublesome device is set as the first information.

[0049] In step S111, the controller 40 determines whether or not there are any features that the user can see. The presence or absence of features is determined based on the current location information on the map, surrounding information, and map information. If there are features that the user can see, the controller 40 executes the process in step S112. On the other hand, if there are no features that the user can see, the controller 40 executes the process in step S121.

[0050] In step S112, the controller 40 determines that the feature that the user can see will be the topic of discussion. It also queries the information storage database 46 for the feature information of the feature and determines the summary of the feature as the first information.

[0051] In step S121, the controller 40 determines whether or not there is a cause for confusion. As mentioned above, if new equipment is added to the vehicle, if a predetermined piece of equipment in the vehicle is updated, or if it is detected that a predetermined piece of equipment in the vehicle has not been used for a long time, it is determined that there is a cause for confusion. If there is a cause for confusion, the controller 40 executes the process in step S122. On the other hand, if there is no cause for confusion, the controller 40 executes the process in step S131.

[0052] In step S122, the controller 40 decides on the function of an added or updated device, or a function of a device that has not been used for a long time, and determines an outline of that function as the first information.

[0053] In step S131, the controller 40 randomly selects information from the information storage database 46, determines the object related to the selected information as the topic, and determines the outline of the object as the first piece of information.

[0054] When the first information is determined in step S102, S112, S122, or S131, the controller 40 causes the output device 30 to output the first information in step S103.

[0055] In step S104, the controller 40 estimates the user's level of interest in the topic. As mentioned above, the user's level of interest is estimated based on the user's reaction when the first information is output, particularly the user's actions. For example, if it is detected that the user has made an action to look at at least one of the first character and the second character, the level of interest is estimated to be high.

[0056] In step S105, the controller 40 determines whether the user's level of interest in the first information estimated in step S104 is higher than a predetermined first threshold. The first threshold can be set to a value such that the user's level of interest exceeds the first threshold when, for example, the output device 30 outputs the first information and it is detected that the user nodded in agreement, uttered a sound such as "ah," continued to look at the object being discussed for a predetermined time, or visually recognized at least one of the first character and the second character. If the user's level of interest in the first information is higher than the first threshold, the controller 40 executes the process in step S106. On the other hand, if the user's level of interest in the first information is less than or equal to the first threshold, the controller 40 executes the process in step S115.

[0057] In step S106, the controller 40 causes the output device 30 to output second information, which is more detailed information than the first information about the topic.

[0058] In this way, the controller 40 estimates the user's level of interest based on the user's actions when the first information is output (step S104), and outputs the second information if the level of interest is higher than the first threshold (steps S105, S106). Therefore, the user can obtain detailed information about the topic by performing a simple action, such as looking at a character. In other words, since there is no need to give any special instructions to obtain detailed information, the hassle of operation is reduced, and the user's comfort level is improved.

[0059] In step S105, if the user's level of interest in the first information is below the first threshold, the controller 40 determines in step S115 whether the user's level of interest in the first information estimated in step S104 is above a predetermined second threshold. The second threshold can be set so that, for example, when the first information is output, there is little change in the user's state, or the user utters a negative keyword (for example, "change the subject"). If the user's level of interest in the first information is above the second threshold, the controller 40 continues to output the first information in step S116. On the other hand, if the user's level of interest in the first information is below the second threshold, the controller 40 executes the process in step S125.

[0060] In step S125, the controller 40 changes the topic and the first information, and outputs the changed first information to the output device 30. The topic and the first information can be randomly selected from the information stored in the information storage database 46, similar to step S131. After outputting the changed first information in step S125, the controller 40 returns to the process in step S104 and estimates the user's level of interest in the changed topic.

[0061] Thus, if the user's level of interest in the first information is lower than the second threshold, the information provided to the user (the first information) is changed (steps S115, S125). This prevents the user from becoming annoyed by being continuously provided with information that is not of interest to them.

[0062] In this embodiment, for convenience, the steps are described as determining the topic and the first information separately. However, in reality, by providing the user with first information about a predetermined object, that object automatically becomes the topic. Therefore, it can be interpreted that the controller 40 determines the topic simultaneously by determining the first information.

[0063] According to the information provision method of the first embodiment described above, the following effects can be obtained.

[0064] In this embodiment, the information provision method estimates the user's level of interest in the first piece of information provided to the user based on the user's reaction to the output of the first piece of information related to the topic. If the level of interest is higher than a first threshold, it outputs second piece of information, which is more detailed information about the topic. This allows the user to obtain detailed information only on topics that interest them. In other words, it provides the user with the information they desire.

[0065] Furthermore, because the user's level of interest is estimated based on their response to the initial information, the accuracy of the interest estimation is improved compared to estimating interest before providing the information.

[0066] In this embodiment, the information provision method estimates the user's level of interest in the first information based on the user's actions when the first information is output. If the level of interest is higher than a first threshold, it outputs second information, which is more detailed information about the topic. As a result, the user can obtain detailed information about the topic with simple actions without giving any special instructions. Therefore, the user's comfort level is improved.

[0067] In this embodiment, the information provision method divides the first information into a first part and a second part, outputs the first part as speech by a first character displayed on the display unit 32, and outputs the second part as speech by a second character displayed on the display unit 32. By dividing the first information into two parts and having two characters speak in this way, the first information can be provided to the user in the form of a conversation between the characters. Therefore, the user's responsibility to respond to speech from the information provision system 100 is reduced, and the user's comfort level is improved.

[0068] In this embodiment, when outputting the first information, if it is detected that the user has performed an action of viewing at least one of the first character and the second character, the user's level of interest in the first information is estimated to be high. Therefore, by performing the simple action of viewing the characters, the level of interest can be raised above the first threshold, and detailed information (second information) about the topic can be obtained. In other words, since there is no need to give separate special instructions to obtain detailed information, the hassle of operation is reduced, and the user's comfort level is improved.

[0069] In this embodiment, the information provision method determines, as first information to be provided to the user, information regarding the handling of the added, updated, or unused equipment detected when it is detected that new equipment has been added to the vehicle, a predetermined piece of equipment in the vehicle has been updated, or a predetermined piece of equipment in the vehicle has not been used for a long period of time. This makes it possible to encourage users to use functions that are recommended for use by the user but are not being used because the user is unaware of their existence or does not know how to use them.

[0070] In this embodiment, the information provision method determines, as first information, information regarding the handling of a specific device when it is presumed from the user's behavior that the user is having difficulty handling that device. This makes it possible to provide the user with the information they need regarding the handling of the device when the user is having difficulty.

[0071] In this embodiment, the information provision method determines, if there is a feature that the user can see, to be the topic of discussion and determines that the feature information related to that feature will be provided to the user as first information. By discussing something that is within the user's field of vision in this way, the user can infer where the information being provided is coming from, thus providing the user with a sense of security.

[0072] In this embodiment, the information provision method modifies the first information if the user's level of interest in the first information is lower than a predetermined second threshold, which is lower than the first threshold. This prevents the user from becoming annoyed by being continuously provided with information that is not of interest to them.

[0073] In this embodiment, information is provided to the user by outputting audio and images, but this is not limited to this. For example, information may be provided by audio only.

[0074] Furthermore, in this embodiment, information is provided to the user through the speech of two characters displayed on the display unit 32, but this is not necessarily the case. For example, there may be one character to display, or there may be three or more. Also, for example, the appearance of the output device 30 may be used as the appearance of the character. In this case, instead of displaying the character on the display unit 32, the output device 30 itself becomes the character. Therefore, for example, first information can also be provided through a conversation between the characterized output device 30 and the character displayed on the display unit 32. Here, the characterized output device 30 includes, for example, an agent device which is a small robot installed on the dashboard of a vehicle. The agent device may be, for example, a robot modeled after an animal (e.g., a rabbit), a robot modeled after a virtual creature (e.g., the face of an anime character), or a robot modeled after another object (e.g., a television-type device, a radio-type device), but is not limited to these. Other devices equipped with a user interface such as a display unit (e.g., a smartphone, a tablet terminal, a car navigation system, IVI (In-Vehicle Infotainment)) may be used as agent devices. In this case, the robot described above can be displayed on the display unit, and the robot's operation can be changed by changing the display mode of the robot.

[0075] Furthermore, in this embodiment, the first information is changed when the user's level of interest in the first information is lower than the second threshold. However, the first information may also be changed if the user's level of interest remains below the second threshold for a predetermined period of time. This ensures that the first information is changed only after it becomes clearer that the user is not interested in the first information, thus preventing the first information from being changed when the user is interested in the currently provided first information.

[0076] Furthermore, in this embodiment, if there is a feature that the user can see, the feature is selected as the topic and the feature information related to that feature is selected as the first information, but this is not necessarily limited to this. If there is a feature around the vehicle, even if the user cannot see it, the feature around the vehicle may be selected as the topic and the feature information related to the feature selected as the topic may be selected as the first information.

[0077] Furthermore, in this embodiment, the first information to be provided to the user is randomly selected when there are no confusing devices, features that the user can see, or confusing factors, and when the user's level of interest in the first information is lower than the second threshold, but this is not necessarily limited to this. For example, instead of randomly selecting the first information, an importance level may be set for each piece of information, and the information with the highest importance may be prioritized and selected as the first information.

[0078] Furthermore, in this embodiment, the second information is output as speech from the first or second character, but this is not necessarily limited to this. For example, if the second information is information that can be output as an image, the second information may be displayed as an image on the display unit 32. This allows the second information to be provided to the user by an image displayed on the display unit 32, separately from the voice output device 31. In this case, the image may include not only photographs and pictures, but also text and the like.

[0079] Furthermore, in this embodiment, if there is a confusing factor, information about the equipment equipped with the confusing function is determined as the first information, and if there is a confusing device, information about that device is determined as the first information, but this is not necessarily limited to this. For example, even if there is no confusing factor or confusing device, the first information may be information about how to use any of the devices available to the user in the vehicle. This improves the user's understanding of the functions of the devices in the vehicle and allows for more effective use of the various devices in the vehicle.

[0080] (Second Embodiment) The information provision method of the second embodiment will be described with reference to Figures 3 and 4. Elements similar to those in the first embodiment are denoted by the same reference numerals, and their descriptions are omitted.

[0081] Figure 3 is a block diagram showing the schematic configuration of the information provision system 200 employing the information provision method according to the second embodiment. In the second embodiment, the method for determining the first information and the output format of the first information differ from those of the first embodiment.

[0082] As shown in Figure 3, the information provision system 200 includes a user status information acquisition device 10, a vehicle information acquisition device 20, an output device 30, and a controller 40.

[0083] The user status information acquisition device 10 and the output device 30 are the same as in the first embodiment, so their description will be omitted.

[0084] The vehicle information acquisition device 20 includes a current location information acquisition device 21, which includes a GPS receiver or the like, for acquiring vehicle location information, and a surrounding conditions acquisition device 22, which includes an external camera or the like, for acquiring surrounding information. In this embodiment, it also includes a load detection device 23.

[0085] The load detection device 23 includes a steering angle sensor for the steering wheel and an accelerator pedal position sensor, etc. The steering angle and accelerator pedal position detected by the load detection device 23 are transmitted to the controller 40 and used for estimating the driving load, which will be described later.

[0086] In addition to the configuration described in the first embodiment, the controller 40 includes a user behavior information acquisition unit 61, a preference information database 62, a viewing area extraction unit 63, an attention level estimation unit 64, an attention target object identification unit 65, a confidence level estimation unit 66, and an output feature generation unit 67. Note that in Figure 3, the feature information acquisition unit 42, the function database 43, the confusion factor detection unit 44, the confusion estimation unit 45, and the interest level estimation unit 47 are omitted from the description.

[0087] The user behavior information acquisition unit 61 includes a driving load detection unit 61A, a voice feature detection unit 61B, and a gaze detection unit 61C, and acquires user behavior information. The user behavior information acquisition unit 61 receives user status information from the user status information acquisition device 10, location information, surrounding information, steering angle and accelerator opening information from the vehicle information acquisition device 20, and current location information from the map information acquisition unit 41.

[0088] The driving load detection unit 61A estimates the driving load of the user (driver) from user status information, location information, surrounding information, steering angle and accelerator opening information, and current location information. The driving load is estimated, for example, according to the steering entropy method. Preferably, the driving load detection unit 61A acquires driver skill data based on current location information, surrounding information, time-series data of steering angle, time-series data of accelerator opening, etc., and uses this for driving load estimation. This can improve the accuracy of driving load estimation.

[0089] The voice feature detection unit 61B detects the voice features of the user's speech based on the voice spoken by the user acquired by the in-vehicle microphone. Here, voice features include the content of the speech (for example, whether a specific keyword is included), the volume of the voice, the pitch of the voice, the accent, etc. The voice feature detection unit 61B acquires voice feature quantities, which are numerical representations of these voice features from the user's speech. Whether or not the voice is from the user's speech can be determined, for example, by pre-memorizing the user's voice features.

[0090] The gaze detection unit 61C detects the time the user's gaze is directed in the direction of their gaze, based on the in-vehicle image acquired by the in-vehicle camera (user status information acquisition device 10) and the user's gaze direction detected by the eye tracker (user status information acquisition device 10). The gaze detection unit 61C acquires the user's gaze direction and the time the gaze is directed as gaze information. Note that the method of acquiring gaze information is not limited to the above. For example, gaze information may be acquired from the in-vehicle image alone.

[0091] The driving load information estimated by the driving load detection unit 61A, the voice feature quantity acquired by the voice feature detection unit 61B, and the gaze information acquired by the gaze detection unit 61C are transmitted as user behavior information to the attention estimation unit 64 and the confidence estimation unit 66, which will be described later. The gaze information acquired by the gaze detection unit 61C is also transmitted to the viewing area extraction unit 63.

[0092] The preference information database 62 is a database that stores user preference information. User preference information is obtained based on the user's past behavior, etc. For example, the more frequently a user visits a particular type of restaurant or facility, the more likely that the restaurant or facility is to be registered as being highly preferred by the user. However, the method of obtaining preference information is not limited to the method described above.

[0093] The visibility area extraction unit 63 identifies the visibility area visible to the user and the objects (targets) contained within that visibility area. The visibility area extraction unit 63 receives gaze information acquired by the gaze detection unit 61C, current location information on the map from the map information acquisition unit 41, and surrounding information from the surrounding situation acquisition device 22 (images of the outside of the vehicle taken by the external camera). The visibility area extraction unit 63 compares the user's gaze direction included in the gaze information with the current location information and surrounding information on the map to identify the visibility area visible to the user, and also identifies the objects contained within that visibility area based on the current location information and surrounding information on the map. However, if the time the user has been looking in the direction of their gaze is shorter than a predetermined time (fourth threshold), the visibility area extraction unit 63 does not identify the object, deeming it invisible. The predetermined time here can be set, for example, to a time when the user cannot recognize what the object in the direction of their gaze is. Note that the identification of the visibility area is not limited to the above method, but it is identified at least based on the user's gaze direction. The identified viewing area and object information are transmitted to the attention estimation unit 64 and the confidence estimation unit 66, which will be described later.

[0094] The attention estimation unit 64 estimates the user's level of attention to each object within the viewing area. The attention estimation unit 64 receives user behavior information from the user behavior information acquisition unit 61 and information on the viewing area and the objects contained within the viewing area from the viewing area extraction unit 63. The attention estimation unit 64 also queries the preference information database 62 to obtain user preference information regarding the objects within the viewing area. Based on the user behavior information and preference information (hereinafter referred to as user information), the attention estimation unit 64 estimates the user's level of attention to each object within the viewing area. Specifically, among the objects within the viewing area, objects closer to the center of the viewing area are estimated to have a higher level of user attention (points are added), and objects with a high user preference are estimated to have a higher level of user attention (points are added). If preference information for an object within the viewing area is not registered in the preference information database 62, no points are added to the level of attention. Furthermore, if the time the user spends looking at the central object closest to the center of the viewing area exceeds a predetermined time (5th threshold), the system detects the user's voice while the user is looking at the central object. If the voice features exceed a predetermined value (6th threshold), the system estimates (adds points to) the user's level of attention to the central object as higher. In this way, instead of simply estimating the highest level of attention to the central object, the system considers preference information, the time the user spends looking at the central object, voice features, etc., to estimate the user's level of attention to the object, thus improving the accuracy of the attention estimation.

[0095] Furthermore, if the user's driving load is high, the user's level of attention to the object is estimated to be low. Specifically, in this embodiment, if the user's driving load is above a predetermined value (the seventh threshold), the user's level of attention to all objects within the viewing area is estimated to be 0. That is, if the user's driving load is high, it is estimated that the user wants to concentrate on driving, so the level of attention is not estimated in practice, and the system waits until the driving load falls below the predetermined value before estimating the level of attention. Therefore, for example, if the user wants to concentrate on driving, it is prevented from mistakenly estimating the level of attention to objects within the viewing area to be high. In this way, the user's driving load is also taken into consideration in the estimation of the level of attention, and the user's level of attention to the object is estimated to be low when the driving load is high, thus improving the accuracy of the estimation of the level of attention.

[0096] The attention level information (attention level information) for each object estimated by the attention level estimation unit 64 is transmitted to the attention object identification unit 65 and the confidence level estimation unit 66.

[0097] The object of interest identification unit 65 identifies which object within the viewing area will be the object of interest. Here, the object of interest is the object that will be the topic of discussion provided to the user. The object of interest identification unit 65 receives attention level information from the attention level estimation unit 64. Based on the attention level information, the object of interest identification unit 65 identifies the object with the highest level of attention among the objects within the viewing area, and identifies that object as the object of interest if the user's level of attention to that object is higher than a predetermined threshold (third threshold). By setting a threshold (third threshold) for identifying an object of interest, it is possible to prevent, for example, identifying an object as the object of interest if it is not actually being noticed by the user but simply comes into view. Normally, the object of interest identification unit 65 identifies the central object closest to the center of the viewing area as the object of interest. In this embodiment in particular, if the user utters a specific keyword while looking at the central object, the central object is designated as the object of attention (the attention estimation unit 64 is set so that the attention level of the central object is highest when the specific keyword is uttered). For example, if the user utters a specific question word (keyword) such as "What is that?" while looking at a building which is the central object, the amount of voice features increases, and the central object (the building) is identified as the object of attention. In some cases, an object other than the central object may become the object of attention. For example, if there is a cat and a park which are symmetrical objects within the viewing area, the park is more likely to be in the center of the viewing area because it covers a larger area than the cat. However, if the preference information database 62 is registered as indicating that the user likes cats, the attention level to the cat will be estimated to be high, and the cat may be identified as the object of attention. The information on the object of attention identified by the object of attention identification unit 65 (object of attention information) is transmitted to the confidence estimation unit 66 and the information determination unit 48.

[0098] The confidence estimation unit 66 estimates the confidence level, which is the degree of correctness of identifying the object of interest. The confidence estimation unit 66 receives user behavior information from the user behavior information acquisition unit 61, information on the viewing area and objects contained within the viewing area from the viewing area extraction unit 63, attention level information from the attention level estimation unit 64, and information on objects of interest from the object of interest identification unit 65. The confidence estimation unit 66 also queries the preference information database 62 to obtain user preference information regarding objects within the viewing area. The confidence level is estimated based on the information on objects contained within the viewing area and user information (user behavior information and preference information). Specifically, the confidence level is lower the more objects contained within the viewing area there are, lower the confidence level is estimated the more objects within the viewing area that are highly to the user's preference, and higher the confidence level is estimated the larger the speech features when the user speaks. The more objects there are within the field of view, the more difficult it becomes to determine which object the user is actually focusing on, resulting in lower accuracy in identifying the object of focus. Conversely, if there are few objects within the field of view, there is a higher probability that the user is actually focusing on the identified object of focus, leading to a higher confidence level. Similarly, if the field of view contains many objects that the user highly prefers, it becomes difficult to determine which object the user is actually focusing on, resulting in lower accuracy in identifying the object of focus. Conversely, if there are few objects within the field of view that the user highly prefers, the accuracy of identifying the object of focus increases, leading to a higher confidence level. For example, if there is only one object within the field of view that the user highly prefers, that object will be identified as the object of focus, and there is a high probability that the user is actually focusing on that object. Also, if the audio features generated by the user's speech are substantial, there is a higher probability that the user is actually focusing on the identified object of focus, leading to a higher confidence level. For example, as given in the explanation of the object identification unit 65, if there is a building in the direction of the user's gaze and the user utters a specific question word (keyword) such as "What is that?", the amount of speech features will increase. In this case, there is a high probability that the user is actually focusing on the building (object of interest, central object), so the confidence level is estimated to be high.

[0099] As described above, the confidence level estimation unit 66 quantitatively estimates the confidence level, which is the degree of correctness of a particular object of interest. The confidence level information (confidence level information) estimated by the confidence level estimation unit 66 is transmitted to the information determination unit 48, the information output unit 50, and the output feature generation unit 67.

[0100] When the information on the object of interest and the confidence level information are input to the information determination unit 48, the information determination unit 48 determines the topic and information to be provided to the user, similar to the first embodiment. In this embodiment, for simplicity, it is assumed that there are no confusing devices, feature presence signals, or confusing factors. The information determination unit 48 determines the object of interest as the topic and determines the information (summary) related to the object of interest as the first information. Information related to the object of interest (including not only the summary (first information) but also detailed information (second information)) is obtained from the information storage database 46 if it is registered in the information storage database 46, or from an external database (not shown) if it is not registered. If there are confusing devices, feature presence signals, confusing factors, etc., the information determination unit 48 selects one of the following as the topic: confusing device, feature, device equipped with a function that is a confusing factor, or object of interest. In this case, the priority is not particularly limited, but for example, the priority is set in the order of confusing device, feature information, confusing factor, and object of interest. The topic and first information to be provided to the user, as determined by the information determination unit 48, are transmitted to the information output unit 50.

[0101] The output feature generation unit 67 receives confidence information from the confidence estimation unit 66. The output feature generation unit 67 is configured to send and receive information from the information output unit 50 and characterizes the output form of the first information determined by the information determination unit 48 according to the level of confidence. Specifically, when the first information is spoken using a character (including cases where the output device 30 itself becomes the character), the character's appearance, movements, voice, etc. are given features according to the level of confidence. Here, appearance features include the character's facial expression, size, color, shape, etc., and movement features include rotation, vibration, speed of movement, speed of movement, direction of gaze, etc. Also, voice features include intonation, volume, expression at the end of sentences, etc. When the level of confidence is low, these features are used to express a lack of confidence. For example, when the level of confidence is low, the voice data and image data can be characterized by making the character's facial expression look unsure, making the character smaller, or making the character's voice sound weak. To give one specific example, if a character is to utter first information about a cat, and the level of confidence is low, the voice and image data are characterized so that the character lowers its eyebrows and says in a weak voice, "By the way, there was a cat video that XX (user's name) likes... Huh? I thought I was watching a cat just now, but maybe I was wrong?" Also, if the first information is provided by displaying text on the display unit 32, the text is characterized according to the level of confidence. Here, the characteristics of the text include the thickness, size, font, color, and sentence endings. For example, if the level of confidence is low, the image (text) data is characterized to express a lack of confidence by making the text thinner or smaller, or by adding question words to the end of sentences.

[0102] The information output unit 50 outputs the audio data and image data of the first information, which have been characterized by the output feature generation unit 67, via the output device 30. This provides the user with the information.

[0103] As described above, in the information provision method of the second embodiment, the first information is output in an output format corresponding to the level of confidence. This makes it possible to mitigate the deterioration of the user's impression even if information that is of no interest to the user is provided.

[0104] Furthermore, because the degree of confidence is quantitatively estimated, it becomes easy to determine what form of information should be provided to the user. In other words, since the degree of confidence is quantified, it becomes easy to determine, for example, how definitive the language should be, and what volume or font size should be used.

[0105] Furthermore, if it is estimated from the user's reaction when the first information is output that the user's level of interest is low (for example, if the user's level of interest is lower than the second threshold in the first embodiment), a correction value acquisition unit may be provided to acquire necessary correction values ​​for the estimation of attention level and confidence level. This will make the estimation of attention level and confidence level more suitable for the user.

[0106] Figure 4 is a flowchart illustrating the information provision method according to the second embodiment. The following controls are all repeatedly executed by the controller 40 at predetermined intervals. Furthermore, the following controls may be started when the information provision system 100 is started, or they may be started when some information has already been output to the user. The controller 40 continuously acquires user status information from the user status information acquisition device 10, and location information, surrounding information, and steering angle and accelerator opening information from the vehicle information acquisition device 20 at predetermined intervals. Also, as mentioned above, for simplicity, in this embodiment, there are no confusion devices, feature signals, or confusion factors.

[0107] In step S201, the controller 40 estimates the user's (driver's) driving load and determines whether the driving load is less than a predetermined value (the seventh threshold). If the driving load is greater than or equal to the seventh threshold, the controller 40 temporarily suspends the information provision process until the driving load falls below the seventh threshold. On the other hand, if the driving load is less than the seventh threshold, the controller 40 executes the process in step S202.

[0108] In step S202, the controller 40 acquires the user's gaze direction and the time the gaze is directed there (gaze information), and determines whether the time the gaze is directed towards the user's gaze direction (hereinafter also referred to as gaze time) is equal to or greater than a predetermined time (fourth threshold). If the gaze time is shorter than the fourth threshold, the controller 40 returns to the process of step S201 and does not execute the processes from step S203 onward until the conditions are met that the operating load is less than the seventh threshold and the gaze time is equal to or greater than the fourth threshold. On the other hand, if the gaze time is equal to or greater than the fourth threshold, the controller 40 executes the process of step S203.

[0109] In step S203, the controller 40 identifies a viewing area based on the line of sight information, and identifies objects included within the viewing area based on the line of sight information, the current location information on the map, and surrounding information.

[0110] In step S204, the controller 40 determines whether the user has uttered a specific keyword. The specific keyword can be any word other than the example above ("What is that?"), such as "What is that?" or "Hmm?". If it is detected that the user has uttered a specific keyword, the controller 40 executes the process in step S214. On the other hand, if it is not detected that the user has uttered a specific keyword, the controller 40 executes the process in step S205.

[0111] In step S214, the controller 40 identifies the central object as the object of interest and executes the process in step S208.

[0112] If it is not detected that the user has uttered a specific keyword, in step S205, the controller 40 estimates the user's level of attention to each object included in the viewing area. As mentioned above, the closer an object is to the center of the viewing area, and the more the user prefers an object, the higher the estimated level of attention it receives. Furthermore, if the time the user has been looking at the central object closest to the center of the viewing area exceeds a predetermined time (fifth threshold), and if the controller detects the user's voice while the user is looking at the central object, and the voice features of that voice exceed a predetermined value (sixth threshold), the central object is estimated to receive an even higher level of attention.

[0113] In step S206, the controller 40 determines whether the level of attention (maximum level of attention) of the object with the highest level of attention is higher than a predetermined threshold (third threshold). The third threshold can be set to a value such that the level of attention exceeds the third threshold if, for example, the user may have even a slight desire for information about that object. If the level of attention of the object with the highest level of attention is higher than the third threshold, the controller 40 executes the process in step S207. On the other hand, if the level of attention of all objects is below the third threshold, the controller 40 returns to the process in step S201.

[0114] In step S207, the controller 40 identifies the object of the highest level of attention as the object of focus.

[0115] Once the object of interest is identified, the controller 40 estimates the confidence level in step S208. As mentioned above, the confidence level is estimated based on information about the objects included in the viewing area and user information. The greater the number of objects included in the viewing area, the lower the confidence level is estimated. The greater the number of objects included in the viewing area, the more objects that the user is likely to like, the lower the confidence level is estimated. The greater the amount of speech features generated when the user speaks, the higher the confidence level is estimated.

[0116] Once the confidence level is estimated, the controller 40 outputs information (first information) in step S209 in an output format corresponding to the magnitude of the confidence level. For example, if the confidence level is low, the audio data and image data are characterized and output to express a lack of confidence. On the other hand, if the confidence level is high, the audio data and image data are characterized in ways such as making the text or voice louder, or giving the character a confident expression, and then output.

[0117] Upon outputting the first information, the controller 40 executes the processes shown in steps S104 to S106 and S115, S116, and S125. These processes are the same as those in the first embodiment, so their description is omitted.

[0118] According to the information provision method of the second embodiment described above, the following effects can be obtained.

[0119] The information provision method of this embodiment identifies the object of interest that attracts the most attention from among the objects visible to the user within the user's visible area, and estimates the degree of confidence, which is the degree of correctness of identifying the object of interest, based on information about the objects within the visible area and user information (preference information regarding the user's preferences and user behavior information regarding the user's actions). Then, the object of interest is decided to be the topic, and information about the object of interest is decided to be provided to the user as first information, and this first information is output in an output format according to the level of confidence. In this way, because the first information is output in an output format according to the level of confidence, even if information that is of no interest to the user is provided, it is possible to mitigate the deterioration of the user's impression.

[0120] The information provision method of this embodiment estimates the degree of attention the user is paying to each object within the field of view based on preference information and user behavior information, including the direction of the user's gaze, speech features when the user speaks, and the driving load of the vehicle the user is driving. Then, the object that the user is paying the most attention to is determined to be the first information to be provided to the user. In this way, since the degree of attention the user is paying to each object within the field of view is estimated from preference information and user behavior information, it is possible to determine which object's information should be provided to the user even when multiple objects are included in the field of view.

[0121] Furthermore, because the system estimates the user's level of attention not only based on the user's gaze direction and speech (voice characteristics) but also on the user's driving load, it prevents the system from incorrectly overestimating the level of attention when the user wants to concentrate on driving. Consequently, the accuracy of the attention level estimation is further improved.

[0122] In this embodiment, the information provision method estimates the user's level of attention to an object higher for objects closer to the center of the viewing area, and higher for objects that the user has a high preference for. Furthermore, the user's level of attention to the central object, which is the object closest to the center of the viewing area, is estimated to be higher if the time the user gazes at the central object exceeds a predetermined time (5th threshold), and if the user's voice is detected while the user is gazing at the central object and the voice features exceed a predetermined value (6th threshold). In this way, instead of simply estimating the level of attention to the central object as the highest, the system estimates the user's level of attention to an object by considering preference information, the time the user gazes at the central object, voice features, etc. This improves the accuracy of the attention estimation.

[0123] In this embodiment, the information provision method estimates the user's level of attention to an object to be low if the user's driving load while driving the vehicle exceeds a predetermined value. This prevents the system from mistakenly estimating the user's level of attention to an object to be high and providing the user with information about that object when the user wants to concentrate on driving.

[0124] The information provision method of this embodiment identifies an object as a focus object if it is the object that attracts the most attention from the user and the level of attention is higher than a predetermined third threshold. By setting a threshold (third threshold) for identifying an object as a focus object in this way, it is possible to prevent situations such as identifying an object as a focus object when the user is not paying attention to it, but simply because it comes into their field of vision. In other words, it is possible to prevent providing information that the user does not particularly want, thereby preventing the user from feeling annoyed.

[0125] In this embodiment, the confidence level of the information provision method decreases as the number of objects within the viewing area increases. Visibility The confidence level is estimated lower the more objects within the domain that are highly appealing to the user, and higher the confidence level is estimated the greater the amount of speech features generated when the user speaks. Because the confidence level is estimated quantitatively in this way, it is easy to determine what form of information should be provided to the user.

[0126] In this embodiment, the degree of confidence is estimated based on the object information and user information, and the first information is output in an output format corresponding to the level of confidence. However, for example, a timing determination unit may be provided to estimate whether the timing of information provision was appropriate, and the information may be output in an output format corresponding to the degree of appropriateness of the timing of information provision. In this case, whether the timing is appropriate can be determined from the user's operating conditions (operating load, etc.) or data on the user's reaction when information was provided in the past.

[0127] Furthermore, the information may be output in a format that corresponds to the recency of the information provided to the user (information freshness) and the degree of reliability of the information source. In other words, the output format may be determined not only based on the confidence level of this embodiment, but also after comprehensively evaluating the information.

[0128] Furthermore, in all embodiments, the information determination unit 48 acquires information to be provided to the user from the information storage database 46, but is not limited to this, and the information to be provided to the user may be acquired from an external database.

[0129] Furthermore, the processing steps shown in Figures 2 and 4 are merely examples of how to implement this embodiment. The order of some of the processing steps may be rearranged to the extent that this embodiment can be implemented, and some of the processing steps may be omitted or other processing steps may be added.

[0130] Although embodiments of the present invention have been described above, these embodiments only represent a part of the application examples of the present invention, and are not intended to limit the technical scope of the present invention to the specific configurations of the above embodiments.

[0131] Although each of the embodiments described above has been explained as a standalone embodiment, they may be combined as appropriate.

[0132] This application claims priority based on Japanese Patent Application No. 2022-078464, filed with the Japan Patent Office on May 11, 2022, and all contents of that application are incorporated herein by reference.

Claims

1. An information provision method that provides information to a user, who is an occupant of a vehicle, using an output device installed in the vehicle, Determine the topic to be provided to the user and the first information relating to that topic. The output device is made to output the first information, The output device detects the user's reaction when it outputs the first information. Based on the user's response, the user's level of interest in the topic is estimated. If the level of interest is higher than a predetermined first threshold, the output device is instructed to output second information, which is more detailed information than the first information, relating to the topic. The direction of the user's gaze is detected, Based on the detected direction of the user's gaze, the system identifies the field of view visible to the user and the objects within that field of view. The system detects the user's actions and estimates the user's level of attention to each object within the viewing area based on user information including preference information about the user's preferences stored in the database and user behavior information about the detected user's actions. Identify the object of interest that is the object of the user's highest level of attention, Based on the information of the object within the viewing area and the user information, the confidence level, which is the degree of correctness of the specific object of interest, is estimated. The object of interest is determined to be the topic, the information relating to the object of interest is determined to be the first information, and the first information is output in an output format corresponding to the level of confidence. Information provision method.

2. The information provision method according to claim 1, The user's response includes the user's actions, Based on the user's actions when the output device outputs the first information, the degree of the user's interest in the first information is estimated. Information provision method.

3. A method for providing information according to claim 1 or 2, The output device includes a display unit that outputs an image and an audio output device that outputs sound. The first information is information that can be output at least by sound, The aforementioned first information is divided into a first part and a second part, The display unit is made to display at least one of the first character and the second character. The first part is output to the audio output device as speech by the first character. The second part is output to the audio output device as speech by the second character. Information provision method.

4. The information provision method described in claim 3, When the output device outputs the first information, if it is detected that the user has performed an action of viewing at least one of the first character and the second character, the level of interest is estimated to be high. Information provision method.

5. The information provision method described in claim 4, The second information is information that can be output at least by sound, The second information is output to the voice output device as speech by the first character or the second character. Information provision method.

6. The information provision method according to claim 1, The output device includes a display unit that outputs an image and an audio output device that outputs sound. The first information is information that can be output at least by sound, The aforementioned first information is divided into a first part and a second part, The display unit is configured to display at least one character. The first part and the second part are output to the voice output device as speech by the character. When the output device outputs the first information, if it is detected that the user has performed an action to view the character, the level of interest is estimated to be high. Information provision method.

7. A method for providing information according to claim 1 or 2, The output device includes a display unit that outputs an image. The second information is information that can be output as an image, The second information is output to the display unit as an image. Information provision method.

8. A method for providing information according to claim 1 or 2, The first information is information relating to the operation of equipment in the vehicle that can be used by the user. Information provision method.

9. The information provision method according to claim 8, If it is detected that new equipment has been added to the vehicle, that a predetermined piece of equipment in the vehicle has been updated, or that a predetermined piece of equipment in the vehicle has not been used for a long period of time, the system determines the equipment that has been added, updated, or not been used for a long period of time as the topic, and outputs information regarding the handling of that equipment as the first information. Information provision method.

10. The information provision method according to claim 8, Before determining the first information, the user's actions are detected. If, based on the detected user behavior, it is inferred that the user is having difficulty handling a specific device, the specific device is selected as the topic of discussion, and information regarding the handling of the specific device is selected and output as the first information. Information provision method.

11. A method for providing information according to claim 1 or 2, Location information regarding the current position of the aforementioned vehicle is obtained, Based on the aforementioned location information and map information, feature information regarding features around the vehicle's current location is obtained. The aforementioned feature is selected as the topic, and the aforementioned feature information is selected as the first information and output. Information provision method.

12. The information provision method according to claim 1, The user behavior information includes the direction of the user's gaze, the speech characteristics when the user speaks, and the driving load of the vehicle driven by the user. Information provision method.

13. A method for providing information according to claim 1 or 12, The user's level of attention to the object is estimated to be higher for objects closer to the center of the viewing area, and higher for objects that the user finds appealing. The degree of attention the user is paying to the central object, which is the object closest to the center in the viewing area, is estimated to be higher if the time the user directs their gaze towards the central object exceeds a predetermined time, and if the user's voice is detected while the user is directing their gaze towards the central object and the amount of voice features of that voice exceeds a predetermined value. Information provision method.

14. A method for providing information according to claim 1 or 12, If the driving load while the user is driving the vehicle exceeds a predetermined value, To estimate the user's level of attention to the object to be low, Information provision method.

15. A method for providing information according to claim 1 or 12, The object that attracts the most attention from the user is identified as the object of interest if that attention level is higher than a predetermined third threshold. Information provision method.

16. A method for providing information according to claim 1 or 12, The aforementioned confidence level is estimated to be lower the more objects there are in the viewing area, lower the more objects in the viewing area are that are highly to the user's liking, and higher the greater the amount of speech features when the user speaks. Information provision method.

17. The information provision method according to claim 1, If the user's level of interest in the estimated first information is lower than a predetermined second threshold which is lower than the first threshold, the first information is modified. Information provision method.

18. The information provision method according to claim 1, If the user's level of interest in the estimated first information remains below a predetermined second threshold, which is less than the first threshold, for a predetermined period of time, the first information is changed. Information provision method.

19. An information provision system that provides information to users who are occupants of a vehicle, An output device that outputs the aforementioned information, An information determination unit that determines the information to be provided to the aforementioned user, An information output unit that causes the information to be provided to the user, determined by the information determination unit, to be output to the output device, A user status information acquisition device that detects the user's response to the outputted information, Based on the user's response acquired by the user status information acquisition device, the interest level estimation unit estimates the user's level of interest in the outputted information. Equipped with, If the information output unit estimates that the user's level of interest in the outputted first information is higher than a predetermined first threshold, it will cause the output device to output second information, which is more detailed information about the first information. The user status information acquisition device detects the direction of the user's gaze, The aforementioned information provision system further, A viewing area extraction unit identifies a viewing area that the user can see and an object within the viewing area based on the detected direction of the user's gaze. A user behavior information acquisition unit that detects user behavior information related to the user's actions, A focus estimation unit estimates the degree to which the user pays attention to each object within the viewing area, based on user information including preference information about the user's preferences stored in the database and detected user behavior information. A unit for identifying objects of interest that identify the object of interest that is of the highest interest to the user, A confidence estimation unit estimates the degree of certainty, which is the degree of correctness of the object of interest, based on the information of the object within the viewing area and the user information. Equipped with, The information determination unit determines the object of interest as the topic, and determines the information relating to the object of interest as the first information. The information output unit outputs the first information in an output format corresponding to the level of confidence. Information provision system.

Images