Information processing device

Abstract

If a user has questions or concerns regarding the guidance voice, such as not understanding the guidance voice or finding that certain information was missing, the device or system will be able to understand the situation and identify the cause of the user's questions regarding the guidance voice, so that this information can be used to improve the guidance voice. [Solution] The information processing device outputs guidance audio to the user using an audio output unit. The detection unit detects the user's response to the guidance audio. The extraction unit extracts the user's confirmation actions related to the guidance audio from the responses detected by the detection unit. The estimation unit estimates the matters the user wants to confirm based on the confirmation actions. The storage unit stores the information about the matters estimated by the estimation unit.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus.

Background Art

[0002] There is known a car navigation system that performs route guidance by voice information (see, for example, Patent Documents 1 and 2).

[0003] Patent Document 1 discloses a route guidance device including a guidance output unit that outputs guidance voice and a direction input unit as an input means. Patent Document 2 discloses a voice recognition device including a determination unit that determines a recognition vocabulary comprehension degree indicating how much the user understands a vocabulary that can be voice recognized from the acquired utterance content of the user.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] In a route guidance device as described in Patent Document 1, a user who determines that the direction to proceed is unclear even after listening to the guidance voice inputs the direction to the direction input unit and makes an inquiry to the system side to confirm whether the recognized direction is correct. However, in this case, it is difficult for the system side to grasp the reason why the user determines that the direction to proceed is unclear, and it is difficult to improve the guidance voice that is determined to be unclear by the user. <s

[0006] In a speech recognition device like the one described in Patent Document 2, a microphone collects audio representing the user's utterances, a speech recognition unit performs speech recognition processing on the collected audio, and determines the level of recognition vocabulary comprehension based on the number of timeouts counted during the speech recognition processing. The guidance is then changed according to the level of recognition vocabulary comprehension. However, while the system can determine whether or not the user was able to speak with the system when determining the level of recognition vocabulary comprehension, it is difficult to determine why the user was able to speak (or why they were not).

[0007] One of the problems that this invention aims to solve is to enable the device or system to understand the situation when a user has questions about the guidance voice because they did not understand it or there was something missing in the guidance voice, and to use this information to improve the guidance voice by enabling the device or system to understand the cause of the questions the user had about the guidance voice. [Means for solving the problem]

[0008] To solve the above problems, the invention described in claim 1 is characterized by comprising: an audio output unit that outputs guidance audio to be provided to the user; a detection unit that detects the user's response to the guidance audio; an extraction unit that extracts the user's confirmation actions regarding the guidance audio from the responses detected by the detection unit; an estimation unit that estimates matters that the user wants to confirm based on the confirmation actions; and a storage unit that stores information about the matters estimated by the estimation unit. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram of an information processing device according to one embodiment of the present invention. [Figure 2] Figure 1 is a flowchart illustrating the operation of the information processing device. [Figure 3] This is a schematic diagram of the information processing device in Example 2. [Figure 4]This is a schematic diagram of the information processing device in Example 3. [Modes for carrying out the invention]

[0010] The following describes an information processing device according to one embodiment of the present invention. In the information processing device of the present invention, the voice output unit outputs guidance voice to the user. The detection unit detects the user's response to the guidance voice. The extraction unit extracts the user's confirmation actions regarding the guidance voice from the responses detected by the detection unit. The estimation unit estimates the matters the user wants to confirm based on the confirmation actions. The storage unit stores information about the matters estimated by the estimation unit. In this way, the user's confirmation actions can be extracted by the extraction unit from the user's responses detected by the detection unit, so the information processing device can understand that the user needs some kind of confirmation regarding the guidance voice, that is, that a situation has occurred where the user wants to confirm something because the user did not understand the guidance voice or there was something missing in the guidance voice. Furthermore, since the estimation unit estimates the matters the user wants to confirm based on the confirmation actions and the storage unit stores information about the matters estimated by the estimation unit, the information about the estimated matters can be accumulated and utilized. For this reason, by analyzing the accumulated information, the device or system can understand the cause of the matters the user wants to confirm regarding the guidance voice, which can be used to improve the guidance voice.

[0011] Furthermore, the memory unit may be configured to classify and store information about matters estimated by the estimation unit according to predetermined conditions. By doing so, information about matters estimated by the estimation unit can be divided into categories such as "content of the guidance voice," "the voice of the guidance voice itself," "matters related to place names," "matters related to the speed of the guidance voice," and "matters related to the pronunciation of the guidance voice." This makes it easier to handle the information stored in the memory unit in the future, for example, when analyzing, by category, what kind of content a user needed to confirm for a particular guidance voice.

[0012] Furthermore, the information processing device may include a route search unit that searches for a route for a moving object, and the guidance voice may be voice that conveys the route of the moving object searched by the route search unit to the user. And the confirmation action does not have to be an action directed at the route search unit. In this way, when a problem arises such as the user not understanding the guidance voice or there being missing information in the guidance voice, it is possible to extract as a confirmation action when the user attempts to resolve the problem using means other than using the route search unit. Specifically, when a user driving a moving object pulls the vehicle over to the side of the road after hearing the guidance voice and searches for information using a device such as a smartphone, or when the driver of a moving object instructs a passenger to re-check the route using a mobile device, etc., when the user attempts to resolve the above problem using means other than using the route search unit, this action can be extracted as a confirmation action. For this reason, it becomes easier for the information processing device to understand when a situation has occurred where the user wants to confirm something about the guidance voice, and this can be used to analyze the cause of the situation where the user wants to confirm something about the guidance voice, and to improve the guidance voice.

[0013] Furthermore, the information processing device includes a path search unit that searches for the path of a moving object, and the guidance voice is a voice that conveys the path of the moving object searched by the path search unit to the user. The path search unit, voice output unit, detection unit, extraction unit, estimation unit, and storage unit may be provided in a terminal device that moves with the moving object. In this way, the information processing device can be configured with a single terminal device without using external devices such as a server.

[0014] Furthermore, the information processing device includes a path search unit that searches for the path of a moving object, and the guidance voice is a voice that conveys the path of the moving object searched by the path search unit to the user. The path search unit, voice output unit, extraction unit, estimation unit, and storage unit may be provided in a first terminal device that moves with the moving object. The detection unit may be provided in a second terminal device that moves with the moving object and can communicate with the first terminal device. By doing so, the load on each device can be reduced compared to an information processing device composed of a single terminal device.

[0015] Furthermore, the information processing device includes a path search unit that searches for the path of a moving object, and the guidance voice is a voice that conveys the path of the moving object searched by the path search unit to the user. The path search unit and the voice output unit may be provided in a first terminal device that moves with the moving object. The detection unit may be provided in a second terminal device that moves with the moving object and can communicate with the first terminal device. The extraction unit, estimation unit, and storage unit may be provided in a server device that can communicate with the first and second terminal devices. By doing so, the load on each device can be reduced compared to an information processing device composed of a single terminal device or an information processing device composed of a first terminal device and a second terminal device.

[0016] Furthermore, the second terminal device may include an information retrieval unit independent of the route search unit. In this case, the confirmation action may be any of the following: the user using the information retrieval unit to search for a route for a moving object; the user using the information retrieval unit to search for the coordinates of a specific point; or the user using the information retrieval unit to perform a keyword search for a specific word. By doing so, if a problem arises, such as the user not understanding the guidance voice or there being missing information in the guidance voice, the confirmation action can be extracted as the user attempting to resolve the problem by using the information retrieval unit independent of the route search unit. For this reason, even if, for example, the user attempts to resolve the problem by performing a route search using a terminal device other than the terminal device that initially performed the route search (e.g., a smartphone), it becomes less likely for the information processing device to overlook the fact that such a problem occurred, and it becomes possible to improve the content of the guidance voice by utilizing the information stored in the memory unit.

[0017] Furthermore, the confirmation action involves the user making a sound in response to the guidance voice, and the estimation unit may estimate the matters the user wants to confirm based on the voice the user makes. In this way, the information processing device can understand that the user needs to confirm something about the guidance voice based on the voice the user makes, and can estimate the matters the user wants to confirm based on the voice the user makes. For this reason, it becomes possible to analyze data for each user regarding aspects of the guidance voice that were difficult to understand, such as "the guidance voice was too fast to hear" or "the guidance voice was poorly pronounced and I couldn't hear it," and adjust the guidance voice for subsequent times to be the most suitable for each user.

[0018] Furthermore, the voice guidance does not need to require a response from the user. By doing so, the extraction unit can extract actions other than the user responding to the voice guidance as confirmation actions. For example, it can extract actions such as the user saying to themselves, "Where is (place name)?", "I don't know (place name)", "It was too fast, I couldn't hear it", or "The pronunciation was bad, I couldn't hear it", or the user spontaneously giving instructions to a passenger, such as "Look up (place name) on your smartphone", or "Check the route again on your smartphone", as confirmation actions, rather than speaking to the information processing device in response to a request for a response. Therefore, there is no need for the information processing device to deliberately prompt the user to speak while driving, and it is possible to prevent distracting the user's attention or causing them annoyance through such prompting.

[0019] Furthermore, an information processing method according to one embodiment of the present invention is an information processing method executed by a computer. The information processing method outputs guidance audio to be provided to the user in a voice output step. Then, in a detection step, the user's response to the guidance audio is detected. Then, in an extraction step, the user's confirmation actions regarding the guidance audio are extracted from the responses detected in the detection step. Then, in an estimation step, the matters that the user wants to confirm are estimated based on the confirmation actions. Then, in a storage step, information about the matters estimated in the estimation step is stored. In this way, since the user's confirmation actions are extracted in the extraction step from the user's responses detected in the detection step, it is possible to understand that confirmation actions have occurred. That is, it is possible to understand that the user needs to confirm the content of the guidance audio. In addition, since the estimation step estimates the matters that the user wants to confirm based on the confirmation actions, and the storage step stores information about the estimated matters, the information about the estimated matters can be accumulated and utilized. For this reason, by analyzing the accumulated information, the device or system can understand the cause of the matters that the user wants to confirm regarding the guidance audio, which can be used to improve the guidance audio.

[0020] Also, the above-described information processing method may be executed by a computer as an information processing program. By doing so, when a situation occurs where the user has something to confirm about the guidance voice, the computer can be used to grasp the situation. Further, in order to be able to utilize it for improving the guidance voice, the cause of the occurrence of the matter that the user wants to confirm about the guidance voice can be grasped.

[0021] Also, the above-described information processing program may be stored in a computer-readable storage medium. By doing so, the information processing program can be distributed not only when incorporated into a device but also alone, and version updates and the like can be easily performed.

Example

[0022] [Example 1] FIG. 1 is a schematic configuration diagram of an information processing apparatus 1 according to Example 1 of the present embodiment. The information processing apparatus 1 includes, for example, a terminal device 107 (first terminal device) that moves together with a vehicle (mobile object). The terminal device 10 is, for example, a navigation device installed in a vehicle, and includes a control unit 11, an input unit 12, an output unit 13 (voice output unit), and a storage unit 14.

[0023] The control unit 11 is composed of a CPU having a memory such as a RAM or a ROM, for example, and controls the entire terminal device 10. The control unit 11 includes a user identification unit 110, a route search unit 111, a detection unit 112, an extraction unit 113, and an estimation unit 114, and performs identification processing for identifying the user of the terminal device 10, route search processing for searching for the route of the vehicle, voice output processing for outputting information about the searched route as guidance voice to the user, and the like.

[0024] Furthermore, the control unit 11 monitors various devices installed in the vehicle and performs detection processing to detect responses to the user's voice guidance. The various devices monitored by the control unit 11 include, for example, GPS receivers, various sensors such as acceleration sensors and gyro sensors, in-vehicle cameras, and microphones.

[0025] Furthermore, the control unit 11 performs an extraction process to extract user confirmation actions related to guidance voice from the user's responses detected by the detection process, and also performs an estimation process to estimate the confirmation items that the user wants to confirm based on the extracted confirmation actions. The control unit 11 also performs a classification process to classify the confirmation items according to predetermined conditions, and a storage process to save the classified confirmation items in the storage unit 14. Details of the user's responses, user confirmation actions, and confirmation items will be explained later when describing the operation of the information processing device 1.

[0026] The input unit 12 is composed of, for example, a microphone, input buttons, or a touch panel. The input unit 12 receives user instructions through the user's voice or operation of the user's input buttons or touch panel, and transmits a signal indicating the instruction to the control unit 11.

[0027] The output unit 13 consists of a speaker that outputs guidance audio and a device such as an amplifier that drives the speaker. The output unit 13 outputs information about the vehicle route found by the route search process as guidance audio and transmits it to the user.

[0028] The memory unit 14 is composed of, for example, a hard disk or non-volatile memory, and stores programs and data for the control unit 11 to perform the above-mentioned control (for example, map data necessary for route search processing, audio data necessary for audio output processing, etc.), confirmation data 140 indicating confirmation actions (described later), estimation data 141 indicating confirmation items (described later), etc.

[0029] Next, an example of the operation (information processing method) of the information processing device 1 with the above configuration will be explained with reference to the flowchart in Figure 2. This information processing method can be executed as an information processing program on a computer equipped with a CPU, etc. This information processing program may also be stored on a computer-readable storage medium.

[0030] First, when the terminal device 10 is started up, the user identification unit 110 performs user identification processing (step S100). In this step, the user who will use the terminal device 10 is identified. At this time, the user may be automatically identified by matching the in-vehicle image taken by the in-vehicle camera with images of the vehicle driver and passengers that have been previously stored in the storage unit 14. Alternatively, the terminal device 10 may output a question to the vehicle driver and passengers via the output unit 13 and request the user to input the answer to the question into the input unit 12, and the user may be identified based on that answer. Furthermore, if a user terminal such as a smartphone owned by the user is pre-registered with the terminal device 10, the user may be identified based on the detection of the user terminal by the terminal device 10 via wireless means or by direct connection to the terminal device 10.

[0031] Next, the control unit 11 determines whether to issue an instruction for route search processing (step S200). In this step, the input unit 12 is first monitored to determine whether an instruction for route search processing has been input. Specifically, an instruction for route search processing is when the user inputs the departure point and destination into the input unit 12, which may be done by voice, such as "Find a route from XX (departure point) to YY (destination)," or by operating the touch panel or input buttons. If the result of the determination is YES, the system proceeds to the next step S300; if it is NO, the system waits until the instruction is input.

[0032] In step S300, the route search unit 111 performs route search processing. In this step, a route from the departure point to the destination, which is entered in the input unit 12, is searched, and guidance information for that route is created. The details of the route search processing are well known, so a detailed explanation is omitted, but the route from the departure point to the destination is searched using map data stored in the storage unit 14 and the vehicle's current position and current time received by the GPS receiver, and guidance information for the searched route is generated. Once the route search processing is completed, step S300 ends, and the process proceeds to step S400.

[0033] Step S400 corresponds to the voice output step in the embodiment described above. In this step, the control unit 11 performs voice output processing, and guidance voice is output from the output unit 13. Guidance voice is voice provided to the user, and in this case, it indicates the guidance information generated in step S300 described above. Through this guidance voice, the vehicle route searched by the route search unit is communicated to the user.

[0034] The voice guidance may explain the route to the user using road names, such as, "Proceed along XX Road (road name), enter the □□ Expressway (expressway name) from the ×× Interchange (interchange name), and exit at the △△ Interchange (interchange name)," or it may explain the route to the user using place names, such as, "Pass through ●● (place name) and head towards ◎◎ (place name)." In addition, the voice guidance may explain the route to the user using facility names, such as, "Turn right at the intersection before ◇◇ (facility name), and turn left at the traffic light after ◆◆ (facility name)," or it may explain the route to the user using a combination of road names, place names, and facility names.

[0035] The voice guidance may include a request for a response from the user, such as "If you did not understand the guidance, please say that you did not understand." However, from the perspective of preventing distractions to the driver and causing inconvenience to the user, it is preferable that the voice guidance does not request a response from the user. Once the voice guidance is output, step S400 ends and the system proceeds to step S500.

[0036] Step S500 is the detection step in the embodiment described above. In this step, the detection unit 112 performs the detection process. Here, first, devices such as a GPS receiver, various sensors such as an acceleration sensor and a gyro sensor, an in-vehicle camera, or a microphone are monitored. Then, through these devices, sounds, user actions, or the behavior of the vehicle operated by the user that occur after the output of the guidance voice are detected as user reactions. The detected reactions are then converted into reaction data and temporarily stored in the storage unit 14 in ring buffer format. Thus, in step S500, the detection unit 112 detects the user's reaction to the guidance voice, and that reaction is temporarily stored in the storage unit 14. Then, the process proceeds to step S600.

[0037] Step S600 corresponds to the extraction step in the embodiment described above. In this step, the extraction process is performed by the extraction unit 113. First, the storage unit 14 is monitored for a predetermined period of time. This predetermined period may be, for example, 1 to 3 minutes after the guidance voice is output. After this monitoring, it is determined whether or not data indicating the user's confirmation action regarding the guidance voice is found among the temporarily stored response data.

[0038] If data indicating a confirmation action is found, that data is extracted as confirmation data 140. In this way, the extraction unit 113 extracts user confirmation actions related to the guidance voice from the responses detected by the detection unit 112. If confirmation data 140 can be extracted (YES), the process proceeds to step S700; if confirmation data 140 cannot be extracted (NO), the operation of the terminal device 10 ends.

[0039] Here, the confirmation actions extracted as confirmation data 140 refer to actions taken by a user who has heard the guidance voice when they have something they want to confirm about that guidance voice. These actions indicate that the user did not understand the guidance voice for some reason, or that there was something missing in the guidance voice. For example, a confirmation action might be "the user stopping the vehicle on the shoulder of the road and operating a device such as a smartphone." This confirmation action is detected and extracted by the extraction unit 113 when the reaction data created by the detection unit 112 contains data indicating that the vehicle has stopped and video data of the user operating a device such as a smartphone.

[0040] Furthermore, confirmation behaviors include "talking to oneself" such as "Where is ●● (place name)?", "I don't really know what ●● (place name) is," "It was too fast to hear," or "The pronunciation was bad and I couldn't hear," as well as "giving instructions to a passenger" such as "Look up ●● (place name) on your smartphone" or "Check the route again on your smartphone." These confirmation behaviors are detected and extracted by the extraction unit 113 when the response data created by the detection unit 112 contains audio data indicating the aforementioned talking to oneself or giving instructions to a passenger.

[0041] Thus, confirmation actions include actions other than those performed by the route search unit 111, that is, actions other than those performed by the route search unit 111 that initially performed the route search, and such actions are also extracted by the extraction unit 113. Furthermore, as mentioned above, when a user makes a sound in response to the guidance voice, it is also extracted as a confirmation action, and this includes spontaneous actions other than responding to the guidance voice, such as "talking to oneself" or "giving instructions to a passenger."

[0042] Step S700 corresponds to the estimation step in the embodiment described above. In this step, estimation processing is performed by the estimation unit 114. Here, the items to be confirmed, which are the things the user wants to confirm, are estimated based on the confirmation data 140 (i.e., the extracted confirmation actions). For example, if the content of the confirmation data 140 is "the vehicle has been stopped on the shoulder of the road and the user is operating a device such as a smartphone," then the user has stopped driving and is operating a device such as a smartphone, so it is highly likely that they are doing something to look up about the guidance voice. On the other hand, if it is highly likely that they are doing something to look up, it is highly likely that they are able to hear the guidance voice. Therefore, when such confirmation data is extracted, the item to be confirmed is estimated to be "the content of the guidance voice."

[0043] This is also true when confirmation data 140 containing the content "giving instructions to a passenger" is extracted. Since instructions are being given to a passenger, it can be assumed that the driver can hear the voice guidance, but since the passenger is being asked to look up the route again using a separate device, such as a smartphone, it can be assumed that the driver does not understand what the voice guidance is saying. The same applies when confirmation data 140 containing the content "talking to oneself," such as "Where is ●● (place name)?" or "I don't really know ●● (place name)."

[0044] In response to these, if confirmation data 140 is extracted that includes "talking to oneself," such as "it was too fast to hear" or "the pronunciation was poor and I couldn't hear," it is highly likely that the user did not accurately hear the content of the guidance voice, or even the voice itself. Therefore, if confirmation data 140 of this nature is extracted, the item to be confirmed is presumed to be "the voice of the guidance voice itself." Thus, in step S700, the item to be confirmed may be presumed based on confirmation actions taken by the user, such as using other devices after listening to the guidance voice, or making a sound in response to the guidance voice.

[0045] Furthermore, in step S700, whether the item to be confirmed is the "content of the guidance voice" or the "voice of the guidance voice itself," it may be possible to identify the item to be confirmed based on the voice spoken by the user, in which case such identification will be performed. For example, if the content of confirmation data 140 is "talking to oneself" such as "Where is ●● (place name)?" or "I don't really know ●● (place name)," the item to be confirmed will be identified as "something related to place names." If the content of confirmation data 140 is "talking to oneself" such as "It was too fast to hear," the item to be confirmed will be identified as "something related to the speed of the guidance voice." If the content of confirmation data 140 is "talking to oneself" such as "The pronunciation was poor and I couldn't hear," the item to be confirmed will be identified as "something related to the pronunciation of the guidance voice."

[0046] Then, once these confirmation items are estimated or identified, the estimation unit 114 creates estimated data 141 as information about the confirmation items. This estimated data 141 indicates the confirmation items. This completes step S700, and the process proceeds to step S800.

[0047] Step S800 is the storage step in the embodiment described above. In this step, the control unit 11 performs classification and storage processing. Here, first, the estimated data 141 is classified according to predetermined conditions. In this case, for example, the estimated data 141 can be classified by determining the conditions to sort by the types of confirmation items described above ("content of the guidance voice", "the voice of the guidance voice itself", "things related to place names", "things related to the speed of the guidance voice", "things related to the pronunciation of the guidance voice"), and assigning a classification code to each type.

[0048] Then, once the classification of the estimated data 141 is complete, a saving process is performed to save the classified estimated data 141 to the storage unit 14. Unlike the temporary saving of the response data described above, in this saving process, the estimated data 141 is saved as data that cannot be overwritten. At this time, the estimated data 141 may be saved simply, but for example, data of the user identified by the user identification unit 110, instruction data entered into the input unit 12, guidance voice data output by the output unit 13, confirmation data 140 that formed the basis for creating the estimated data 141, and data of the current location and current time received by the GPS receiver may be added to the estimated data 141 before saving. In this way, the reasons why the guidance voice was not understood by the user can be analyzed from various perspectives. Then, the operation of the terminal device 10 ends.

[0049] According to Example 1, the detection unit 112 can detect the user's response, and the extraction unit 113 can extract the user's confirmation behavior. This allows the information processing device 1 to understand that the user needs some kind of confirmation regarding the guidance voice, that is, that the user did not understand the guidance voice, or that there was something missing in the guidance voice that the user wants to confirm. Furthermore, the estimation unit 114 estimates the confirmation items (items) that the user wants to confirm based on the confirmation behavior, and the storage unit can store the estimated data 141 (information about the confirmation items estimated by the estimation unit 114), so that confirmation items can be accumulated and utilized. Therefore, by analyzing the accumulated estimated data 141, the device or system can understand the cause of the user's need to confirm something regarding the guidance voice, which can be used to improve the guidance voice.

[0050] Furthermore, in this case, if the estimated data 141 is accompanied by the user data mentioned above, the instruction data entered into the input unit 12, the guidance voice data output by the output unit 13, the confirmation data 140 that formed the basis for creating the estimated data 141, and the current location and time data received by the GPS receiver, then, for example, it becomes possible to investigate the relationship between the guidance voice and the confirmation items for each user. This makes it possible to optimize the guidance voice to suit the characteristics of each user and to investigate the expressions of the guidance voice that many users find difficult to understand. In addition, since it becomes possible to investigate the relationship between user information, current location and time, and confirmation items, it also becomes possible to investigate locations and routes where the guidance voice is difficult for many users to understand.

[0051] Furthermore, the information about the confirmation items estimated by the estimation unit 114 can be divided into categories such as "content of the guidance voice," "the voice of the guidance voice itself," "things related to place names," "things related to the speed of the guidance voice," and "things related to the pronunciation of the guidance voice." This makes it easier to handle the information stored in the memory unit 14 in the future, for example, when analyzing, by category, what kind of content the user needed to confirm for a single guidance voice.

[0052] Furthermore, if a problem arises, such as the user not understanding the guidance voice or the guidance voice lacking certain information, the system can extract as a confirmation action if the user attempts to resolve the problem using means other than the route search unit 111. Specifically, if a user driving a vehicle pulls over to the side of the road after hearing the guidance voice and operates a device such as a smartphone, or if the driver instructs a passenger to re-check the route using a mobile device, the system can extract as a confirmation action if the user attempts to resolve the problem using means other than the route search unit 111. This makes it easier for the information processing device 1 to recognize when a situation arises where the user wants to confirm something about the guidance voice, which can contribute to analyzing the cause of the issue and improving the guidance voice.

[0053] Furthermore, since the information processing device 1 can be configured using a single terminal device 10 without the need for external devices such as server equipment, the configuration of the information processing device 1 can be simplified.

[0054] Furthermore, the information processing device 1 can understand from the voice spoken by the user that the user needs to confirm something about the guidance voice, and can estimate what needs to be confirmed based on the voice spoken by the user. For this reason, it becomes possible to analyze data for each user regarding aspects of the guidance voice that were difficult to understand, such as "the guidance voice was too fast to hear" or "the guidance voice was poorly pronounced and I couldn't hear it," and adjust the guidance voice for subsequent visits to be the most suitable for each user.

[0055] Furthermore, according to this embodiment, the extraction unit 113 can extract actions other than the user responding to the guidance voice as confirmation actions. For example, it can extract actions such as the user saying to themselves, "Where is ●● (place name)?", "I don't know ●● (place name)", "It was too fast to hear", or "The pronunciation was bad and I couldn't hear", or the user spontaneously giving instructions to a passenger, such as "Look up ●● (place name) on your smartphone", or "Check the route again on your smartphone", as confirmation actions, rather than speaking to the information processing device 1 in response to a request for a response. Therefore, there is no need for the information processing device 1 to deliberately guide the user to speak while driving, and it is possible to prevent the user from being distracted or feeling annoyed by such guidance.

[0056] [Example 2] Next, the information processing device 1 according to Embodiment 2 of this embodiment will be described with reference to Figure 3. Note that parts identical to those in Embodiment 1 described above are denoted by the same reference numerals, and their descriptions are omitted or simplified.

[0057] As shown in Figure 3, the information processing device 1 according to this embodiment includes a terminal device 10 (first terminal device) and a mobile terminal 20 (second terminal device).

[0058] The terminal device 10 is a device that moves with the vehicle, similar to the one in Embodiment 1, and has the same configuration as the terminal device 10 in Embodiment 1, except that it does not have a detection unit 112 and has a communication unit 15.

[0059] The mobile terminal 20 is a device that moves with the vehicle (mobile object) and is configured to communicate with the terminal device 10. This mobile terminal 20 is, for example, a smartphone or tablet, and comprises a control unit 21 and a communication unit 22.

[0060] The control unit 21 is composed of a CPU equipped with memory such as RAM or ROM, and controls the entire mobile terminal 20. The control unit 21 includes a detection unit 211 and an information retrieval unit 212. The detection unit 211 has a configuration equivalent to the detection unit 112 in Embodiment 1.

[0061] The information retrieval unit 212 is configured to perform information retrieval independently of the route search unit 111 of the terminal device 10. It is capable of performing route search processing (route search) to search for vehicle routes independently of the route search unit 111, location search processing (location search) to find the coordinates of a specific location, and keyword search processing (keyword search) to search for specific words.

[0062] The information retrieval unit 212 may also utilize a publicly known search engine used on the World Wide Web, and the search results may be output as images or audio by an output unit (not shown) provided on the mobile terminal 20.

[0063] The communication unit 22 is configured to communicate with the communication unit 15 of the terminal device 10, and the reaction data created by the detection unit 211 can be transmitted from the mobile terminal 20 to the terminal device 10.

[0064] Next, the operation (information processing method) of the information processing device 1 with this configuration will be explained. Although the devices on which processing is performed differ in some respects from those in Embodiment 1, the flow of operation (steps) is the same as in Embodiment 1, so we will explain it with reference to Figure 2.

[0065] First, in step S100, in addition to user identification, a process is performed to link the mobile terminal 20 with the terminal device 10. Here, authentication is performed to determine whether the mobile terminal 20 is a terminal that can link with the terminal device 10. If the linkage is approved, the terminal device 10 and the mobile terminal 20 become able to communicate. This allows the terminal device 10 to understand the user's response detected by the detection unit 211 of the mobile terminal 20 and the contents of the information retrieval performed by the information retrieval unit 212. Next, the process proceeds to step S200.

[0066] Steps S200 to S400 are the same as in Example 1, so their explanation will be omitted. In step S500, a detection process is performed. Here, in addition to the configuration of Example 1, the information retrieval unit 212 of the mobile terminal 20 is monitored, and the operation of the information retrieval unit 212 that occurs after the output of the guidance voice is detected as a user response.

[0067] In step S600, in addition to the example from Embodiment 1, the confirmation action is defined as "the user performing a search action using the information retrieval unit 212." This confirmation action is detected by the extraction unit 113 when there is data in the response data created by the detection unit 211 indicating that the information retrieval unit 212 was operated for purposes such as route search, location search, or keyword search, and is extracted as confirmation data 140. In other words, the confirmation action includes, in addition to the confirmation action from Embodiment 1, the user performing a vehicle route search using the information retrieval unit 212, the user performing a location search to find the coordinates of a specific location using the information retrieval unit 212, and the user performing a keyword search for a specific word using the information retrieval unit 212.

[0068] According to Example 2, if a problem arises such as the user not understanding the guidance voice or the guidance voice lacking certain information, the user's attempt to resolve this problem using the information retrieval unit 212, which is independent of the route search unit 111, can be extracted as a confirmation action. For example, if a user driving a vehicle pulls over to the side of the road after hearing the guidance voice and searches for information using a smartphone or other device, this action can be extracted as a confirmation action. Furthermore, since the terminal device 10 and the mobile terminal 20 can be linked, the items to be confirmed can be estimated by referring to the search words (keywords) entered into the information retrieval unit 212 and the method of outputting the search results (i.e., audio or image), thereby gaining a more detailed understanding of what the user wanted to confirm.

[0069] Therefore, for example, an action in which a user performs a route search using the information retrieval unit 212 and displays the results on an unshown display unit can be considered a confirmation action and extracted as confirmation data 140. In this case, it can be inferred that the user could not visualize the route from the voice guidance and wanted to confirm the route with an image. Alternatively, an action in which a user performs a keyword search using the information retrieval unit 212 can be considered a confirmation action and extracted as confirmation data 140. In this case, if the search terms for the keyword search are the names of facilities, places, or roads, it can be inferred that the user did not understand the names of facilities, places, or roads in the voice guidance.

[0070] Furthermore, in this case, especially if the keywords used in the search are in hiragana or katakana, it can be inferred that the user was unfamiliar with the name of the facility, place, or road. Also, if the same facility, place, or road name is searched for multiple times, it can be inferred that the user is thinking of multiple locations with the same pronunciation as the facility, place, or road name that was suggested.

[0071] In this way, even if a user attempts to solve the problem by performing an information search using a mobile device 20 such as a smartphone (a terminal device other than the terminal device 10 that initially performed the route search), it becomes less likely for the information processing device 1 to overlook the fact that the problem occurred, and it becomes possible to improve the content of the guidance voice by utilizing the estimated data 141 stored in the memory unit 14. Furthermore, since the information processing device 1 can be composed of a terminal device 10 and a mobile device 20, the load on each device can be reduced compared to an information processing device 1 composed of a single terminal device 10.

[0072] [Example 3] Next, the information processing device 1 according to Embodiment 3 of this embodiment will be described with reference to Figure 4. Note that parts identical to those in Embodiments 1 and 2 described above are denoted by the same reference numerals, and their descriptions are omitted or simplified.

[0073] As shown in Figure 4, the information processing device 1 according to this embodiment includes a terminal device 10 (first terminal device), a mobile terminal 20 (second terminal device), and a server device 30.

[0074] The terminal device 10 is a device that moves with the vehicle, similar to the first embodiment, and has the same configuration as the terminal device 10 of the second embodiment, except that it does not have an extraction unit 113, an estimation unit 114, and a storage unit 14. The portable terminal 20 has the same configuration as the portable terminal 20 of the second embodiment.

[0075] The server device 30 is configured to communicate with the terminal device 10 and the mobile terminal 20, and includes a control unit 31, a storage unit 32, and a communication unit 33. The control unit 31 includes an extraction unit 310 and an estimation unit 311. The extraction unit 310, estimation unit 311, and storage unit 32 correspond to the extraction unit 113, estimation unit 114, and storage unit 14 in Embodiments 1 and 2, respectively, and have the same functions as those configurations. The communication unit 33 is configured to communicate with the communication unit 15 and the communication unit 22, and is capable of sending and receiving the above-mentioned reaction data, confirmation data 140, and estimation data 141.

[0076] By doing so, the load on each device can be reduced compared to an information processing device 1 consisting of a single terminal device 10, or an information processing device 1 consisting of a terminal device 10 (first terminal device) and a mobile terminal 20 (second terminal device).

[0077] It should be noted that the present invention is not limited to the embodiments described above. That is, those skilled in the art can implement the invention in various ways, without departing from the core principles, in accordance with prior art knowledge. As long as such modifications still provide the information processing device of the present invention, they are of course included within the scope of the present invention.

[0078] For example, the information that the user is expected to want to confirm in the estimation process may include "detailed facility information." Specifically, the first step is to enable voice guidance about the destination facility in the voice output process. When providing guidance about the facility, actions such as "the user looked up detailed facility information on their smartphone, etc." are extracted as confirmation actions. From these confirmation actions, it is estimated that the information the user needed (or that was missing from the voice guidance) was "detailed facility information." In this way, the device or system can grasp more information about what the user needed (or what was missing from the voice guidance). [Explanation of Symbols]

[0079] 1. Information Processing Device 13. Output section (Audio output section) 14 Storage section 112 Detection unit 113 Extraction part 114 Estimation Department

Claims

[Claim 1] A voice output unit that outputs guidance audio to be provided to the user, A detection unit for detecting the user's response to the guidance voice, An extraction unit extracts the user's confirmation actions related to the guidance voice from the reactions detected by the detection unit, An estimation unit that estimates the matters the user wants to confirm based on the aforementioned confirmation action, An information processing apparatus comprising a storage unit for storing information about the matters estimated by the estimation unit.

Images