Update program, update device and update method

Abstract

To propose appropriate search queries in a service used by users.SOLUTION: An update program pertaining to the present invention is characterized in causing a computer to execute: a display step for displaying character strings indicating candidates of a search query corresponding to a service used by a user, which are the candidates of the search query to be inputted through voice; a reception step for receiving an input of information through voice from the user; and an update step for updating and displaying a display mode of the character strings based on a voice recognition result of the information received by the reception step.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an update program, an update device, and an update method. [Background technology]

[0002] Conventionally, there are known techniques related to voice input of search queries. For example, a technique has been disclosed for creating a user's search instructions based on a voice recognition result that recognizes a voice input, a demonstrative term extracted from the voice recognition result, and a user's instruction estimated from the user's actions when the user does not know words related to a search target. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-153324 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional techniques may not be able to suggest appropriate search queries for the services used by the users.

[0005] For example, the above-mentioned conventional technology merely creates search instructions when a user does not know words related to the search target, and cannot be said to suggest appropriate search queries for the services used by the user.

[0006] The present application has been made in consideration of the above, and aims to provide an update program, an update device, and an update method that can suggest appropriate search queries for a service used by a user. [Means for solving the problem]

[0007] The update program of the present application is characterized in that it causes a computer to execute a display step of displaying character strings indicating search query candidates input via voice, the character strings being search query candidates corresponding to a service used by a user; a reception step of receiving information input via voice from the user; and an update step of updating and displaying the display mode of the character strings based on the results of voice recognition of the information received by the reception step. Effect of the Invention

[0008] According to one aspect of the embodiment, it is possible to provide an effect of suggesting appropriate search queries for a service used by a user. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of an update process according to the embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a screen of the terminal device according to the embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of the configuration of a terminal device according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of a voice input application according to the embodiment. [Diagram 5] FIG. 5 is a flowchart illustrating an example of an update process procedure according to the embodiment. [Figure 6] FIG. 6 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the terminal device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment (hereinafter, referred to as "embodiment") for carrying out the update program, update device, and update method according to the present application will be described in detail with reference to the drawings. Note that the update program, update device, and update method according to the present application are not limited to the embodiment. In addition, the same components in the following embodiments are given the same reference numerals, and duplicated descriptions will be omitted.

[0011] 1. Embodiment An update process realized by an update program etc. of this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the update process according to the embodiment. Fig. 1 shows an example in which a voice input application (hereinafter, sometimes referred to as "voice input app") that displays a character string that is a result of voice recognition in response to a voice input by a user and outputs the displayed character string to a predetermined external device or another application is implemented in a terminal device 100 used by a user. Note that in Fig. 1, it is assumed that the update process according to the embodiment is realized by the function of the voice input app.

[0012] As shown in Fig. 1, an update system 1 according to an embodiment includes a service server 10 and a terminal device 100. The service server 10 and the terminal device 100 are connected to each other via a network N (see Fig. 3, for example) so as to be able to communicate with each other by wire or wirelessly. The network N is, for example, a wide area network (WAN) such as the Internet. Note that the update system 1 shown in Fig. 1 may include a plurality of service servers 10 and a plurality of terminal devices 100.

[0013] 1 is an information processing device that provides various services to users, and is realized by, for example, a server device, a cloud system, etc. For example, the service server 10 distributes to the terminal device 100 information to be displayed on web pages related to, for example, a portal site, a news site, an auction site, a weather forecast site, a shopping site, a finance (stock price) site, a route search site, a map providing site, a travel site, a restaurant introduction site, a web blog, etc., and various applications (for example, a portal application, a news application, an auction site, a weather forecast application, a shopping application, a finance (stock price) application, a route search application, a map providing application, a travel application, a restaurant introduction application, a blog viewing application, etc.) installed in the terminal device 100.

[0014] The service server 10 may function as a distribution device that distributes control information to the terminal device 100. Here, the control information is described in, for example, a script language such as JavaScript (registered trademark) or a style sheet language such as CSS (Cascading Style Sheets). The application itself distributed from the service server 10 may be regarded as the control information.

[0015] The terminal device 100 shown in FIG. 1 is an information processing device used by a user. The terminal device 100 is realized by, for example, a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like. In the example shown in FIG. 1, the terminal device 100 is a smartphone used by a user. The terminal device 100 has an output unit (hereinafter, may be referred to as a screen) such as a liquid crystal display, and is assumed to employ a touch panel. That is, a user U1 performs various operations of the terminal device 100 by touching the display surface of the output unit with a finger or a dedicated pen. In the example of FIG. 1, the terminal device 100 is assumed to include a sound collecting device (such as a microphone) that collects sound.

[0016] The following describes the update process of the voice input application performed by the terminal device 100 with reference to Fig. 1. In the following description, an example is shown in which the terminal device 100 is used by a user U1. In the following description, an example is shown in which the service server 10 provides a route search site that provides a route search service and a route search application.

[0017] First, the terminal device 100 displays a screen related to a service corresponding to an operation by the user U1 (step S1). For example, in the example of Fig. 1, the terminal device 100 displays a screen C11 related to a route search service provided by the service server 10 in response to an operation by the user U1. As a specific example, the terminal device 100 displays the screen C11 including an area AR11 for inputting search conditions (search query) such as a departure station and an arrival station, and a button B11 for instructing voice input of the search query into the area AR11.

[0018] 1, it is assumed that the user U1 presses the button B11. In this case, the terminal device 100 starts a voice input application and transitions the screen C11 to a screen displaying character strings indicating search query candidates that can be input in the route search service provided by the service server 10 (step S2). For example, the terminal device 100 transitions to the screen C12 including an area AR12 displaying the recognition result of the input voice, areas AR13 and AR14 displaying character strings that are preset as search query candidates in the route search service and can be input in the area AR11, and a button B12 for instructing the start of voice input.

[0019] As a specific example, the terminal device 100 displays in the area AR13 a character string (hereinafter, sometimes referred to as "character string #11") for specifying the departure station and arrival station, and the first train (first train) or last train on the route from the departure station to the arrival station. As a more specific example, the terminal device 100 displays the character string #11 including areas AR131 and AR132 that respectively display the departure station and arrival station that are voice-input by the user U1, and an area AR133 that alternatively displays "first train" and "last train."

[0020] Furthermore, the terminal device 100 displays in the area AR14 a character string (hereinafter, sometimes referred to as "character string #12") for specifying the departure station and the arrival station, and the departure time for departing from the departure station or the arrival time for arriving at the arrival station. As a more specific example, the terminal device 100 displays the character string #12 including areas AR141 and AR142 respectively displaying the departure station and the arrival station input by voice by the user U1, areas AR143 and AR144 displaying the time input by voice by the user U1, and an area AR145 alternatively displaying "depart" and "arrival".

[0021] Next, the user U1 presses the button B12 and performs voice input (utterance) according to either the character string #11 or #12 displayed on the screen C12 (step S3). In the example of Fig. 1, it is assumed that the user U1 wishes to search for the last train for a route starting from Tokyo Station and arriving at Ikebukuro Station, and utters "last train from Tokyo to Ikebukuro" into the microphone included in the terminal device 100 according to the character string #11.

[0022] Next, the terminal device 100 performs voice recognition on the speech of the user U1, and updates the display mode of the character strings #11 and #12 based on the result of the voice recognition (step S4). For example, the terminal device 100 displays the character string "last train from Tokyo to Ikebukuro" which is the result of the voice recognition in the area AR12. Then, the terminal device 100 changes the character color of the character string that matches the result of the voice recognition in the character string #11 and displays it in a highlighted manner, and displays the character string indicated by the result of the voice recognition in each area included in the character strings #11 and #12. As a specific example, the terminal device 100 changes the character color of the character strings that correspond to "from" and "until" spoken by the user U1 in the character string #11. The terminal device 100 also displays "Tokyo" spoken by the user U1 before "from" in the area AR131, and displays "Ikebukuro" spoken between "from" and "until" in the area AR132. Furthermore, the terminal device 100 changes the text color of "last train" corresponding to the speech of the user U1, out of "first train" and "last train" displayed in the area AR133. Then, the terminal device 100 displays a check mark in the area AR13 indicating that the update process for the character strings and each area included in the character string #11 has been completed (in other words, the information that is a required item for performing the search process based on the character string #11 has been normally input by voice).

[0023] Similarly, the terminal device 100 updates the display mode of the character string #12 based on the result of the voice recognition. As a specific example, the terminal device 100 changes the character color of the character strings corresponding to "from" and "until" spoken by the user U1 in the character string #12. The terminal device 100 also displays "Tokyo" spoken by the user U1 before "from" in the area AR141, and displays "Ikebukuro" spoken between "from" and "until" in the area AR142. In the example of FIG. 1, the utterance by the user U1 after "until" is "last train", and the utterance ends with "last train", so the terminal device 100 displays "last train" in the area AR143 located after the character string "until" in the character string #12, and does not perform processing to update the display mode of the subsequent character strings. In other words, the update processing for the character strings and each area included in the character string #11 is incomplete.

[0024] Next, in response to an operation by the user U1, the terminal device 100 transmits the updated character string as a search query to the service server 10 (step S5). For example, the terminal device 100 transmits, as the search query, the updated character string #11 selected by the user on the screen C13 from among the updated character strings #11 and #12.

[0025] In addition, the terminal device 100 may be able to select only the character string #11 for which the update process has been completed among the updated character strings #11 and #12, and may automatically transmit the character string #11 for which the update process has been completed to the service server 10 as a search query.

[0026] Next, the terminal device 100 receives the search result from the service server 10 (step S6). For example, the terminal device 100 receives a transfer guide for the last train on the route from "Tokyo Station" to "Ikebukuro Station" identified from "Tokyo" and "Ikebukuro" included in the updated character string #11.

[0027] Next, the terminal device 100 displays the received search results (step S7). For example, a screen C14 including information on transfer guidance from Tokyo Station to Ikebukuro Station (for example, the route to be used, the stations to be passed through, and the fare) and a button B13 for instructing voice input of a new search query is displayed. As a specific example, the terminal device 100 displays the button B13 for instructing voice input of a search query for narrowing down the information on transfer guidance.

[0028] Here, an example of a screen displayed by the terminal device 100 when the button B13 is pressed will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of a screen of the terminal device according to the embodiment.

[0029] 2, the terminal device 100 starts the voice input application and displays character strings indicating search query candidates corresponding to the narrowing down functions available in the route search service, as shown in a screen C21. For example, the terminal device 100 displays the screen C21 including an area AR21 displaying the search query transmitted to the route search service in step S5 in FIG. 1, an area AR22 displaying the recognition result of the input voice, areas AR23-AR25 displaying character strings preset as search query candidates for narrowing down the search results, and a button B21 for instructing the start of voice input.

[0030] As a specific example, the terminal device 100 displays in the area AR23 a character string (hereinafter, sometimes referred to as "character string #21") for specifying whether the fare for the route from Tokyo Station to Ikebukuro Station should be displayed as the fare when an IC card is used (IC card priority) or as the fare when cash (ticket) is used (cash priority). As a more specific example, the terminal device 100 displays the character string #21 including the area AR231 that alternatively displays "IC card priority" and "cash priority".

[0031] In addition, the terminal device 100 displays in area AR24 a character string (hereinafter, sometimes referred to as "character string #22") to instruct the display of fares for the route from Tokyo Station to Ikebukuro Station, excluding the fares for the route that can be used with the commuter pass owned by user U1.

[0032] Furthermore, when transferring on the route from Tokyo Station to Ikebukuro Station, the terminal device 100 displays in the area AR25 a character string (hereinafter, may be referred to as "character string #23") for instructing the display of transfer guidance assuming the speed at which the user U1 moves within the station premises, etc. is to be displayed. As a more specific example, the terminal device 100 displays the character string #23 including the area AR251 that alternatively displays a case in which the user U1 moves quickly ("hurry"), a case in which the user U1 moves a little quickly ("a little quickly"), a case in which the user U1 moves slowly ("slowly"), and a case in which the user U1 moves a little slowly ("a little slowly").

[0033] In the example of FIG. 2, it is assumed that user U1 desires to display fares with cash priority, and presses button B21 to say "Search with cash priority" according to character string #21 into the microphone included in the terminal device 100. In this case, the terminal device 100 performs voice recognition on the speech from user U1, and displays a screen C22 in which the display mode of character strings #21-#23 is updated based on the result of the voice recognition. For example, the terminal device 100 displays the character string "Search with cash priority" which is the result of the voice recognition in the area AR22. Furthermore, the terminal device 100 changes the character color of "Cash priority" corresponding to the speech of user U1 among "IC card priority" and "Cash priority" displayed in the area AR231 included in the character string #21. Furthermore, the terminal device 100 changes the character color of the character string corresponding to "Search with" spoken by user U1 after "Cash priority" among the character strings included in the character string #21. Then, the terminal device 100 displays in the area AR23 a check mark indicating that the update process for the character strings included in the character string #21 and each area has been completed.

[0034] Similarly, the terminal device 100 updates the display mode of character strings #22 and #23 based on the results of speech recognition. For example, the terminal device 100 changes the character color of the character string included in character string #22 that corresponds to "Search with" spoken by user U1. The terminal device 100 then inserts into character string #22 the character string that corresponds to "Cash preferred" that was spoken by user U1 before "Search with." Note that the terminal device 100 does not perform a process to update the display mode of the character string "Taking into consideration the regular interval" included in character string #22, because this character string is not included in the speech of user U1.

[0035] Also, for example, the terminal device 100 inserts into character string #23 a character string that corresponds to "Search for cash preferred" uttered by user U1, among the character strings included in character string #22. Note that the terminal device 100 does not perform a process of updating the display mode of each character string displayed in the area AR251 included in character string #22, and the character string "Transfer" included in character string #22, because these character strings are not included in the utterance of user U1.

[0036] After the above update process is completed, the terminal device 100 transmits the updated character string #21 selected by the user on the screen C22 from among the updated character strings #21-#23 to the service server 10 as a search query. Then, the terminal device 100 displays on the screen information related to transfer guidance from Tokyo Station to Ikebukuro Station, including a fare with a cash priority. In other words, the terminal device 100 displays on the screen information corresponding to the information displayed on the screen C14 of FIG. 1, which has been narrowed down based on the updated character string #21.

[0037] The terminal device 100 may accept a change in the information displayed in the area AR21. For example, when the area AR211 is pressed, the terminal device 100 deletes the character string "Tokyo" and accepts a voice input of a new departure station. When the area AR211 is pressed, the terminal device 100 deletes the character string "Ikebukuro" and accepts a voice input of a new arrival station.

[0038] As described above, the terminal device 100 according to the embodiment displays character strings that are candidates for search queries corresponding to functions available in a service used by the user, and enables the user to input a search query by voice according to the character strings. As a result, when the user performs an operation to call up a function of a service using a voice UI, the terminal device 100 according to the embodiment can make the user recognize what functions are available by voice input, even if the user does not know in advance information such as "what kind of speech can be used to perform what operation" and "what functions can be operated by voice." In other words, the terminal device 100 according to the embodiment can suggest an appropriate search query in a service used by the user.

[0039] 2. Terminal Device Configuration Next, a terminal device 100 for realizing the above-mentioned update process will be described with reference to Fig. 3. Fig. 3 is a diagram showing a configuration example of a terminal device according to an embodiment. As shown in Fig. 3, the terminal device 100 has a communication unit 110, an input unit 120, an output unit 130, and a control unit 140.

[0040] (Regarding communication unit 110) The communication unit 110 is realized by, for example, a network interface card (NIC) or the like. The communication unit 110 is connected to a network N by wire or wirelessly, and transmits and receives information between the service server 10 and an interpretation server 200 that executes a process of interpreting the contents of information input by voice. For example, the interpretation server 200 provides a function of interpreting the contents of information input by voice via an application programming interface (API).

[0041] (Regarding the input unit 120 and the output unit 130) The input unit 120 is an input device that accepts various operations from a user. For example, the input unit 120 is realized by a keyboard, a mouse, operation keys, etc. The output unit 130 is a display device for displaying various information. That is, the output unit 130 corresponds to the display screen of the terminal device 100. For example, the output unit 130 is realized by a liquid crystal display, etc. Note that, when a touch panel is adopted in the terminal device 100, the input unit 120 and the output unit 130 are integrated.

[0042] (Regarding the control unit 140) The control unit 140 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs stored in a storage device inside the terminal device 100 using a RAM as a working area. When control information or an application for realizing the above-mentioned display process is distributed from an application distribution server or the like, the control unit 140 operates according to the control information or application to realize the display process.

[0043] The control unit 140 is a controller, and is realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). As shown in FIG. 3, the control unit 140 according to the embodiment has a voice input application 141, a first application 142, a second application 143, and the like. The voice input application 141 realizes or executes the functions and actions of the above-mentioned update process, and transmits information that is the result of voice recognition of the voice input of the user to the service server 10. The voice input application 141 also outputs information that is the result of voice recognition to other applications such as the first application 142 and the second application 143. Hereinafter, an example of the functional configuration of the voice input application 141 will be described with reference to FIG. 4. FIG. 4 is a diagram showing an example of the configuration of the voice input application according to the embodiment. As shown in FIG. 3, the voice input application 141 according to the embodiment has a display unit 1411, a receiving unit 1412, an update unit 1413, a presentation unit 1414, and a transmission unit 1415, and realizes or executes the functions and actions of the information processing described below.

[0044] (Regarding the display unit 1411) The display unit 1411 displays character strings indicating search query candidates input via voice and corresponding to a service used by the user. For example, in the example of Fig. 1 and Fig. 2, the display unit 1411 displays character strings indicating search query candidates that can be input in a route search service provided by the service server 10, which are preset as search query candidates in the route search service, on a screen C12, a screen C21, etc.

[0045] The display unit 1411 may display character strings corresponding to a search function provided by the service. For example, in the examples of Fig. 1 and Fig. 2, the display unit 1411 displays character strings indicating candidates of search queries corresponding to a route search function in a search service provided by a route search service or a narrowing-down function available in the route search service on a screen C12, a screen C21, or the like.

[0046] Furthermore, the display unit 1411 may display a character string indicating a target that can be searched for using a search function provided by the service. For example, in the example of Fig. 1, the display unit 1411 displays a character string related to a route that can be searched for using a search function provided by the route search service on the screen C12, the screen C21, etc.

[0047] The display unit 1411 may also display a character string including a display area in which a character string input through the user's voice is displayed. For example, in the example of Fig. 1, the display unit 1411 displays a character string #11 including areas AR131 and AR132 that respectively display the departure station and arrival station input by the user U1 through voice. The display unit 1411 also displays a character string #12 including areas AR141 and AR142 that respectively display the departure station and arrival station input by the user U1 through voice, and areas AR143 and AR144 that display the time input by the user U1 through voice.

[0048] Furthermore, the display unit 1411 may display the type of information that can be input via voice in the display area. For example, in the example of Fig. 1, the display unit 1411 displays the type of information that can be input via voice, "station", in each of the areas AR131, AR132, AR141, and AR142. Furthermore, the display unit 1411 displays the type of information that can be input via voice, "number", in each of the areas AR143 and AR144.

[0049] In addition, if a character string that does not correspond to the type to be displayed in a display area is input by voice, the update unit 1413 described later does not have to display the character string in the display area.

[0050] The display unit 1411 may display a character string including an area for displaying a plurality of character strings having a predetermined correlation. For example, in the example of FIG. 1, the display unit 1411 displays a character string #12 including an area AR145 for alternatively displaying "departure" and "arrival". In the example of FIG. 1, the display unit 1411 displays a character string #12 including an area AR145 for alternatively displaying "departure" and "arrival". In the example of FIG. 2, the display unit 1411 displays a character string #21 including an area AR231 for alternatively displaying "IC card preferred" and "cash preferred". In the example of FIG. 2, the display unit 1411 displays a character string #23 including an area AR251 for alternatively displaying "hurry", "hurry a little", "slowly" and "slowly".

[0051] (Regarding the reception unit 1412) The receiving unit 1412 receives information input from a user through speech. For example, in the example of FIG.

[0052] (Regarding update part 1413) The update unit 1413 updates and displays the display mode of the character strings based on the result of the voice recognition of the information accepted by the acceptance unit 1412. For example, in the example of Fig. 1, the update unit 1413 updates the display mode of the character strings #11 and #12 based on the character string "Last train from Tokyo to Ikebukuro" which is the result of the voice recognition of the utterance from the user U1. Also, in the example of Fig. 2, the update unit 1413 updates the display mode of the character strings #21-#23 based on the character string "Search for cash preferred" which is the result of the voice recognition of the utterance from the user U1.

[0053] Furthermore, the update unit 1413 may update the display mode of the portion of the character string that corresponds to the result of the voice recognition. For example, in the example of Fig. 1 and Fig. 2, the update unit 1413 updates the display mode of the portion of the character strings #11, #12, and character strings #21-#23 that corresponds to the result of the voice recognition.

[0054] Furthermore, the update unit 1413 may highlight character strings that match the results of voice recognition among the character strings. For example, in the example of Fig. 1, the update unit 1413 changes the character color of character strings #11 and #12 that match the results of voice recognition of the utterance from the user U1. Furthermore, in the example of Fig. 2, the update unit 1413 changes the character color of character strings #21-#23 that match the results of voice recognition of the utterance from the user U1.

[0055] Furthermore, the update unit 1413 may display a character string indicated by the result of the speech recognition in a display area. For example, in the example of Fig. 1, the update unit 1413 displays "Tokyo" spoken by the user U1 before "kara" in the area AR131, and displays "Ikebukuro" spoken between "kara" and "made" in the area AR132.

[0056] Furthermore, the update unit 1413 may display, in an area, one of the multiple character strings that corresponds to the result of the voice recognition. For example, in the example of Fig. 1, the update unit 1413 changes the character color of "last train" corresponding to the utterance of user U1 among "first train" and "last train" displayed in area AR133 included in character string #11. Furthermore, in the example of Fig. 2, the update unit 1413 changes the character color of "cash preferred" corresponding to the utterance of user U1 among "IC card preferred" and "cash preferred" displayed in area AR231 included in character string #21.

[0057] Here, even if the result of speech recognition of an utterance from a user does not match the character string displayed by the display unit 1411, if the content obtained by interpreting the result of speech recognition corresponds to the character string, it is conceivable that a user may wish to update the display mode of the character string. Therefore, the update unit 1413 may update and display the display mode of the character string based on the result of interpretation by an external device. For example, the update unit 1413 updates the display mode of the character string displayed by the display unit 1411 based on the content obtained by interpreting the information input by the user's voice, which is the content obtained by interpreting the information input by the user's voice, and the content obtained by interpreting the information input by the user's voice, by the interpretation server 200. As a specific example, if the content interpreted by the interpretation server 200 corresponds to the content indicated by the character string displayed by the display unit 1411, the update unit 1413 changes the display mode of the character string.

[0058] (Regarding the presentation unit 1414) Here, for example, when the user's speech is not correctly recognized and a part of the character string displayed by the display unit 1411 is not updated, or when the user does not utter a part of the character string (such as a required item) displayed by the display unit 1411, the user may wish to prompt the user to input the voice again. Therefore, when the character string updated by the update unit 1413 satisfies a predetermined condition, the presentation unit 1414 presents the user with content for accepting input of new information via voice. For example, when a part of the character string displayed by the display unit 1411 that corresponds to a service used by the user and that corresponds to a required item set in the service has not been updated by the update unit 1413, the presentation unit 1414 presents the user with content indicating that necessary information has not been input by voice and for accepting input of new information via voice. As a specific example, the presentation unit 1414 presents content for accepting voice input of the part corresponding to the required item. Then, the update unit 1413 updates the display mode of the part corresponding to the required item based on the information input by voice via the content presented by the presentation unit 1414.

[0059] Furthermore, when there is a part of the character string whose display mode has not been updated, the presentation unit 1414 may present content for inputting information corresponding to that part. For example, when all of the character string displayed by the display unit 1411 has not been updated by the update unit 1413, the presentation unit 1414 presents content for receiving a voice input for the part of the character string that has not been updated.

[0060] Furthermore, when a character string is not displayed in an area included in a character string where a character string inputted via the user's voice is displayed, the presentation unit 1414 may present content for inputting information corresponding to the area. For example, in a description using FIG. 1 as an example, when a character string is not displayed in either of the areas AR131 and AR132 included in the character string #11, the presentation unit 1414 presents content for inputting information corresponding to the area AR131 or AR132. In a description using FIG. 1 as an example, when the display mode of neither "first train" nor "last train" has been updated in the area AR133 included in the character string #11, the presentation unit 1414 presents content for selecting either "first train" or "last train".

[0061] Furthermore, the presentation unit 1414 may present content for selecting one of the candidate character strings, which are candidates for character strings to be displayed in an area included in the character string and in which a character string inputted via the user's voice is displayed, when there are a plurality of candidate character strings based on the result of voice recognition. For example, when there are a plurality of conversion candidates for the information inputted by the user's voice into kanji, the presentation unit 1414 presents content for selecting one of the conversion candidates. Furthermore, when there are a plurality of place names corresponding to the information inputted by the user's voice, the presentation unit 1414 presents content for selecting one of the place names. As a specific example, when the user inputs "Nakano" by voice, the presentation unit 1414 presents content for selecting one of "Nakano" located in Tokyo, "Nakano" located in Gunma Prefecture, and "Nakano" located in Nagano Prefecture.

[0062] In addition, when the display unit 1411 displays a plurality of character strings that are candidates for a search query, the presentation unit 1414 may not present content to the user if any of the character strings does not satisfy a predetermined condition. For example, in the case of FIG. 1, the presentation unit 1414 may not display content for performing voice input of information related to character string #12, because the update process for character strings and each area included in character string #11 has been completed among character strings #11 and #12. In other words, when any of the character strings displayed by the display unit 1411 can be transmitted as a search query, the presentation unit 1414 may not present content.

[0063] In addition, the presentation unit 1414 may present content for accepting information input by means other than voice, such as a touch panel of the terminal device 100, a keyboard, a mouse, or operation keys, in addition to the content for accepting new information input via voice.

[0064] (Regarding the transmitting unit 1415) When the transmission unit 1415 receives a transmission request for a character string updated by the update unit 1413 from a user, the transmission unit 1415 transmits the updated character string as a search query to an external device that executes a search process corresponding to the service. For example, in the example of Fig. 1, the transmission unit 1415 transmits updated character string #11 selected by the user on screen C13 to the service server 10 as a search query.

[0065] The transmission unit 1415 may transmit the updated character string as a search query to an external device that executes a search process corresponding to a service different from the service currently being used by the user. For example, in the example of FIG. 1, the transmission unit 1415 transmits the updated character string #11 as a search query to the service server 10 that provides a service different from the route search service in response to an operation by the user. The terminal device 100 then receives the search result from the service server 10 and displays it on the screen.

[0066] Furthermore, the transmitting unit 1415 may transmit the information accepted by the accepting unit 1412 to an external device that executes a process of interpreting the content of the voice. For example, the transmitting unit 1415 transmits the information input by voice to the interpretation server 200. Then, the updating unit 1413 executes an updating process based on the content interpreted by the interpretation server 200.

[0067] [3. Update process flow] Here, a procedure of an update process of a voice input application according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of a procedure of the update process according to the embodiment.

[0068] As shown in FIG. 5, the terminal device 100 displays character strings indicating search query candidates corresponding to services used by the user (step S101). Then, the terminal device 100 accepts input of information via voice (step S102). Then, the terminal device 100 updates the display mode of the character string based on the result of the voice recognition and displays it (step S103). Then, the terminal device 100 determines whether the updated character string satisfies a predetermined condition (step S104). If the predetermined condition is satisfied (step S104; Yes), the terminal device 100 presents content for accepting input of new information via voice (step S105), and returns to step S102 to repeat the process.

[0069] On the other hand, if the predetermined condition is not satisfied (step S104; No), the terminal device 100 judges whether or not a request to transmit the character string has been received from the user (step S106). If a request to transmit has not been received (step S106; No), the terminal device 100 waits until a request to transmit is received. On the other hand, if a request to transmit has been received (step S106; Yes), the terminal device 100 transmits the updated character string as a search query to an external device that executes the search process (step S107), and ends the process.

[0070] 4. Modifications The above-described embodiment is merely an example, and various modifications and applications are possible.

[0071] 4-1. Services provided by the service server 10 In the above embodiment, an example in which the service server 10 provides a route search service has been described, but the service provided by the service server 10 is not limited to this example, and any service may be provided. The display unit 1411 may display character strings according to the service provided by the service server 10. For example, when the service server 10 provides a shopping service, the display unit 1411 may display character strings corresponding to search functions available in the shopping service as search query candidates. As a specific example, the display unit 1411 displays character strings indicating the type, manufacturer, price range, etc. of a product sold in the shopping service.

[0072] [4-2. Update process by update unit 1413] In the above embodiment, an example has been shown in which the update unit 1413 performs the update process by changing the character color of a character string that matches the result of voice recognition of an utterance from a user among the character strings displayed by the display unit 1411, but the function of the update unit 1413 is not limited to such an example. For example, the update unit 1413 may perform the character string update process in any manner as long as the user can determine whether or not the content of the utterance has been voice recognized.

[0073] 4-3. Character strings displayed on the display unit 1411 In the above embodiment, the display unit 1411 displays a character string corresponding to a search function provided by a service used by the user. However, the function of the display unit 1411 is not limited to this example. For example, the display unit 1411 may display a character string based on the content uttered by the user when the user activates a voice input application. For example, when the user activates a voice input application and inputs some information by voice, the transmission unit 1415 transmits the information to the interpretation server 200. Then, the display unit 1411 displays a character string indicating a candidate search query in the service corresponding to the result of the interpretation by the interpretation server 200. As a specific example, when the user utters "I'm hungry," the display unit 1411 displays a character string indicating a candidate search query for searching for groceries in a shopping service, a character string indicating a candidate search query for searching for a store corresponding to the user's current location in a restaurant search service, and the like.

[0074] [4-4. Processing mode] Among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, and conversely, all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above text and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0075] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0076] Furthermore, the above-described embodiments can be appropriately combined as long as the processing contents are not contradictory.

[0077] 5. Effects As described above, the voice input application 151 included in the terminal device 100 according to the embodiment includes the display unit 1411, the reception unit 1412, the update unit 1413, the presentation unit 1414, and the transmission unit 1415. The display unit 1411 displays a character string indicating a search query candidate input via voice, which is a search query candidate corresponding to a service used by the user. The reception unit 1412 receives information input via voice from the user. The update unit 1413 updates and displays the display mode of the character string based on the result of voice recognition of the information received by the reception unit 1412. The update unit 1413 updates and displays the display mode of the character string based on the result of voice recognition of the information received by the reception unit 1412. The presentation unit 1414 presents content for receiving input of new information via voice to the user when the character string updated by the update unit 1413 satisfies a predetermined condition. When the transmitting unit 1415 receives a transmission request for the character string updated by the updating unit 1413 from the user, the transmitting unit 1415 transmits the updated character string as a search query to an external device that executes a search process corresponding to the service.

[0078] As a result, the terminal device 100 according to the embodiment can enable the user to recognize what functions are available via voice input in the service the user is using, and can therefore suggest appropriate search queries for the service the user is using.

[0079] In the terminal device 100 according to the embodiment, for example, the display unit 1411 displays a character string corresponding to a search function provided by the service. Also, the display unit 1411 displays a character string indicating an object that can be searched using the search function provided by the service.

[0080] As a result, the terminal device 100 according to the embodiment can make the user aware of what search functions are available in the service the user uses, and can therefore suggest an appropriate search query for the service the user uses.

[0081] In the terminal device 100 according to the embodiment, for example, the update unit 1413 updates the display mode of a portion of a character string that corresponds to a result of speech recognition. In addition, the update unit 1413 highlights a character string that matches a result of speech recognition.

[0082] This allows the terminal device 100 according to the embodiment to know whether the contents uttered by the user have been correctly voice-recognized, thereby improving the convenience of voice input.

[0083] In the terminal device 100 according to the embodiment, for example, the display unit 1411 displays a character string including a display area in which a character string inputted through the user's voice is displayed. Then, the update unit 1413 displays a character string indicated by the result of the voice recognition in the display area. In addition, the display unit 1411 displays the type of information that can be inputted through voice in the display area.

[0084] As a result, the terminal device 100 of the embodiment can prompt the user to input the information by voice by displaying the type of information that the user should input by voice, and can provide search results based on the information input arbitrarily, thereby improving the convenience of voice input.

[0085] In the terminal device 100 according to the embodiment, for example, the display unit 1411 displays a character string including an area for displaying a plurality of character strings having a predetermined correlation. Then, the update unit 1413 displays one character string corresponding to a result of the speech recognition in the area among the plurality of character strings.

[0086] As a result, the terminal device 100 according to the embodiment can alternatively display information to be input by voice and allow the user to select it, thereby reducing the burden of voice input and improving convenience.

[0087] In the terminal device 100 according to the embodiment, for example, the transmission unit 1415 transmits the information accepted by the acceptance unit 1412 to an external device that executes a process of interpreting the content of the voice. Then, the update unit 1413 updates and displays the display mode of the character string based on the result of the interpretation by the external device.

[0088] As a result, even if the result of voice recognition of a speech utterance from a user does not match the character string displayed by the display unit 1411, the terminal device 100 according to the embodiment can update the display mode of the character string if the content obtained by interpreting the result of voice recognition corresponds to the character string, thereby improving the convenience of voice input.

[0089] In addition, in the terminal device 100 according to the embodiment, for example, when there is a part of a character string whose display mode has not been updated, the presentation unit 1414 presents content for inputting information corresponding to the part. When a character string is not displayed in an area included in the character string and in which a character string inputted through the user's voice is displayed, the presentation unit 1414 presents content for inputting information corresponding to the area. When there are a plurality of candidates for character strings based on the results of speech recognition that are candidates for character strings to be displayed in an area included in the character string and in which a character string inputted through the user's voice is displayed, the presentation unit 1414 presents content for selecting one of the candidates for character strings.

[0090] As a result, the terminal device 100 according to the embodiment can prompt the user to input voice again when the user's speech is not correctly recognized and part of the character string displayed on the display unit 1411 is not updated, or when the user does not speak part of the character string displayed on the display unit 1411 (such as a required field), thereby improving the convenience of voice input.

[0091] [6. Hardware Configuration] Moreover, the terminal device according to each of the above-described embodiments is realized by a computer 1000 having a configuration as shown in Fig. 6, for example. The terminal device 100 will be described below as an example. Fig. 6 is a hardware configuration diagram showing an example of a computer that realizes the functions of the terminal device. The computer 1000 has a CPU 1100, a ROM 1200, a RAM 1300, a HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0092] The CPU 1100 operates and controls each unit based on a program stored in the ROM 1200 or the HDD 1400. The ROM 1200 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, programs that depend on the hardware of the computer 1000, and the like.

[0093] The HDD 1400 stores programs executed by the CPU 1100, data used by such programs, etc. The communication interface 1500 receives data from other devices via a communication network 500 (corresponding to the network N in the embodiment) and sends the data to the CPU 1100, and also transmits data generated by the CPU 1100 to other devices via the communication network 500.

[0094] The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output interface 1600. The CPU 1100 acquires data from the input devices via the input / output interface 1600. The CPU 1100 also outputs data generated via the input / output interface 1600 to the output devices.

[0095] Media interface 1700 reads a program or data stored in recording medium 1800 and provides it to CPU 1100 via RAM 1300. CPU 1100 loads the program from recording medium 1800 onto RAM 1300 via media interface 1700 and executes the loaded program. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0096] For example, when the computer 1000 functions as the terminal device 100, the CPU 1100 of the computer 1000 executes a program loaded onto the RAM 1300 to realize the function of the control unit 140. In addition, the HDD 1400 stores each piece of data in the storage device of the terminal device 100. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, these programs may be obtained from another device via a predetermined communication network.

[0097] [7. Other] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be embodied in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the forms described in the Disclosure of the Invention section.

[0098] Further, the configuration of the above-described terminal device 100 can be flexibly changed, for example, by calling an external platform or the like using an API (Application Programming Interface) or network computing to realize some functions.

[0099] Furthermore, the term "unit" in the claims may be read as "means" or "circuit," etc. For example, a display unit may be read as a display means or a display circuit. [Explanation of symbols]

[0100] 100 Terminal Equipment 110 Communications Department 120 Input section 130 Output section 140 Control section 1411 Display section 1412 Reception 1413 Update Department 1414 Presentation section 1415 Transmitter 142 Application No. 1 143 Second Application

Claims

1. A display step of displaying on a screen predetermined character strings as search query candidates corresponding to a service used by a user, the predetermined character strings displayed on the screen including a first display area in which types of information that can be input via voice are displayed in advance, thereby suggesting search queries for the service used by the user; a receiving step of receiving a search query input from the user via voice; an updating step of changing information in the result of the voice recognition corresponding to the first display area, the information being included in the predetermined character string, into a type of information based on a result of the voice recognition of the search query received in the receiving step, and displaying the information in the first display area, and updating a display mode of a portion of the predetermined character string that matches the result of the voice recognition; An update program characterized by causing a computer to execute the following:

2. The display procedure includes: Displaying the predetermined character string corresponding to the search function provided by the service The update program according to claim 1 .

3. The display procedure includes: Displaying the predetermined character string indicating a searchable target using a search function provided by the service 3. The update program according to claim 1 or 2.

4. The update procedure comprises: updating a display mode of a portion of the predetermined character string that matches the result of the voice recognition; The update program according to any one of claims 1 to 3.

5. The update procedure comprises: A part of the predetermined character string that matches the result of the voice recognition is highlighted.

5. The update program according to claim 4.

6. The predetermined character string displayed on the screen further includes a second display area in which a plurality of character strings having a predetermined correlation are displayed in advance, The update procedure comprises: Among the plurality of character strings previously displayed in the second display area, a character string that matches the result of the voice recognition is displayed in an updated manner. The update program according to any one of claims 1 to 5.

7. a first presentation step of presenting content for inputting information corresponding to a portion of the predetermined character string whose display mode has not been updated when the portion exists in the predetermined character string; 7. The update program according to claim 1, further comprising causing a computer to execute the steps of:

8. A second presentation step of presenting content for inputting information corresponding to the first display area when information input via the user's voice is not displayed in the first display area.

8. The update program according to claim 1, further comprising causing a computer to execute the steps of:

9. A third presentation step of presenting content for selecting one of a plurality of information candidates to be displayed in the first display area based on the results of the speech recognition, when the plurality of information candidates are present.

9. The update program according to claim 1, further comprising causing a computer to execute the steps of:

10. a first transmission step of transmitting the search query received in the reception step to an external device that executes a process of interpreting the content of the voice; Further executing the following on the computer, The update procedure comprises: A result of the interpretation by the external device is used as a result of speech recognition of the search query accepted by the accepting step. The update program according to any one of claims 1 to 9.

11. a second transmission step of transmitting, when a transmission request for the predetermined character string updated by the update step and displayed on the screen is received from the user, the predetermined character string updated by the update step and displayed on the screen as a search query to an external device that executes a search process corresponding to the service.

11. The update program according to claim 1, further comprising:

12. A display unit which displays on a screen predetermined character strings as candidates for search queries corresponding to a service used by a user, the predetermined character strings displayed on the screen including a first display area in which types of information which can be input via voice are displayed in advance, thereby suggesting search queries for the service used by the user; A reception unit that receives a search query input from the user via voice; an update unit that changes information in the result of the voice recognition corresponding to the first display area and included in the predetermined character string to a type of information based on a result of the voice recognition of the search query accepted by the acceptance unit, and displays the information in the first display area, and updates a display mode of a portion of the predetermined character string that matches the result of the voice recognition. An update device comprising:

13. 1. A computer implemented update method, comprising: a display step of displaying on a screen a character string determined in advance as a search query candidate corresponding to a service used by the user, the character string determined in advance including a first display area in which a type of information that can be input via voice is displayed in advance, thereby suggesting a search query in the service used by the user; a receiving step of receiving a search query input from the user via voice; an updating step of changing information in the result of the voice recognition corresponding to the first display area, the information being included in the predetermined character string, into a type of information based on the result of the voice recognition of the search query received in the receiving step, and displaying the changed type of information in the first display area, and updating a display mode of a portion of the predetermined character string that matches the result of the voice recognition; An update method comprising:

Images