Interaction method, server device, and program

Abstract

To provide a technique for making a proactive action of a virtual assistant the most desirable for a user.SOLUTION: In a dialogue system 1, a server device performs the steps of: acquiring a setting expression including a query and a condition; storing the query and the condition in a memory; and executing control for outputting an inquiry expression including the query in response to generation of a situation specified by the condition, in order to function as a virtual assistant.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] TECHNICAL FIELD This disclosure relates to computer-based user interaction. [Background technology]

[0002] Various studies have been conducted on the proactive behavior of virtual assistants. For example, see Non-Patent Document 1 (Maria Schmidt and two others, "How Users React to Proactive Voice Assistant Behavior While Driving", [online], May 11, 2020, [searched June 13, 2022], Internet<URL: https: / / aclanthology.org / 2020.lrec-1.61 / > ) examines the cognitive load on drivers for passive and active voice assistant actions. Non-Patent Document 2 (O. Miksik et al., "Building Proactive Voice Assistants: When and How (not) to Interact", [online], May 4, 2020, [searched June 13, 2022], Internet<URL: https: / / arxiv.org / pdf / 2005.01322.pdf> ) examines the appropriate timing for proactively triggering voice assistant behavior. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Maria Schmidt, 2 others, “How Users React to Proactive Voice Assistant Behavior While Driving”, [online], May 11, 2020, [Retrieved June 13, 2020], Internet<URL: https: / / aclanthology.org / 2020.lrec-1.61 / > [Non-patent document 2] O. Miksik, 18 others, “Building Proactive Voice Assistants: When and How (not) to Interact”, [online], May 4, 2020, [Retrieved June 13, 2020], Internet<URL: https: / / arxiv.org / pdf / 2005.01322.pdf> Summary of the Invention [Problem to be solved by the invention]

[0004] However, no study has been conducted to date on optimizing the proactive behavior of virtual assistants.

[0005] The present disclosure has been devised in light of the above circumstances, and provides a technique for optimizing proactive actions by a virtual assistant for a user. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, there is provided an interaction method implemented by a server device, the interaction method comprising: acquiring a set expression including a query and a condition; storing the query and the condition in a memory; and, in response to the occurrence of a situation specified by the condition, performing control to output an inquiry expression including the query. [Effects of the Invention]

[0007] According to the present disclosure, an interaction method is provided that acts as a virtual assistant to take proactive actions that are optimal for the user. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a dialogue system according to an embodiment. [Figure 2]2 is a diagram schematically illustrating an example of the operation of the dialogue system 1. FIG. [Figure 3] FIG. 2 is a diagram illustrating an example of the hardware configuration of a main server 100. [Figure 4] FIG. 2 is a diagram illustrating an example of the hardware configuration of a user terminal 200. [Figure 5] FIG. 10 is a diagram illustrating an example of the contents of registration information. [Figure 6] FIG. 10 is a diagram showing a first specific example of a data structure of registration information together with setting expressions. [Figure 7] FIG. 10 is a diagram showing a second specific example of the data structure of registration information together with setting expressions. [Figure 8] FIG. 10 is a diagram showing a third specific example of the data structure of registration information together with setting expressions. [Figure 9] FIG. 10 is a diagram illustrating a first specific example of a data structure of a frame of a polling query. [Figure 10] FIG. 10 is a diagram illustrating a first specific example of a data structure of a polling query. [Figure 11] FIG. 10 is a diagram illustrating a second specific example of the data structure of a polling query frame. [Figure 12] FIG. 10 is a diagram illustrating a second specific example of the data structure of a polling query. [Figure 13] FIG. 10 is a diagram illustrating a third specific example of the data structure of a polling query frame. [Figure 14] FIG. 10 is a diagram illustrating a third specific example of the data structure of a polling query. [Figure 15] FIG. 10 is a diagram for explaining a method for generating a query expression. [Figure 16] 1 is a flowchart of a process performed by the dialogue system 1. [Figure 17] 1 is a flowchart of a process performed by the dialogue system 1. [Figure 18] 1 is a flowchart of a process performed by the dialogue system 1. [Figure 19] 1 is a flowchart of a process performed by the dialogue system 1. DETAILED DESCRIPTION OF THE INVENTION

[0009] An embodiment of a dialogue system for implementing the dialogue method will be described below with reference to the drawings. In the following description, the same parts and components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, their description will not be repeated.

[0010] [1. Dialogue system configuration] Fig. 1 is a diagram showing an example of the configuration of a dialogue system according to an embodiment. The dialogue system 1 includes a main server 100, a user terminal 200, an API (Application Programming Interface) server 800, and a control server 900. In Fig. 1, the number of each of the main server 100, the user terminal 200, the API server 800, and the control server 900 is one, but the number of each of these is not limited to one in the technology described in the present disclosure.

[0011] In the dialogue system 1, the main server 100 and the user terminal 200 function as a virtual assistant for the user 300. A server application program (server app) is installed in the main server 100 to function as a virtual assistant. A terminal application program (terminal app) is installed in the user terminal 200 to function as a virtual assistant. The user terminal 200 may be a smartphone, a smart speaker, an information processing device mounted on an automobile, or an information processing device mounted on a home appliance.

[0012] In order to function as a virtual assistant, the main server 100 sends a request to the API server 800 as necessary, receives a response in response to the request from the API server 800, and uses the received response. In order to function as a virtual assistant, the main server 100 sends an instruction to the control server 900 as necessary.

[0013] The API server 800 is realized, for example, as a server that provides weather information. The control server 900 is realized as a server for controlling the operation of various devices. In one example, the control server 900 controls the operation of elements in a vehicle (e.g., air conditioner, radio) by communicating with a computer installed in the vehicle. In another example, the control server 900 controls the operation of a home appliance by communicating with a computer installed in the home appliance. Note that although FIG. 1 illustrates the main server 100, the user terminal 200, the API server 800, and the control server 900 as communicating via the network 500, they do not necessarily need to communicate via the network 500. For example, both the user terminal 200 and the control server 900 may be installed in the vehicle, and the user terminal 200 and the control server 900 may be configured to communicate directly with each other.

[0014] FIG. 2 is a diagram schematically illustrating an example of the operation of the dialogue system 1. The dialogue system 1 acquires a set expression from the user 300. The set expression includes information specifying a query and a condition. The dialogue system 1 monitors whether a situation specified by the above condition occurs. When the situation specified by the above condition occurs, the dialogue system 1 outputs a query expression including the above query. This allows the dialogue system 1 to start a proactive dialogue with the user 300 when a situation specified by the condition set by the user 300 occurs. The dialogue system 1 can also start such a proactive dialogue with a query expression including the query set by the user 300.

[0015] [2. Hardware configuration (main server)] 3 is a diagram showing an example of the hardware configuration of the main server 100. The main server 100 includes a CPU (Central Processing Unit) 101, a communication I / F (interface) 102, and storage 103. The storage 103 includes a program area 1031 for storing various programs, and a data area 1032 for storing various data.

[0016] The CPU 101 executes various calculations by executing programs stored in the storage 103 or an external storage device. The communication I / F 102 is realized by, for example, a network card, and allows the main server 100 to communicate with other devices in the dialogue system 1. In the dialogue system 1, the API server 800 and the control server 900 may have the same hardware configuration as the main server 100 shown in FIG. 3.

[0017] [3. Hardware configuration (user terminal)] 4 is a diagram showing an example of the hardware configuration of user terminal 200. User terminal 200 includes a CPU 201, a display 202, a microphone 203, a speaker 204, an input device 205, a communication I / F 206, and storage 207. Storage 207 includes a program area 2071 for storing various programs, and a data area 2072 for storing various data.

[0018] The CPU 201 executes various calculations by executing programs stored in the storage 207 or an external storage device.

[0019] The display 202 displays a screen instructed by the CPU 201. The microphone 203 inputs input voice to the CPU 201. The speaker 204 outputs voice instructed by the CPU 201. The input device 205 is realized by, for example, physical keys and / or a touch sensor, and accepts information input from the user. The communication I / F 206 is realized by, for example, a network card, and allows the user terminal 200 to communicate with other devices in the interactive system 1.

[0020] [4. Processing of setting expressions] In the dialogue system 1, when the main server 100 receives a set expression from a user, it extracts a query and a condition from the set expression and stores the query and the condition as registered information in the storage 103. The process of extracting the query and the condition from the set expression and storing them as registered information will be described with reference to Figs. 5 to 7.

[0021] (Contents of the data to be registered) FIG. 5 is a diagram showing an example of the contents of registration information. As shown as "Key" in FIG. 5, the stored data includes seven types of items ("Query Text," "Query Type," "Query Domain," "Trigger Type," "Trigger Value," "Trigger Repeat," and "Trigger Rule"). "Query Text," "Query Type," and "Query Domain" are information that make up the query. "Trigger Type," "Trigger Value," "Trigger Repeat," and "Trigger Rule" are information that make up the condition. In FIG. 5, explanations for each item are added as "Notes." Each item will be explained below.

[0022] "Query Text" identifies the text of the query. "Query Type" identifies the type of query. In one implementation, "Question" and "Command" are defined as query types. "Question" means a query that expresses what the user wants to ask. "Command" means a query that expresses what the user wants to accomplish.

[0023] "Query Domain" identifies the domain to which the query belongs. In one implementation, domain refers to the field that the query content represents. In the example of Figure 5, example domains are "weather," "home appliance control," and "automobile control."

[0024] "Trigger Type" identifies the type of condition. In the example of Figure 5, "time," "temperature," and "speed" are shown as examples of the type of condition.

[0025] "Trigger Value" identifies the value that makes up the condition. The type of value that makes up the condition depends on the type of condition ("Trigger Type"). For example, if "Trigger Type" is time, "Trigger Value" is a value corresponding to the unit of time; if "Trigger Type" is temperature, "Trigger Value" is a value corresponding to the unit of temperature; if "Trigger Type" is speed, "Trigger Value" is a value corresponding to the unit of speed.

[0026] "Trigger Repeat" identifies how often the situation specified by the condition occurs. In the example of FIG. 5, the frequencies at which the condition occurs are indicated as "daily," "weekly (X day of the week)," "monthly (X day of the month)," and "every hour." When the registration information includes a value for the item "Trigger Repeat," the condition included in the registration information specifies how often the situation occurs.

[0027] The "Trigger Rule" identifies the rule in which the "Trigger Value" is utilized. In one implementation, the rules are defined as "equal to," "greater than or equal to," and "less than or equal to."

[0028] (First example of registered data) FIG. 6 is a diagram showing a first specific example of the data structure of the registration information together with setting expressions.

[0029] The data structure in Figure 6 contains data that constitutes the query and conditions extracted from the set expression "I want to ask, 'What's the weather like today,' every day at 8 AM."

[0030] More specifically, the data structure in FIG. 6 includes "How's the weather today?" as "Query Text."

[0031] In one implementation example, natural language interpretation is performed on the set expression, and the part representing the query, "How is the weather today?", is extracted from the set expression as "Query Text."

[0032] For example, the main server 100 stores a grammar for natural language interpretation of a set expression. An example of the grammar is "I would like to ask [first phrase] and [second phrase]." In this grammar, each of [first phrase] and [second phrase] is intended to represent any text. If the set expression conforms to this grammar, the part corresponding to [first phrase] is extracted as the part representing the query.

[0033] The data structure in Figure 6 includes "Question" as the "Query Type." In one implementation, "Question" is identified as the type of query if the set expression includes the phrase "I would like to ask."

[0034] The data structure in FIG. 6 includes "weather" as the "Query Domain." In one implementation example, after a query is extracted from a set expression, a grammar to which the query applies is identified, and a domain to which the identified grammar belongs is identified. The domain identified in this way is specified as the value of the "Query Domain." Note that in the dialogue system 1, for example, a database indicating the domain to which each of multiple grammars used in natural language interpretation belongs is stored in the storage 103.

[0035] The data structure in Figure 6 includes "Time" as the "Trigger Type." In one implementation example, after a query is extracted from the set expression, the value of "Trigger Type" is determined by performing natural language interpretation on the portion of the set expression other than the query. For example, when the query includes the string "at XX o'clock" (where "XX" represents any number), "Time" is determined as the value of "Trigger Type."

[0036] The data structure in Figure 6 includes "08:00" as the "Trigger Value." In one implementation, the value of "Trigger Value" is determined using the phrase used to determine the value of "Trigger Type." More specifically, if the "Trigger Type" value "Time" is determined because the setting expression includes the phrase "at 8:00 AM," the value of "Trigger Type" is determined to be the numeric value "08:00" corresponding to the time portion of the phrase, "8:00 AM."

[0037] The data structure in Figure 6 includes "Every Day" as the "Trigger Repeat." In one implementation, if a string representing frequency is included in a portion of the set expression other than the query, the string is identified as the "Trigger Repeat."

[0038] The data structure in Figure 6 includes "Equivalent" as the "Trigger Rule." In one implementation, the value of "Trigger Value" is identified using the phrase used to identify the value of "Trigger Type." In the example in Figure 6, the phrase "at 8 AM" indicates the time "8 AM" itself, so "Equivalent" is identified as the "Trigger Rule."

[0039] (Second example of registered data) FIG. 7 is a diagram showing a second specific example of the data structure of the registration information together with setting expressions.

[0040] The data structure in Figure 7 includes data constituting the query and conditions extracted from the setting expression "I want to turn on the air conditioner when the temperature is 25°C or higher."

[0041] More specifically, the data structure in FIG. 7 includes "I want to turn on the air conditioner" as "Query Text."

[0042] In one implementation example, natural language interpretation is performed on the setting expression, and the part representing the query, "I want to turn on the air conditioner," is extracted from the setting expression as "Query Text."

[0043] For example, the main server 100 stores a grammar for natural language interpretation of a set expression. An example of the grammar is "When it becomes [first phrase], I want to do [second phrase]." In this grammar, each of [first phrase] and [second phrase] is intended to represent any text. If the set expression conforms to this grammar, the part corresponding to [second phrase] is extracted as the part representing the query.

[0044] The data structure in Figure 7 includes "Command" as the "Query Type." In one implementation, "Command" is identified as the query type when the set expression does not include the phrase "I would like to ask."

[0045] The data structure in Figure 7 includes "Home Appliance Control" as the "Query Domain." In one implementation example, after a query is extracted from the configuration expression, a grammar to which the query applies is identified, and a domain to which the identified grammar belongs is identified. The identified domain is specified as the value of the "Query Domain."

[0046] The data structure in Figure 7 includes "Temperature" as the "Trigger Type." In one implementation example, after a query is extracted from the set expression, the value of "Trigger Type" is determined by performing natural language interpretation on the portion of the set expression other than the query. For example, if the query includes the string "When the temperature reaches or exceeds XX degrees Celsius" (where "XX" represents any number), "Temperature" is determined as the value of "Trigger Type."

[0047] The data structure in FIG. 7 includes "25" as the "Trigger Value." In one implementation, the value of "Trigger Value" is determined using the phrase used to determine the value of "Trigger Type." More specifically, if the value of "Temperature" for "Trigger Type" is determined by including the phrase "when the temperature reaches 25°C or higher" in the setting expression, the value of "Trigger Value" is determined to be the numeric value "25," which corresponds to the temperature portion of the phrase, "25°C."

[0048] The data structure in Figure 7 does not include a value for "Trigger Repeat." This is because, in the example in Figure 7, the part of the setting expression other than the query does not include a string representing frequency.

[0049] The data structure in Figure 7 includes "greater than or equal to" as the "Trigger Rule." In one implementation, the value of the "Trigger Rule" is identified using the phrase used to identify the value of the "Trigger Type." In the example in Figure 7, the phrase "when it reaches 25°C or higher" includes "greater than or equal to," so "greater than or equal to" is identified as the "Trigger Rule."

[0050] (Third example of registered data) FIG. 8 is a diagram showing a third specific example of the data structure of the registration information together with setting expressions.

[0051] The data structure in Figure 8 contains data that constitutes the query and conditions extracted from the setting expression "I want the radio to turn on when my car's speed drops below 40 km / h."

[0052] 8 includes "I want to turn on the radio" as "Query Text." In one implementation example, natural language interpretation is performed on the setting expression to extract the portion of the setting expression that represents the query, "I want to turn on the radio," as "Query Text."

[0053] The data structure in Figure 8 includes "Command" as the "Query Type." In one implementation, "Command" is identified as the query type when the set expression does not include the phrase "I would like to ask."

[0054] The data structure in Figure 8 includes "Automotive Control" as the "Query Domain." In one implementation example, after a query is extracted from a set expression, a grammar to which the query applies is identified, and a domain to which the identified grammar belongs is identified. The identified domain is specified as the value of the "Query Domain."

[0055] The data structure in Figure 8 includes "speed" as the "Trigger Type." In one implementation example, after a query is extracted from the set expression, the value of "Trigger Type" is determined by performing natural language interpretation on the portion of the set expression other than the query. For example, when the query includes the string "When the speed falls below XX kilometers per hour" (where "XX" represents any number), "speed" is determined as the value of "Trigger Type."

[0056] The data structure in FIG. 8 includes "40" as the "Trigger Value." In one implementation, the value of "Trigger Value" is determined using the phrase used to determine the value of "Trigger Type." More specifically, if the value of "Trigger Type" is determined by including the phrase "when the speed drops below 40 km / h" in the set expression, the value of "Trigger Type" is determined to be the numeric value "40," which corresponds to the portion of the phrase that represents the speed, "40 km" (40 kilometers per hour).

[0057] The data structure in Figure 8 does not include a value for "Trigger Repeat." This is because, in the example in Figure 8, the part of the setting expression other than the query does not include a string representing frequency.

[0058] The data structure in FIG. 8 includes "less than" as the "Trigger Rule." In one implementation, the value of the "Trigger Rule" is identified using the phrase used to identify the value of the "Trigger Type." In the example in FIG. 8, the phrase "when the speed drops below 40 km / h" includes "less than," so "less than" is identified as the "Trigger Rule."

[0059] The condition specified by the data structure shown in Fig. 8 specifies that the speed of the automobile is 40 kilometers per hour or less. In this case, the situation specified by the condition is the situation in which the speed of the automobile is 40 kilometers per hour or less, which is an example of a situation related to the automobile.

[0060] [5. Output of query expressions based on conditions] As described with reference to FIG. 2, the dialogue system 1 outputs a query expression in response to the occurrence of a situation specified by a condition.

[0061] In one implementation example, the dialogue system 1 periodically collects data for determining whether a situation specified by a condition has occurred, and determines whether the situation has occurred. As an example of collecting the data, the main server 100 itself collects the data. In another example, the user terminal 200 collects the data and provides it to the main server 100. More specifically, the user terminal 200 provides the data by periodically sending a polling query to the main server 100.

[0062] Regarding the transmission of a polling query, the main server 100 uses part of the data group stored in the storage 103 as registration information to create a polling query frame and transmits the frame to the user terminal 200. The user terminal 200 periodically fills the frame with data to generate a polling query and transmits it to the main server 100. A specific example of a polling query will be described below.

[0063] (First example of a polling query and its frame) 9 is a diagram showing a first specific example of the data structure of a polling query frame. The example in Fig. 9 corresponds to the case where the expression shown in Fig. 6 is input as the setting expression.

[0064] In the example of FIG. 9, the polling query frame includes the message "check_proactive_query_trigger" and the string "RequestInfo:{ExtraValue: {Type: Time, Value: [###]}}." The message "check_proactive_query_trigger" instructs the system to use the data specified by the subsequent string to determine whether a situation specified by the condition in the registration information has occurred. In the string "RequestInfo:{ExtraValue: {Type: Time, Value: [###]}}," "Type: Time" indicates the type of data to be collected, more specifically, "time." The main server 100 sets the type specified as "Trigger Type" in the registration information as the data type in this message. "Value: [###]" indicates the portion to be filled with the collected data. More specifically, "###" is replaced with the collected data.

[0065] Fig. 10 is a diagram showing a first specific example of the data structure of a polling query. The data structure of Fig. 10 is generated using the frame shown in Fig. 9. More specifically, the data structure of Fig. 10 is generated in response to user terminal 200 acquiring the time of 7:50 as data for determining whether a situation specified by a condition has occurred. Even more specifically, the data structure of Fig. 10 is generated by replacing "###" in the data structure shown in Fig. 9 with the collected data ("07:50" representing the time of 7:50).

[0066] As described above, the polling query shown in FIG. 10 instructs that the data representing the time "07:50" be used to determine whether the situation specified by the condition in the registration information has occurred.

[0067] (Second example of a polling query and its frame) Fig. 11 is a diagram showing a second specific example of the data structure of a polling query frame. The example in Fig. 11 corresponds to the case where the expression shown in Fig. 7 is input as the setting expression.

[0068] 11, the polling query frame includes the message "check_proactive_query_trigger" and the string "RequestInfo:{ExtraValue: {Type: Temperature, Value: [###]}}". "Type: Temperature" indicates the type of data to be collected, more specifically, "temperature".

[0069] FIG. 12 is a diagram showing a second specific example of the data structure of a polling query. The data structure of FIG. 12 is generated using the frame shown in FIG. 11. More specifically, the data structure of FIG. 12 is generated in response to user terminal 200 acquiring data on a temperature of 30°C as data for determining whether a situation specified by a condition has occurred. User terminal 200 acquires the data by, for example, communicating with a device that measures temperature. Then, the data structure of FIG. 12 is generated by replacing "###" in the data structure shown in FIG. 11 with the collected data (the temperature of 30°C, "30.0").

[0070] As described above, the polling query shown in FIG. 12 instructs that data representing the temperature "30.0" be used to determine whether or not the situation specified by the condition in the registration information has occurred.

[0071] (Third example of a polling query and its frame) Fig. 13 is a diagram showing a third specific example of the data structure of a polling query frame. The example in Fig. 13 corresponds to the case where the expression shown in Fig. 8 is input as the setting expression.

[0072] 13, the polling query frame includes the message "check_proactive_query_trigger" and the string "RequestInfo:{ExtraValue: {Type: Speed, Value: [###]}}". "Type: Speed" indicates the type of data to be collected, and more specifically, the "(automobile) speed".

[0073] FIG. 14 is a diagram showing a third specific example of the data structure of a polling query. The data structure of FIG. 14 is generated using the frame shown in FIG. 13. More specifically, the data structure of FIG. 14 is generated in response to user terminal 200 acquiring data on a speed of 60 kilometers per hour as data for determining whether a situation specified by a condition has occurred. User terminal 200 is realized, for example, as a computer installed in an automobile, and acquires the speed data by communicating with the automobile's speedometer. The data structure of FIG. 14 is then generated by replacing "###" in the data structure shown in FIG. 13 with the collected data (the data "60.0" representing a speed of 60 kilometers per hour).

[0074] As described above, the polling query shown in FIG. 14 instructs that the data representing the speed "60.0" be used to determine whether the situation specified by the condition in the registration information has occurred.

[0075] [6. Inquiry Expressions] 15 is a diagram for explaining a method for generating a query expression. With reference to FIG. 15, an example of a method for generating a query expression including a query will be described.

[0076] The table shown in Fig. 15 includes an item called "occurrence of a situation specified by a condition." This item indicates whether or not the occurrence of the situation specified by the condition has occurred. The table shown in Fig. 15 further includes three items (query type, query expression generation method, and query expression example).

[0077] In the example of FIG. 15, if the above situation does not occur (if the value of the item "occurrence of situation specified by condition" is "none"), no query expression is generated regardless of the type of query.

[0078] In the example of FIG. 15, when the above situation occurs (when the value of the item "occurrence of situation specified by condition" is "yes"), a query expression is generated in a manner (generation method) according to the type of query. More specifically, when the type of query is "question," the query expression is generated by adding "Do you want to ask?" to the query text registered as the value of "Query Text" in the registration information. On the other hand, when the type of query is "command," the query expression is generated by adding "Is that so?" to the query text registered as the value of "Query Text" in the registration information. In other words, the character string (content) added to the query text to generate the query expression differs depending on the type of query.

[0079] An example of a query expression generated when the query type is "question" is "Would you like to ask, 'What's the weather like today?'" This query expression corresponds to the example shown in FIG. 6. In the example of FIG. 6, the query text is "What's the weather like today?' and the query type is "question." The above query expression is generated by adding "Would you like to ask?" to "What's the weather like today?"

[0080] An example of a query expression generated when the query type is "command" is "Do you want to turn on the air conditioner?". This query expression corresponds to the example shown in FIG. 7. In the example of FIG. 7, the query text is "Do you want to turn on the air conditioner" and the query type is "command". The above query expression is generated by adding "Do you want to turn on the air conditioner?" to "Do you want to turn on the air conditioner?"

[0081] The generated query expression may be a question that requests an affirmative answer (for example, "yes") or a negative answer (for example, "no") from the user 300.

[0082] [7. Processing flow] 16 to 19 are flowcharts of the processing performed by the dialogue system 1. FIGS. 16 to 19 show processing performed in the main server 100 and processing performed in the user terminal 200. In one implementation example, the main server 100 performs the above processing by having the CPU 101 execute a server application. In one implementation example, the user terminal 200 performs the above processing by having the CPU 201 execute a terminal application.

[0083] 16, in step S200, user terminal 200 determines whether a wake word has been input by user 300. User terminal 200 repeats the control of step S200 until it determines that a wake word has been input (NO in step S200), and when it determines that a wake word has been input (YES in step S200), it proceeds to step S202.

[0084] In step S202, the user terminal 200 acquires the voice input by the user 300 after the wake word.

[0085] In step S204, the user terminal 200 transmits the voice acquired in step S202 to the main server 100.

[0086] In step S100, the main server 100 receives the voice transmitted from the user terminal 200 in step S204.

[0087] In step S102, the main server 100 determines whether the voice received in step S100 includes a message (registration message) requesting registration of the registration information. The registration message is an example of a "specific message" in this disclosure. An example of a registration message is "Let me set a query and conditions." In one implementation example, the main server 100 generates text of the voice using speech recognition technology and performs the determination in step S102 based on whether the generated text includes the text of the registration message. If the main server 100 determines that the voice includes the registration message, the control proceeds to step S104 (YES in step S102); otherwise, the control proceeds to step S138 (NO in step S102).

[0088] Referring to FIG. 17, in step S138, main server 100 performs an operation according to the received voice, and ends the process.

[0089] 16, in step S104, the main server 100 instructs the user terminal 200 to output a message (prompting message) prompting the user to input a setting expression. An example of the prompting message is "Tell me your query and conditions."

[0090] In step S206, the user terminal 200 outputs the prompting message in response to the instruction in step S104. One example of outputting the prompting message is to utter a voice representing the prompting message.

[0091] In step S208, the user terminal 200 acquires a voice input from the user 300. The voice input is an utterance made by the user 300 after the prompting message is output, and is usually a set expression.

[0092] In step S210, the user terminal 200 transmits the voice acquired in step S208 to the main server 100.

[0093] In step S106, the main server 100 receives the voice transmitted in step S210.

[0094] In step S108, the main server 100 performs speech recognition on the speech received in step S106, thereby obtaining text corresponding to the speech.

[0095] In step S110, the main server 100 performs natural language interpretation on the text obtained in step S108.

[0096] In step S112, the main server 100 uses the results of the natural language interpretation in step S110 to extract a query (the value of "Query Text") and conditions (the values ​​of "Trigger Type," "Trigger Value," "Trigger Repeat," and "Trigger Rule") from the set expression (the voice input in step S208).

[0097] In step S114, the main server 100 identifies the type of query (the value of "Query Type") based on the set expression (the voice input in step S208).

[0098] In step S116, the main server 100 identifies the grammar to which the query corresponds based on the set expression (the speech input in step S208).

[0099] In step S118, the main server 100 identifies the domain of the query (the value of "Query Domain") based on the grammar identified in step S116.

[0100] 18, in step S120, the main server 100 determines whether the domain identified in step S118 is included in the list. The list refers to a list of domains applicable to the user 300. In one implementation example, the list is stored in the storage 103. If the main server 100 determines that the domain is included in the list (YES in step S120), the main server 100 proceeds to step S122. If the main server 100 determines that the domain is not included in the list (NO in step S120), the main server 100 proceeds to step S140.

[0101] 19, in step S140, main server 100 instructs user terminal 200 to notify that the setting has failed. After that, main server 100 ends the process.

[0102] In step S226, the user terminal 200 notifies the user of the setting failure in response to the instruction in step S140. One example of the notification of the setting failure is to output the message "That query is not applicable." Another example is to output the message "Please enter another query."

[0103] Returning to FIG. 18, in step S122, the main server 100 associates the data extracted or identified in steps S112 to S118 with the user 300 as registration information and stores it in the storage 103.

[0104] As described above, in step S120, the main server 100 determines whether the domain is included in the list, and if it determines that the domain is not included, ends the process without storing the registration information data in step S122. In this sense, step S120 is an example of a step of avoiding registering the registration information (query and conditions) in memory.

[0105] In step S124, the main server 100 generates a polling query frame using the query type identified in step S114.

[0106] In step S126, the main server 100 transmits the polling query frame generated in step S124 to the user terminal 200.

[0107] In step S212, the user terminal 200 receives the polling query frame transmitted in step S126.

[0108] In step S214, the user terminal 200 stores in the storage 207 the polling query frame received in step S212.

[0109] In step S216, the user terminal 200 collects data for the polling query (for example, data represented by "###" in FIG. 9, FIG. 11, or FIG. 13).

[0110] In step S218, the user terminal 200 generates a polling query using the data collected in step S216, and transmits the generated polling query to the main server 100.

[0111] In step S128, the main server 100 receives the polling query sent in step S218.

[0112] In step S130, main server 100 uses the data included in the polling query to determine whether the situation specified by the conditions in the registration information has occurred. If main server 100 determines that the situation has occurred, control proceeds to step S132 (YES in step S130); otherwise, main server 100 ends the process (NO in step S130).

[0113] An example of a situation identified by the registration information is when it is 8:00 a.m. If the data included in the polling query indicates 7:50 a.m., 8:00 a.m. has not yet arrived, and the main server 100 determines that the situation has not occurred. On the other hand, if the data included in the polling query indicates 8:00 a.m., the main server 100 determines that the situation has occurred.

[0114] Another example of a situation identified by the registration information is when the temperature is 25° C. or higher. If the data included in the polling query indicates that the temperature is 20° C., the main server 100 determines that this situation has not occurred. On the other hand, if the data included in the polling query indicates that the temperature is 30° C., the main server 100 determines that this situation has occurred.

[0115] Another example of a situation identified by the registration information is when the vehicle's speed drops below 40 kilometers per hour. If the data included in the polling query indicates that the vehicle's speed is 60 kilometers per hour, the main server 100 determines that the situation has not occurred. On the other hand, if the data included in the polling query indicates that the vehicle's speed is 30 kilometers per hour, the main server 100 determines that the situation has occurred.

[0116] In step S132, the main server 100 generates a query expression using the registration information.

[0117] In step S134, the main server 100 instructs the user terminal 200 to output the query expression generated in step S132.

[0118] In step S136, the main server 100 uses the query expression to update the conversation state with the user 300 in the storage 103. Even if the user 300's response to the query expression is only affirmative or negative, the main server 100 can refer to the updated conversation state and perform an operation according to the response of the user 300. Thereafter, the main server 100 ends the process.

[0119] In step S220, the user terminal 200 receives the instruction in step S134. In step S222, the user terminal 200 outputs the query expression, and returns control to step S202 (FIG. 16).

[0120] In the processing described with reference to FIGS. 16 to 19, the user terminal 200 periodically collects data in step S216 and transmits a polling query in step S218.

[0121] [8. Specific examples of operations in dialogue system 1] In the processing described with reference to FIGS. 16 to 19, the main server 100 acquires a set expression and stores the registration information acquired from the set expression in the storage 103. Then, when a situation specified by a condition in the registration information occurs, the main server 100 instructs the user terminal 200 to output an inquiry expression including a query in the registration information in step S134. In response to this, the user terminal 200 outputs the inquiry expression in step S222. Thereafter, the user terminal 200 acquires a voice response to the inquiry expression in step S202, and transmits the voice to the main server 100 in step S204. The main server 100 receives the voice in step S100, and performs an operation according to the voice in step S138.

[0122] According to the above-described process, a user provides a server device with a query that the user desires to be output as an inquiry expression and conditions for specifying the timing at which the user desires the inquiry expression to be output as set expressions, and when a situation specified by the above conditions occurs, the user can receive proactive action by outputting an inquiry expression that includes the above query.

[0123] A specific example of the operation of the dialogue system 1 will be described below. (First specific example of operation) As a first specific example of the operation of the dialogue system 1, the operation when the registration information shown in FIG. 6 is stored will be described.

[0124] According to the example of FIG. 6, the user terminal 200 periodically transmits time data as a polling query. The main server 100 determines whether the time included in the polling query has reached 8:00 AM, and if so, instructs the user terminal 200 to output a query expression. In response to instructing the user terminal 200 to output a query expression, the main server 100 may store the instructed date in the storage 103. If the main server 100 determines that the time included in the polling query has reached 8:00 AM, the main server 100 may instruct the user terminal 200 to output a query expression, provided that the current date has not yet been stored in the storage 103.

[0125] The query phrase "Do you want to ask, 'What's the weather like today?'" is output. If the user 300 replies "Yes" to this query phrase, the main server 100 performs an operation according to this reply. More specifically, if the main server 100 receives an affirmative reply of "Yes," it refers to the conversation state stored in step S136. The conversation state is, for example, information indicating that a query registered as "Query Text" has been output. Then, in response to receiving an affirmative reply, the main server 100 processes the query registered as "Query Text." That is, to process the query "What's the weather like today?", the main server 100 queries the weather forecast API server 800 for the weather forecast for the area registered in association with the user 300. Then, it obtains a reply from the weather forecast API server 800 and instructs the user terminal 200 to output the reply.

[0126] If the user 300 answers "No," the main server 100 may instruct the user terminal 200 to output a specific message such as "OK."

[0127] After the conversation state is stored in step S136, the main server 100 may delete the conversation state from the storage 103 upon processing the query as described above or upon the establishment of a given condition. One example of the given condition is that a certain period of time has elapsed since the conversation state was stored. Another example is that after the conversation state was stored in step S136, the audio received in step S100 is a message other than a message indicating a positive answer.

[0128] As described above, the dialogue system 1 outputs the query "Would you like to ask, 'What's the weather like today?'" to the user 300 at 8:00 every day. If the user 300 replies "Yes," the dialogue system 1 provides the user 300 with a response from the weather forecast API server 800.

[0129] (Second specific example of behavior) As a second specific example of the operation of the dialogue system 1, the operation when the registration information shown in FIG. 7 is stored will be described.

[0130] According to the example of FIG. 7 , the user terminal 200 periodically transmits temperature data as a polling query. In one implementation example, the user terminal 200 acquires the temperature data by communicating with a device that measures the temperature in a room associated with the user 300. The main server 100 determines whether the temperature included in the polling query is 25° C. or higher, and if it determines that the temperature is 25° C. or higher, it instructs the user terminal 200 to output a query expression. In response to instructing the user terminal 200 to output the query expression, the main server 100 may store the time of the instruction in the storage 103. If the main server 100 determines that the temperature included in the polling query is 25° C. or higher, it may instruct the user terminal 200 to output the query expression, provided that a certain period of time has elapsed since the time the temperature was stored in the storage 103.

[0131] The query phrase "Do you want to turn on the air conditioner?" is output. If the user 300 replies "Yes" to this query phrase, the main server 100 performs an operation according to this reply. More specifically, if the main server 100 receives an affirmative reply of "Yes," it refers to the conversation state stored in step S136. The conversation state is, for example, information indicating that a query registered as "Query Text" has been output. Then, in response to receiving an affirmative reply, the main server 100 processes the query registered as "Query Text." That is, in order to process the query "I want to turn on the air conditioner," the main server 100 instructs the control server 900 for home appliance control to turn on the air conditioner registered in association with the user 300.

[0132] As described above, when the temperature in the room associated with the user 300 reaches or exceeds 25°C, the dialogue system 1 outputs the query phrase "Do you want to turn on the air conditioner?" to the user 300. If the user 300 replies "Yes," the dialogue system 1 uses the control server 900 to turn on the air conditioner associated with the user 300.

[0133] (Third specific example of behavior) As a third specific example of the operation of the dialogue system 1, the operation when the registration information shown in FIG. 8 is stored will be described.

[0134] 8, the user terminal 200 periodically transmits data on the speed of a vehicle as a polling query. In one implementation example, the user terminal 200 acquires the data on the speed of the vehicle by communicating with a device that measures the speed of the vehicle associated with the user 300. The main server 100 determines whether the speed included in the polling query is 40 km / h or less, and if it determines that the speed is 40 km / h or less, instructs the user terminal 200 to output a query expression.

[0135] The query phrase "Do you want to turn on the radio?" is output. If the user 300 replies "Yes" to this query phrase, the main server 100 performs an operation according to this reply. More specifically, if the main server 100 receives an affirmative reply of "Yes," it refers to the conversation state stored in step S136. The conversation state is, for example, information indicating that a query registered as "Query Text" has been output. Then, in response to receiving an affirmative reply, the main server 100 processes the query registered as "Query Text." That is, in order to process the query "Do you want to turn on the radio," the main server 100 instructs the control server 900 for automobile control to turn on the radio of the vehicle registered in association with the user 300.

[0136] As described above, when the speed of the car associated with the user 300 drops below 40 km / h, the dialogue system 1 outputs the query phrase "Do you want to turn on the radio?" to the user 300. If the user 300 replies "Yes," the dialogue system 1 uses the control server 900 to turn on the radio of the car associated with the user 300.

[0137] [9. Variations] In the above-described embodiment, both the setting expression and the query expression are in the form of voice, but these are not limited to voice. The setting expression may be input to the user terminal 200 as text. In this case, the user terminal 200 transmits the input text to the main server 100. The setting expression may be input directly to the main server 100 without going through the user terminal 200. The query expression may also be output as text.

[0138] In the above-described embodiment, the dialogue system 1 recognized the registration message in steps S202, S204, S100, and S102 before acquiring the set expression, and then output the prompt message in step S206. Note that the user 300 may utter the registration message and the set expression as a series of voices. After recognizing the registration message, the dialogue system 1 may treat the expression immediately following it as the set expression. In this case, output of the prompt message is not required.

[0139] The dialogue system 1 also extracts a query and a condition from the set expression by natural language interpretation of the set expression. The dialogue system 1 may display (for example, on the display 202) a user interface having multiple fields for inputting the query and the condition, respectively. The dialogue system 1 may acquire data input by the user 300 into each of the multiple fields. This allows the dialogue system 1 to acquire the registration information shown in each of FIGS. 6 to 8 without performing natural language interpretation of the set expression.

[0140] In the above-described embodiment, the dialogue system 1 may be assumed to have two or more users. Registration information corresponding to each of the two or more users may be stored in the storage 103 in association with each user. The processing described with reference to FIGS. 16 to 19 may be performed for each user. In one implementation example, the dialogue system 1 includes multiple user terminals 200, and each user terminal 200 transmits information to the main server 100 along with the user ID of each user. The main server 100 identifies the user to be processed using the user ID included in the information transmitted from the user terminal 200. Furthermore, the main server 100 may identify a list from two or more lists to be used for the determination in step S120 based on the user ID transmitted from the user terminal 200.

[0141] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. Furthermore, the inventions described in the embodiments and modifications are intended to be practiced, as far as possible, either alone or in combination. [Explanation of symbols]

[0142] 1 dialogue system, 100 main servers, 200 user terminals, 300 users, 800 API servers, 900 control servers.

Claims

1. 1. An interaction method implemented by a server device, comprising: obtaining a configuration expression including a query and a condition from a user terminal; storing the query and the condition in a memory; and executing a control step for outputting a query expression including the query in response to the occurrence of a situation specified by the condition; The control for outputting the query expression includes outputting an instruction to output the query expression to the user terminal.

2. The method of claim 1 , further comprising extracting the query and the condition from the set expression by performing natural language interpretation on the set expression.

3. The method of claim 1 or 2, wherein the step of obtaining the set expression includes receiving a sound corresponding to the set expression.

4. further comprising the step of obtaining input for a specific message; The interaction method according to claim 1 or 2, wherein the step of accepting the set expression is performed in response to the specific message being acquired from the user terminal.

5. performing natural language interpretation on the query to identify a grammar to which the query fits; identifying a domain to which the grammar belongs; determining whether the domain is registered in the list stored in the memory; The method of claim 1 or 2, further comprising: avoiding registering the query and the condition in the memory in response to the domain not being registered in the list.

6. The step of acquiring the set expression includes receiving information identifying a user among two or more users corresponding to the set expression; the list is associated with information identifying one or more users of the two or more users; The step of determining whether the domain is registered on the list includes: identifying a user corresponding to the set representation on which the domain is based; and identifying the list associated with the user.

7. identifying a type of the query based on the set expression; The method of claim 1 or 2, further comprising: generating the query expression according to the type-based aspect.

8. The method of claim 7 , wherein the type-based aspect identifies content to be added to the query in the query formulation.

9. 3. The method according to claim 1, wherein the query expression includes a question requiring an answer that means affirmative or negative.

10. The method of claim 1 or 2, wherein the situation includes a situation related to an automobile.

11. 3. The method of claim 1, wherein the condition defines how often the situation occurs.

12. An interaction method implemented by a server device and a user terminal, comprising: receiving, by the user terminal, a configuration expression including a query and a condition; acquiring, by the server device, the setting expression from the user terminal; storing, by the server device, the query and the condition in a memory; instructing the user terminal by the server device to output a query expression including the query in response to occurrence of a situation specified by the condition; and outputting the query expression by the user terminal in response to an instruction from the server device.

13. The interaction method according to claim 12 , wherein the user terminal receives the set expression by voice.

14. The interaction method according to claim 12 or 13, further comprising a step of transmitting data by the user terminal to the server device for determining whether or not a situation specified by the condition has occurred.

15. a processor; A server device comprising: a memory for storing a program that, when executed by the processor, causes the processor to implement the interaction method according to claim 1 or 2.

16. A program that, when executed by a processor, causes the processor to implement the interaction method according to claim 1 or 2.

Images