Support device, conversation control device and program

Abstract

To reduce the burden of the effort required to implement a technology of interaction between a device and a person.SOLUTION: An assistance device includes an output control unit that controls operations of an output unit that outputs words input into an input unit that receives an input of words which are information output by an output source. In a case in which information indicating an event that is likely to occur in a human-device conversation is input, the output control unit causes the output unit to output an event indicated by the input information. In a case in which information indicating an event that is likely to occur in the (N-1)th place in the human-device conversation is input into the input unit, the output control unit causes the output unit to output information prompting an input of information indicating an event that is likely to occur in the Nth place in the human-device conversation. One of the events that are likely to occur in the human-device conversation is a response process execution event which is an event in which a device to be controlled executes a response process. The output unit displays information on candidates of the response process execution event in natural language.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an assistance device, a conversation control device, and a program. This application claims priority to U.S. Provisional Application No. 63 / 014,721, filed in the United States on April 24, 2020, the contents of which are incorporated herein by reference. [Background technology]

[0002] 2. Description of the Related Art There are devices that play music or provide weather and other information in response to a user's voice command. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 228667 Summary of the Invention [Problem to be solved by the invention]

[0004] This technology involves interaction between devices and people. Therefore, many people have various ideas about how to achieve this interaction. However, even if they have ideas, many people are not familiar with the computer programs that control the devices, and implementing these ideas can be a heavy burden for such people.

[0005] In view of the above circumstances, an object of the present invention is to provide a technology that reduces the burden of labor required to implement technology for interaction between devices and people. [Means for solving the problem]

[0006] One aspect of the present invention is an assistance device that assists in the generation of a computer program that controls the operation of a device to be controlled, and includes an output control unit that controls the operation of an output unit that outputs information output by an output source and words input to an input unit that accepts word input, wherein when information indicating an event that may occur in a human-device conversation, which is an exchange between the device to be controlled and a user of the device to be controlled, is input, the output control unit outputs the event indicated by the input information, and when information indicating an event that may occur as the (N-1)th event (N is an integer greater than or equal to 1) in the human-device conversation is input to the input unit, the output control unit outputs information to prompt the input of information indicating an event that may occur as the Nth event in the human-device conversation. [Effects of the Invention]

[0007] The present invention makes it possible to reduce the burden of effort required to implement technology for interaction between devices and people. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a support system 100 according to an embodiment. [Figure 2] FIG. 1 is a first diagram showing an example of a screen displayed by a display unit 101 in an embodiment. [Figure 3] FIG. 2 is a second diagram showing an example of a screen displayed by the display unit 101 in the embodiment. [Figure 4] FIG. 3 is a third diagram showing an example of a screen displayed by the display unit 101 in the embodiment. [Figure 5] FIG. 4 is a fourth diagram showing an example of a screen displayed by the display unit 101 in the embodiment. [Figure 6] FIG. 5 is a fifth diagram showing an example of a screen displayed by the display unit 101 in the embodiment. [Figure 7] FIG. 10 is a diagram showing an example of interaction flow information according to the embodiment. [Figure 8] FIG. 1 is a diagram showing an example of the hardware configuration of a support device 1 according to an embodiment. [Figure 9]FIG. 2 is a diagram showing an example of the functional configuration of a control unit 11 in the embodiment. [Figure 10] 1 is a flowchart showing an example of a flow of processing executed by the support device 1 of the embodiment. [Figure 11] FIG. 2 is a diagram showing an example of the hardware configuration of a conversation control device 3 in the embodiment. [Figure 12] FIG. 2 is a diagram showing an example of the functional configuration of a control unit 31 in the embodiment. [Figure 13] 4 is a flowchart showing an example of the flow of processing executed by the conversation control device 3 of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] 1 is a diagram showing an example of the configuration of a support system 100 according to an embodiment. The support system 100 includes a support device 1, a management device 2, and a conversation control device 3.

[0010] The support device 1 supports the design of a computer program designer 991 that controls the operation of a device to be controlled. In this embodiment, the "computer program" to be designed includes not only so-called program code but also parameter and table values ​​referenced in the execution of the program. Therefore, in this embodiment, the design of a computer program includes not only the generation of program code but also the determination of parameter values ​​referenced in the execution of the program and the generation of tables. The device to be controlled is, for example, a smart speaker. The device to be controlled may also be, for example, a guide robot or a security robot. The operation terminal 993 and the operated device 994 in FIG. 1 are examples of devices to be controlled.

[0011] The support device 1 includes a display unit 101 and an input unit 102. The display unit 101 includes a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display. The display unit 101 may include an interface that connects these display devices to the support device 1. The input unit 102 includes input devices such as a mouse, a keyboard, or a touch panel. The input unit 102 may include an interface that connects these input devices to the support device 1.

[0012] 2 to 6 show examples of screens displayed by the display unit 101. Fig. 2 is a first diagram showing an example of a screen displayed by the display unit 101 in an embodiment. The screen displayed by the display unit 101 includes an area A101, an area A102, and an area A103.

[0013] Area A101 is an area where input received messages are displayed when the received messages are input to input unit 102. The received messages are words that may be input to the device to be controlled in an exchange (hereinafter referred to as a "human-device conversation") between the device to be controlled and a user 992 of the device to be controlled (hereinafter referred to as a "user 992").

[0014] Human-device conversation is a phenomenon in which one of three types of events, a received word occurrence event, a given word occurrence event, and a response processing execution event, occurs repeatedly in each order due to an event that occurred immediately before. A received word occurrence event is an event in which a user 992 inputs words into a controlled device. A given word occurrence event is an event in which a controlled device outputs words. A response processing execution event is an event in which a controlled device executes processing (hereinafter referred to as "response processing") in response to the input words. Note that a word refers to a combination of one or more phrases of various lengths, from a single word to a long sentence.

[0015] The received words are, for example, "Please tell me what the weather will be like tomorrow." "Human Says:" shown in FIG. 2 is an example of information indicating that this is an area for displaying received words.

[0016] Area A102 is an area where, when a given message is input to the input unit 102, the input given message is displayed. The given message is a word that may be output by the controlled device in a human-device conversation. For example, if the received message is "Please tell me about the weather tomorrow," the given message would be, for example, "It will be sunny tomorrow." "AI Says:" in FIG. 2 is an example of information indicating that this is an area where a given message is displayed.

[0017] Area A103 is an area where the content of the response process indicated by the input response process candidate information (hereinafter referred to as "response process candidate information") is displayed when information on the response process candidate information is input to input unit 102. For example, when the received message is "Please play Spring from Vivaldi's The Four Seasons," the response process candidate is the action of playing "Spring" from Vivaldi's The Four Seasons.

[0018] A candidate response process may be, for example, if the received phrase is "Please play Spring from Vivaldi's Four Seasons" and data for "Spring" from Vivaldi's "Four Seasons" cannot be found, searching for other similar songs, other songs by Vivaldi, or other songs associated with "spring" and playing the song found. In this case, the given phrase in area A102 may be further defined to say "We were unable to find that song." "AI Does:" in Figure 2 is an example of information indicating that this is an area for displaying information indicating candidate response processes.

[0019] Information that prompts the designer 991 to input information indicating the first event that may occur in the human-machine conversation is displayed on the screen displayed by the display unit 101. Information J101 in Fig. 2 is an example of information that prompts the designer 991 to input information indicating the first event that may occur in the human-machine conversation.

[0020] Fig. 3 is a second diagram showing an example of a screen displayed by the display unit 101 in the embodiment. Fig. 3 shows the result when the words "good morning" are input as a given word into the input unit 102 with the screen of Fig. 2 displayed. Since the words "good morning" are input as a given word into the input unit 102, the words "good morning", which are the given word input in the area A102, are displayed on the screen of Fig. 3. Information J102 in Fig. 3 is an example of information indicating the input given word.

[0021] The input words "good morning" are words that are input when the screen of the display unit 101 is the screen of Fig. 2. Therefore, "good morning" input as a given word is a word that occurs in the first event of the human-device conversation, and is a word that is output by the device to be controlled. To show this to the designer 991, the words "good morning" are displayed on the screen of Fig. 3 at the same height position on the screen as the information J101.

[0022] 3 shows information J103. Information J103 is an example of information that prompts the designer 991 to input information indicating a second event that may occur in the human-machine conversation.

[0023] FIG. 4 is a third diagram showing an example of a screen displayed by the display unit 101 in the embodiment. FIG. 4 shows the result when the words "how are you" and "go away" are input as received words into the input unit 102 while the screen of FIG. 3 is displayed. Since "how are you" and "go away" are input as received words into the input unit 102, the input received words "how are you" and "go away" are displayed in the area A101 of the screen of FIG. 4. By defining multiple received words in this way, the flow of the conversation can be branched. Both "how are you" and "go away" input as received words are words that the user 992 may input to the device to be controlled. Information J104 in FIG. 4 is an example of information indicating the input received words. The user 992 may input words with the same or similar meaning as received words. Furthermore, for example, "fine" may be defined as a given word for "how are you." For example, for "go away," the current time is obtained as a response processing candidate without defining a given word, and as the next action, if the current time is in the morning time zone, the given word "good morning" is defined, and if the current time is not in the morning time zone, no action is taken.

[0024] 4 shows information J105. Information J105 is an example of information that prompts the designer 991 to input information indicating a third event that may occur in the human-machine conversation.

[0025] Fig. 5 is a fourth diagram showing an example of a screen displayed by the display unit 101 in the embodiment. Fig. 5 shows the result when the character strings "timeofday" and "morning" are input as response processing candidate information with the screen of Fig. 4 displayed. Information J106 is an example of response processing candidate information displayed on the screen.

[0026] "Time of day" and "morning" are information indicating the operation of the controlled device, but are not computer programs that instruct the operation. Both "time of day" and "morning" are memorandums of the designer 991 regarding what kind of processing the controlled device will execute. The response processing candidate information may be the program itself of the processing that the controlled device may execute, or may be a memorandum regarding the processing that the controlled device may execute, as in the example of Figure 5. When there are multiple response processing candidates like this, the conversation flow may branch. In this case, the branch is defined in the output data (exchange flow information). For example, two or more branches are defined depending on the result of the processing "time of day".

[0027] Since the response process candidate information may be a memorandum, even if the designer 991 is not able to generate a computer program to control the controlled device, the designer 991 can record the content of processes that the controlled device may execute as a memorandum. The content of the memorandum can be viewed by a person with the skills to generate a computer program to control the controlled device. Therefore, even if the designer 991 himself is not able to generate a computer program to control the controlled device, the designer 991 can generate the computer program with the help of another person with the skills.

[0028] 5 shows information J107. Information J107 is an example of information that prompts the designer 991 to input information indicating a fourth event that may occur in the human-machine conversation.

[0029] Fig. 6 is a fifth diagram showing an example of a screen displayed by the display unit 101 in the embodiment. Fig. 6 shows the result when the character string "sorry for disturbing you but I will wake you up now" is input as a given message into the input unit 102 with the screen of Fig. 5 displayed. Since "sorry for disturbing you but I will wake you up now" is input as a given message into the input unit 102, the input given message "sorry for disturbing you but I will wake you up now" is displayed in area A102 of the screen of Fig. 6. Information J108 in Fig. 6 is an example of information indicating the input given message.

[0030] 6 shows information J109. Information J109 is an example of information that prompts the designer 991 to input information indicating a fifth event that may occur in the human-machine conversation.

[0031] In this way, when information indicating an event that may occur in the (N-1)th place in the human-device conversation is input to the input unit 102, the display unit 101 displays information prompting the input of information indicating an event that may occur in the Nth place in the human-device conversation, where N is an integer equal to or greater than 1. Note that such a number (e.g., N) does not necessarily have to be displayed on the display unit 101.

[0032] The support device 1 generates interaction flow information based on information input to the input unit 102. The interaction flow information is text data information indicating the flow of occurrence of events in a human-machine conversation. An example of the flow is the flow of occurrence of events in a human-machine conversation assumed by the designer 991, as illustrated in FIGS. 2 to 6. Therefore, the support device 1 prompts the designer 991 to input information indicating events that may occur in a human-machine conversation, displays the contents of the input events, and generates interaction flow information indicating the flow of occurrence of the input events.

[0033] The interaction flow information includes at least order information, execution subject information, and content information. The order information indicates the order in which each of the events of the received message occurrence event, the given message occurrence event, and the response processing execution event occurs in a human-machine conversation.

[0034] The execution subject information is information indicating the subject that generates the event of each order indicated by the order information. For example, for a received word generation event, the execution subject information indicates the input source that inputs the received word to the device to be controlled, such as the user 992. For example, for a given word generation event, the execution subject information indicates the device to be controlled that is the subject that outputs the given word. For example, for a response processing execution event, the execution subject information indicates the device to be controlled that is the subject that executes the candidate response processing indicated by the response processing execution information.

[0035] The content information indicates the content of each event in the order indicated by the order information. For example, for a received word occurrence event, the content information indicates the words of the received word itself. For example, for a given word occurrence event, the content information indicates the words of the given word itself. For example, for a response process execution event, the content information indicates candidate response processes indicated by the response process execution information.

[0036] The numbers indicated by the information J101, information J103, information J105, information J107, and information J109 in Figures 2 to 6 indicate the order of occurrence of events in the human-machine conversation. Therefore, information J101, information J103, information J105, information J107, and information J109 are examples of sequence information. In this way, the display unit 101 may display sequence information. Furthermore, these pieces of interaction flow information may be individually deleted or inserted. Furthermore, interaction flow information of subsequent processing may be defined in common for multiple pieces of interaction flow information. Furthermore, the order of the numbers displayed does not necessarily have to follow the order of operations.

[0037] Fig. 7 is a diagram showing an example of interaction flow information according to an embodiment. Fig. 7 shows information J21, information J22, and information J23. Information J21, information J22, and information J23 are information relating to events that occur in interactions between a device to be controlled and a user 992. The information shown in Fig. 7 is a specific example of interaction flow information in the processes described with reference to Figs. 2 to 6.

[0038] Information J21 indicates the first event that occurs in the interaction between the controlled device and the user 992. Information J211 indicates that information J21 is information relating to the first event that occurs in the interaction between the controlled device and the user 992. Information J211 in FIG. 7 is an example of order information. Information J212 in FIG. 7 indicates that the subject of the first event that occurs in the interaction between the controlled device and the user 992 is the controlled device. Information J212 in FIG. 7 is an example of execution subject information.

[0039] Information J213 in Fig. 7 indicates the content of the first event that occurs in the interaction between the device to be controlled and user 992. Specifically, information J213 indicates that the word "hello" is uttered by the subject indicated by information J212 in the first event that occurs in the interaction between the device to be controlled and user 992. Information J213 in Fig. 7 is an example of content information.

[0040] Information J22 indicates an event that occurs second in the interaction between the device to be controlled and the user 992. Information J221 indicates that information J22 is information relating to an event that occurs second in the interaction between the device to be controlled and the user 992. Information J221 in FIG. 7 is an example of order information. Information J222 in FIG. 7 indicates that the subject of the event that occurs second in the interaction between the device to be controlled and the user 992 is the user 992. Information J222 in FIG. 7 is an example of execution subject information.

[0041] Information J223 in Fig. 7 indicates the content of the second event that occurs in the interaction between the device to be controlled and the user 992. Specifically, information J223 indicates that the subject indicated by information J222 utters the words "how are you" in the second event that occurs in the interaction between the device to be controlled and the user 992. Information J223 in Fig. 7 is an example of content information.

[0042] Returning to the explanation of FIG. 1, the support device 1 is communicatively connected to the management device 2 via a network 901. The management device 2 includes a control unit 21 and a storage unit 22. The control unit 21 includes a processor such as a CPU (Central Processing Unit) and a memory connected via a bus. The control unit 21 controls the operation of each functional unit included in the management device 2. The management device 2 is communicatively connected to the support device 1 via the network 901. The management device 2 acquires interaction flow information generated by the support device 1 via the network 901 and records it in the storage unit 22. The control unit 21 executes the process of acquiring the interaction flow information and the process of recording the acquired interaction flow information in the storage unit 22.

[0043] The management device 2 is communicatively connected to the conversation control device 3 via a network 902. The conversation control device 3 controls the operation of a device to be controlled, such as an operated device 994, in accordance with the received words output by the user 992, based on the interaction flow information generated by the assistance device 1. More specifically, the conversation control device 3 acquires text data indicating the received words output by the user 992, and causes the device to be controlled to perform an operation in accordance with the interaction flow information based on the acquired text data.

[0044] The conversation control device 3 acquires interaction flow information from the management device 2 via the network 902. The conversation control device 3 controls the operation of the operated device 994 via the network 903, causing the controlled device to perform an operation in accordance with the interaction flow information based on the acquired text data. The conversation control device 3 acquires the text data from the operating terminal 993 via the network 903.

[0045] The operation terminal 993 accepts input of received speech from the user 992. The operation terminal 993 converts the input received speech into text data. The operation terminal 993 transmits the text data generated by the conversion to the conversation control device 3 via the network 903. The operation terminal 993 is, for example, a personal computer capable of executing a process of converting input received speech into text data.

[0046] The operation terminal 993 may be any device as long as it can accept input of speech from the user 992 and can convert the input speech into text data. The operation terminal 993 may be, for example, a mobile phone or a smartphone. The operation terminal 993 may be, for example, a device having an AI (artificial intelligence) assistant function, such as a smart speaker. The operation terminal 993 does not necessarily need to be mounted in a housing separate from the operated device 994, and may be configured integrally with the operated device 994.

[0047] The method by which the user 992 inputs the received words into the operation terminal 993 may be any method that can convert the received words into text data. The method by which the user 992 inputs the received words into the operation terminal 993 may be, for example, a method in which the operation terminal 993 executes an executable SNS (Social Networking Service) app and inputs the received words using the running SNS app. For example, the input may be performed by the user 992 speaking a voice corresponding to the received words. In this case, a voice recognition process is performed, and the voice uttered by the user 992 is converted into text data and input to the operation terminal 993.

[0048] 8 is a diagram showing an example of the hardware configuration of the support device 1 in an embodiment. The support device 1 has a control unit 11 including a processor 91 such as a CPU (Central Processing Unit) and a memory 92 connected by a bus, and executes a program. By executing the program, the support device 1 functions as a device including a user interface 10, the control unit 11, a storage unit 12, and a communication unit 13.

[0049] More specifically, in the support device 1, the processor 91 reads a program stored in the storage unit 12 and stores the read program in the memory 92. When the processor 91 executes the program stored in the memory 92, the support device 1 functions as a device including a user interface 10, a control unit 11, a storage unit 12, and a communication unit 13.

[0050] The user interface 10 includes a display unit 101 and an input unit 102. The display unit 101 and the input unit 102 may be implemented as separate devices, or may be implemented as an integrated device, such as a touch panel. The control unit 11 controls the operation of various functional units included in the support device 1. The control unit 11 generates, for example, interaction flow information.

[0051] The storage unit 12 is configured using a computer-readable storage medium device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 12 stores various information related to the assistance device 1. The storage unit 12 stores, for example, various information generated as a result of processing executed by the control unit 11. The storage unit 12 stores, for example, interaction flow information generated by the control unit 11. The storage unit 12 pre-stores image data of a screen (hereinafter referred to as an "initial screen") before any of the received words, given words, and response processing execution information are input to the input unit 102, such as the screen shown in FIG. 2.

[0052] The communication unit 13 includes a communication interface for connecting the support device 1 to an external device. The communication unit 13 communicates with the external device via a network 901, either wired or wirelessly. The external device is, for example, the management device 2.

[0053] 9 is a diagram showing an example of the functional configuration of the control unit 11 in the embodiment. The control unit 11 includes an input control unit 110, a display control unit 111, an interaction flow information generation unit 112, a storage control unit 113, and a communication control unit 114.

[0054] The input control unit 110 controls the operation of the input unit 102 .

[0055] The display control unit 111 controls the operation of the display unit 101. The display control unit 111 executes, for example, an initial screen display process. The initial screen display process is a process of reading image data of an initial screen stored in the storage unit 12 and displaying the initial screen on the display unit 101.

[0056] The display control unit 111 executes, for example, a received message display process. The received message display process is a process for causing the display unit 101 to display the received message input to the input unit .

[0057] The display control unit 111 executes, for example, a prompting information display process. The prompting information display process is a process for displaying, on the display unit 101, information prompting input of information indicating an event that may occur Nth in the human-device conversation when information indicating an event that may occur (N-1)th in the human-device conversation is input to the input unit 102. Note that the information indicating each event can be deleted individually or inserted in the middle.

[0058] An example of a change in the screen displayed by the display unit 101 due to the execution of both the received word display process and the prompting information display process is the change from the screen shown in Fig. 3 to the screen shown in Fig. 4. The change from the screen shown in Fig. 5 to the screen shown in Fig. 6 is also an example of a change in the screen displayed by the display unit 101 due to the execution of both the received word display process and the prompting information display process.

[0059] The display control unit 111 executes, for example, a response process candidate display process. The response process candidate display process is a process for displaying, on the display unit 101, response process candidates indicated by the input response process candidate information when the response process candidate information is input to the input unit 102.

[0060] An example of a change in the screen displayed by the display unit 101 due to the execution of both the response process candidate display process and the prompting information display process is a change from the screen shown in FIG. 4 to the screen shown in FIG.

[0061] The display control unit 111 executes, for example, a given-word display process. The given-word display process is a process of causing the display unit 101 to display a given word input to the input unit 102.

[0062] An example of a change in the screen displayed by the display unit 101 due to the execution of both the given message display process and the prompting information display process is a change from the screen shown in FIG. 2 to the screen shown in FIG.

[0063] The interaction flow information generation unit 112 generates interaction flow information based on the information input to the input unit 102 .

[0064] The memory control unit 113 records various types of information generated by the control unit 11 in the memory unit 12. The memory control unit 113 records, for example, interaction flow information generated by the control unit 11 in the memory unit 12. The memory control unit 113 records, for example, information input to the input unit 102 (i.e., information acquired by the input control unit 110) in the memory unit 12.

[0065] The communication control unit 114 controls the operation of the communication unit 13. The communication control unit 114 controls the operation of the communication unit 13 to, for example, transmit exchange flow information to the management device 2. The management device 2 receives the transmitted exchange flow information and records it in the storage unit 22.

[0066] 10 is a flowchart showing an example of the flow of processing executed by the support device 1 of the embodiment. Information is input to the input unit 102 (step S101). Next, the display control unit 111 executes any one of a received message display process, a response process candidate display process, or a given message display process according to the input information, thereby displaying the input information on the display unit 101 and executing a prompting information display process (step S102). By the processing of step S102, the display unit 101 displays the input information and information prompting the input of information indicating an event that may occur in the human-device conversation after the occurrence of the event that causes the input information.

[0067] 11 is a diagram showing an example of the hardware configuration of a conversation control device 3 in an embodiment. The conversation control device 3 has a control unit 31 equipped with a processor 93 such as a CPU and a memory 94 connected via a bus, and executes a program. By executing the program, the conversation control device 3 functions as a device equipped with a user interface 30, control unit 31, storage unit 32, and communication unit 33.

[0068] More specifically, in the conversation control device 3, the processor 93 reads a program stored in the storage unit 32 and stores the read program in the memory 94. When the processor 93 executes the program stored in the memory 94, the conversation control device 3 functions as a device equipped with a user interface 30, a control unit 31, a storage unit 32, and a communication unit 33.

[0069] The user interface 30 includes a display unit 301 and an input unit 302. The display unit 301 includes a display device such as a CRT display, a liquid crystal display, or an organic EL display. The display unit 301 may include an interface that connects these display devices to the conversation control device 3. The input unit 302 includes input devices such as a mouse, a keyboard, or a touch panel. The input unit 302 may include an interface that connects these input devices to the conversation control device 3.

[0070] The control unit 31 controls the operation of various functional units included in the conversation control device 3. The control unit 31 acquires interaction flow information from the management device 2 via the communication unit 33 and the network 902, for example.

[0071] The storage unit 32 is configured using a computer-readable storage medium device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 32 stores various information related to the conversation control device 3. The storage unit 32 stores various information resulting from processing executed by the control unit 31, for example. The storage unit 32 stores, for example, interaction flow information acquired by the control unit 31 from the management device 2 via the communication unit 33. Note that the storage unit 32 may have already stored the interaction flow information before the text data indicating the received words is acquired by the communication unit 33, or the reverse order may be used. There is no restriction on the order of storage. The information stored in the storage unit 32 may also be stored in another device (for example, the management device 2).

[0072] The communication unit 33 includes a communication interface for connecting the conversation control device 3 to an external device. The communication unit 33 communicates with the management device 2 via a network 902, either wired or wirelessly. The communication unit 33 communicates with the operating terminal 993 and the operated device 994 via a network 903, either wired or wirelessly. The communication unit 33 receives, for example, text data indicating received messages through communication with the operating terminal 993. The communication unit 33 transmits, to the operated device 994, instructions to cause the operated device 994 to execute, for example, an operation corresponding to the text data indicating received messages and indicated by interaction flow information, through communication with the operated device 994. More specifically, for example, a script matching the word entered in area A103 (AIDoes) is started, and text data, interaction flow information, and the like are passed as arguments to the script. The script then ultimately returns several arguments, including the character string in the input field to the right of area A103 (AIDoes).

[0073] 12 is a diagram showing an example of the functional configuration of the control unit 31 in the embodiment. The control unit 31 includes an input control unit 310, a display control unit 311, a conversation control unit 312, a storage control unit 313, and a communication control unit 314.

[0074] The input control unit 310 controls the operation of the input unit 302. The display control unit 311 controls the operation of the display unit 301.

[0075] The conversation control unit 312 determines an instruction to be given to the operated device 994 based on the interaction flow information and the text data indicating the received words input to the communication unit 33. For example, if the received words indicated by the text data are "Play Spring from Vivaldi's The Four Seasons," the conversation control unit 312 determines to instruct the operated device 994 to play "Spring" from Vivaldi's "The Four Seasons."

[0076] The memory control unit 313 records various information generated by the control unit 31 in the memory unit 32. The memory control unit 313 records, for example, information indicating an instruction to the operated device 994 determined by the control unit 31 in the memory unit 12. The memory control unit 313 records, for example, text data indicating received words acquired via the communication unit 33 in the memory unit 12.

[0077] The communication control unit 314 controls the operation of the communication unit 33. The communication control unit 314 controls the operation of the communication unit 33 to, for example, acquire interaction flow information from the management device 2. The communication control unit 314 controls the operation of the communication unit 33 to, for example, transmit an instruction determined by the conversation control unit 312 to the operated device 994. The communication control unit 314 acquires, for example, text data indicating the received words via the communication unit 33.

[0078] In this way, the instruction determined by the conversation control unit 312 is sent to the operated device 994 by the communication control unit 314 via the communication unit 33. The operated device 994 that receives the instruction performs an operation in accordance with the instruction. Therefore, the conversation control unit 312 is a functional unit that controls the operation of the operated device 994 based on the received words. In other words, the conversation control unit 312 controls the operation of the operated device 994, thereby controlling the interaction between the user 992 and the operated device 994.

[0079] 13 is a flowchart showing an example of the flow of processing executed by the conversation control device 3 of the embodiment. The communication control unit 314 acquires text data indicating the received words via the communication unit 33 (step S201). Next, the conversation control unit 312 determines an instruction to be given to the operated device 994 based on the text data indicating the received words and the interaction flow information (step S202). The communication control unit 314 transmits the instruction determined by the conversation control unit 312 to the operated device 994 (step S203). Note that there is no restriction that a response process is necessarily performed after the received words. A giving word may be displayed in response to the received words, or another operation may be performed.

[0080] The control in steps S202 and S203 is an example of control of the operated device 994 by the conversation control unit 312. Note that the operated device 994 has been programmed in advance based on the interaction flow information by a person or device capable of generating a computer program for controlling the operated device 994 so that the operated device 994 operates in accordance with an instruction.

[0081] When a message is input to the input unit 102, the assistance device 1 in this embodiment configured causes the display unit 101 to display the input message. Furthermore, the assistance device 1 also displays information prompting the user to input information indicating an event that may occur next. Therefore, the assistance device 1 makes it easy for the designer 991 using the assistance device 1 to understand the interaction between the device to be controlled and the user 992. Therefore, the assistance device 1 can reduce the burden of effort required to implement technology for interaction between devices and people.

[0082] As described with reference to FIGS. 2 to 6, the support device 1 displays on the display unit 101 the response words input to the input unit 102, and also displays information prompting the input of information indicating an event that may occur next. The support device 1 thereby supports the expression of ideas by the designer 991. Therefore, the designer 991 does not need to input the code of a computer program into the support device 1. The information that the designer 991 inputs into the support device 1 may be words exchanged in the exchange between the device to be controlled and the user 992, and information indicating the content of the process to be executed by the device to be controlled (which may be a device different from the device to be controlled described above).

[0083] The designer 991 can express what he or she wants to achieve with a computer program using only the words exchanged in the interaction between the controlled device and the user 992 and information indicating the content of the processing executed by the controlled device. The information generated by the assistance device 1 may be implemented by a person with the technology to implement the controlled device (e.g., a person other than the designer 991), or by a device that generates a computer program based on the interaction flow information. Therefore, the assistance device 1 can reduce the burden of labor required to implement technology for interaction between a device and a person. A program managing interactions between a person and a device (e.g., interactions between the user 992 and the operation terminal 993) may be pre-installed in the conversation control device 3. The newly generated interactions by the assistance device 1 may then be defined in the operation terminal 993, causing the operation terminal 993 to operate in such a way that the defined interactions are realized. Naturally, when a new operation terminal 993 is added, a control program for the new operation terminal 993 is required. For such a new operation terminal 993, the control program does not necessarily require the interactions generated by the assistance device 1. In this case, the exchange is carried out with contents predefined in a control program installed on the operation terminal 993 .

[0084] As described above, the interaction flow information is text data. Therefore, the support system 100 enables the user 992 to control the operated device 994 regardless of the application executed by the operating terminal 993 or the specifications of the operated device 994.

[0085] (Application example) <Medicinal application> The support system 100 is used to enable the user 992 to obtain, for example, more accurate information about medications. Medication information can be obtained, for example, by searching the name of a medication on the Internet. However, the information that appears when searching on the Internet is, for example, only a list of side effects, and it may not be easy to obtain information about interactions between multiple medications. As a result, the user 992 may obtain incorrect information about medications.

[0086] By using the support system 100, it is possible to obtain more accurate information about medicines. This will be explained below.

[0087] When the assistance system 100 is used to provide drug information, the device to be controlled is, for example, a device capable of outputting a description of the drug. The device capable of outputting a description of the drug is, for example, a smart speaker. The user 992 reads a two-dimensional barcode previously attached to a drug container using an operation terminal 993 capable of reading two-dimensional barcodes, such as a smartphone. The smartphone may transmit the read information to the smart speaker, for example, via wireless communication or wired communication. By reading the two-dimensional barcode with the operation terminal 993, the user 992 becomes able to communicate with the device to be controlled via the operation terminal 993. In this way, the two-dimensional barcode indicates information for communicating with the device to be controlled.

[0088] The medicine container does not necessarily have to have a two-dimensional barcode attached, and the operation terminal 993 used by the user 992 does not have to be a smartphone. Any type of medicine container may be used as long as it has information that enables communication with the device to be controlled. The operation terminal 993 may also be used as long as it can acquire the information attached to the medicine container, communicate with the device to be controlled, and convert words input by the user 992 into text data.

[0089] The user 992 who has started communication with the device to be controlled inputs words inquiring about information about medicine into the operation terminal 993 as received words. The words inquiring about information about medicine are, for example, questions such as "The user 992 takes medicine A, medicine B, and medicine C three times a day, but is it okay if he takes medicine D in addition?" The conversation control device 3 acquires text data indicating the received words input into the operation terminal 993. The conversation control device 3 controls the operation of the operated device 994 based on the acquired text data and interaction flow information generated in advance by the support device 1.

[0090] More specifically, the conversation control unit 312 of the conversation control device 3 controls the operation of the operated device 994 based on text data indicating the received words and interaction flow information generated in advance by the support device 1. For example, if the received words are the question "The user 992 takes medicine A, medicine B, and medicine C three times a day, but is it okay if he takes medicine D in addition?" as described above, the conversation control unit 312 issues the following medicine determination instruction to the operated device 994. The medicine determination instruction is an instruction to execute a process to determine whether it is okay for the user 992 to take medicine D in addition, and to execute a process to output words indicating the result of the determination. More specifically, the operation may be as follows: The operated device 994 returns two different texts to the conversation control device 3. For example, if the text is "talkdoctor", the next step will be the given phrase "Talk to a doctor. Want to find one nearby?", leading the user 992 to a doctor search conversation. If the text is "noproblem", the given phrase will be "That's fine." Then, the user will be asked "Would you like to participate in a survey?" and the survey will begin.

[0091] The operated device 994, in accordance with an instruction from the conversation control unit 312, determines whether or not there is any problem for the user 992 to take more medicine D, and outputs a message indicating the result of the determination.

[0092] The interaction flow information used in the process of responding to such medication information is information that has been generated in advance by the designer 991 using the support device 1. The designer 991 inputs, for example, a question about medication information as a received message into the input unit 102. The display control unit 111 executes a received message display process to display the input received message on the display unit 101, and also executes a prompting information display process. The designer 991 then inputs a process to be executed by the device to be controlled in response to the received message. The display control unit 111 displays the input process on the display unit 101, and also executes a prompting information display process. Next, the designer 991 inputs a given message to be output by the device to be controlled. The display control unit 111 displays the input given message on the display unit 101.

[0093] The exchange flow information generation unit 112 generates medication flow information based on the information input to the input unit 102. The medication flow information is text data information that indicates the flow of events occurring in medication-related exchanges between the user 992 and the device to be controlled. In other words, the medication flow information is a type of exchange flow information.

[0094] The drug flow information indicates, for example, that the received words input to the input unit 102 are words that the user 992 will output in the first event of a drug-related exchange between the user 992 and the controlled device. The drug flow information indicates, for example, that the next process input to the input unit 102 is a process that the controlled device will execute in the event following the first event of a drug-related exchange between the user 992 and the controlled device. The drug flow information indicates, for example, that the next given words input to the input unit 102 are words that the controlled device will output in the third event of a drug-related exchange between the user 992 and the controlled device.

[0095] A person or device capable of generating a computer program for controlling a controlled device using the generated medicine flow information programs the operation of the controlled device to follow the flow of events indicated by the medicine flow information, and therefore the controlled device operates in accordance with the flow indicated by the medicine flow information.

[0096] In this way, the support system 100 of the embodiment can be used to provide information on medicines. When used for providing information on medicines, the user 992 can obtain more accurate information than if the user 992 were to search for information on medicines themselves. When generating medicine flow information, the designer 991 may input prompts encouraging the user to seek a diagnosis from a doctor if a predetermined condition is met. In such a case, depending on a question from the user 992, the device to be controlled can output prompts encouraging the user to seek a diagnosis from a doctor.

[0097] <Application to vehicle technology> The assistance system 100 may be applied to vehicle technology, for example. When applied to MaaS (Mobility as a service) / vehicle technology, the user 992 is, for example, a tourist, and the device to be controlled is a communication robot that acts as a tour guide. Text data information (hereinafter referred to as "tourist flow information") indicating the flow of events occurring in the interactions between the tourist and the communication robot is generated in advance by the designer 991 using the assistance device 1. The tourist flow information is a type of interaction flow information.

[0098] When generating tourist flow information, the designer 991 inputs words that are expected to be asked by tourists as received words into the input unit 102. The received words are displayed on the display unit 101, along with the results of the execution of the prompt information display process. Next, the designer inputs given words that indicate a response to the received words into the input unit 102. The display unit 101 displays the given words, along with the results of the execution of the prompt information display process.

[0099] The interaction flow information generation unit 112 generates tourist flow information. The tourist flow information indicates, for example, that a received message input to the input unit 102 is a received message output by the tourist in the first event of an interaction between the tourist and the communication robot. The tourist flow information indicates, for example, that a given message input to the input unit 102 is a given message output by the communication robot in the event following the first event of an interaction between the tourist and the communication robot.

[0100] The received speech output by the tourist may be, for example, "How high is Tokyo Tower?" In this case, the given speech output by the communication robot may be, "The height of Tokyo Tower is 333 meters." The received speech output by the tourist may be, "I want to go to the station." In this case, the given speech output by the communication robot may be, for example, a question such as, "Do you want to go to Yotsuya Station or Shinjuku Station?" If there is a response to such a question, the response may be input to a navigation device corresponding to the operated device 994. Thereafter, the interaction between the user 992 and the navigation device may be controlled.

[0101] If the communication robot is installed on a bus or the like, when the communication robot passes a predetermined specific location, it may output a given phrase such as "You can see Tokyo Tower on your left."

[0102] As described above, when using the support device 1, all the designer 991 has to do is input the words and processing content that he or she wants the device to control to output. Therefore, by using the support device 1, even someone who is not familiar with computer programs can change the words that the communication robot outputs.

[0103] The language input to the support device 1 is not limited to a specific language and may be any language. The input language may be translated into another language using natural language processing technology. In such a case, even if the designer 991 inputs in one language, the device to be controlled can output words in multiple languages. Instead of translation, mapping to another language may be performed. Mapping is not simply translation, but is performed by executing language processing according to the other language.

[0104] Therefore, when the support system 100 is used for tourist guidance or when a sudden disaster such as an earthquake occurs and evacuation routes must be communicated to tourists of various nationalities on the spot, the support system 100 can easily communicate the evacuation routes because, if the designer 991 inputs the evacuation routes in a specific language into the support device 1, the words are translated into multiple languages ​​and output from the communication robot.

[0105] <Application to robot management> The assistance system 100 may be applied to the management of a robot. As an example, an application example of the assistance system 100 to the management of a robot will be described using a case where the robot has a movable arm.

[0106] Let us assume that the user 992 is the administrator of the robot, and that the administrator inputs a received message, such as "Please move arm 2 up," into the operation terminal 993. The robot, which is the device to be controlled, is controlled by the conversation control device 3 to operate in accordance with the received message. However, depending on the robot's situation, it may not always be possible to raise arm 2 up. Therefore, the robot outputs a given message indicating whether arm 2 has moved up. The administrator can understand the robot's situation by recognizing the given message output by the robot. The operation may be as follows. First, the robot's arm moves. The operated device 994 outputs the result of the arm's movement to the conversation control device 3. This output process may be an operation defined in area A103. The conversation control device 3 selects a given word and transmits the selected given word to the operating terminal 993. The operating terminal 993 outputs the received given word (for example, displays it on a screen or plays it as audio). Operating in this manner also provides the above-mentioned effects. Furthermore, there is also the effect that the operated device 994 does not need to communicate directly with the operating terminal 993.

[0107] Text data information (hereinafter referred to as "management flow information") indicating the flow of events occurring in the interactions between the manager and the robot to be managed is generated in advance by the designer 991 using the support device 1. The management flow information is a type of interaction flow information.

[0108] When generating management flow information, the designer 991 inputs words that are expected to be asked by the manager as received words into the input unit 102. The received words are displayed on the display unit 101, along with the results of the execution of the prompt information display process. Next, the designer inputs, into the input unit 102, given words that the managed robot will output in response to the received words. The display unit 101 displays the given words, along with the results of the execution of the prompt information display process.

[0109] The interaction flow information generation unit 112 generates management flow information. The management flow information may indicate, for example, that the behavior of a robot is the first event of an interaction between an administrator and a managed robot. For example, the management flow information indicates that a received message input to the input unit 102 is a received message output by the administrator in the first event of an interaction between the administrator and the managed robot. For example, the management flow information indicates that a given message input to the input unit 102 is a given message output by the managed robot in the event following the first event of an interaction between the administrator and the managed robot.

[0110] The support system 100 may be used to manage such robots.

[0111] <Application to nursing care management> In nursing homes for the elderly, nurses sometimes manage the health of residents by writing down information about the residents' health, such as the resident's name, the nurse's name, the date, the time, and the temperature, on a sheet of paper. However, writing down information on paper is a heavy burden. By using the support system 100, it is possible to record information about the residents' health by voice, without having to write it down on paper.

[0112] When the support system 100 is applied to the management of information related to the health of residents, the operation terminal 993 is a device capable of receiving speech and converting it into text data, such as a voice speaker or a smartphone. When the support system 100 is applied to the management of information related to the health of residents, the operated device 994 is a device capable of recording text data. The operated device 994 is, for example, a computer equipped with a storage device such as a hard disk. When the support system 100 is applied to the management of information related to the health of residents, the user 992 is a nurse.

[0113] When the support system 100 is applied to managing information related to the health of residents, the conversation control device 3 receives text data output by the operating terminal 993. The conversation control device 3 controls the operated device 994 based on the received text data and information in the text data indicating the flow of events occurring in the exchange between the nurse and the operated device 994 (hereinafter referred to as "nursing flow information"). In this case, the operated device 994 may be a device configured as a database.

[0114] Nursing flow information is a type of interaction flow information.

[0115] When generating nursing flow information, the designer 991 inputs the words that the nurse is expected to utter as received words into the input unit 102. The received words are displayed on the display unit 101, along with the results of the execution of the prompting information display process. Next, the designer inputs into the input unit 102 the process to be executed by the operated device 994 in response to the received words. The process to be executed by the operated device 994 is, for example, a process of recording the received words in a storage device. The display unit 101 displays the content of the process, along with the results of the execution of the prompting information display process.

[0116] The interaction flow information generator 112 generates nursing flow information. The nursing flow information indicates, for example, that the received message input to the input unit 102 is the received message uttered by the nurse in the first event of the interaction between the nurse and the operated device 994. The nursing flow information indicates, for example, that the process input to the input unit 102 is the process to be executed by the operated device 994 in the event following the first event of the interaction between the nurse and the operated device 994. The process to be executed by the operated device 994 in the event following the first event of the interaction between the nurse and the operated device 994 is, for example, the process of recording the received message in a storage device.

[0117] In such a case, the nurse can record the information about the resident's health by voice into the operated device 994. Words indicating the information about the resident's health are input into the operating terminal 993. The words indicating the information about the resident's health are an example of received words.

[0118] When recording information, information indicating the time when the information was recorded may also be recorded. The operated device 994 may output a message indicating that the information has been recorded. The information recorded in the operated device 994 can also be viewed by other interested parties such as other nurses and doctors.

[0119] Management of resident health information does not have to be done by voice, but can be done by text input. Even in this case, if the words spoken by the nurse in the case of voice input are the same as the words input in the case of text input, the text data received by the conversation control device 3 will be the same.

[0120] Although the application example of the support system 100 has been described using a nursing home for the elderly as an example, the support system 100 can be similarly applied to any facility that has a nursing recipient. In other words, residents are an example of a nursing recipient. For example, the support system 100 may be applied as required by a user who needs to simply register information while working.

[0121] (Variation) As explained in the description of the example of application to a tourist guide, even if the designer 991 of the support system 100 generates interaction flow information in one language, the operated device 994 can support multiple languages, and this is not limited to the example of a tourist guide. By applying natural language processing technology to the support system 100, even if the designer 991 generates interaction flow information in one specific language, the operated device 994 can support multiple languages.

[0122] Because the interaction flow information is text data, the assistance system 100 performs highly accurate translation, and the operated device 994 can output the highly accurate translation result. Furthermore, because the conversation control device 3 also handles text data, the operated device 994 can output the highly accurate translation result. Therefore, the designer 991 using the assistance device 1 can implement the operated device 994 that supports multiple languages ​​simply by generating interaction flow information in one language. Note that the translation process using natural language processing is performed, for example, by an interaction flow information generation unit. The translation process using natural language processing may also be performed, for example, by a conversation control unit.

[0123] It should be noted that the output of the operated device 994 does not necessarily have to be directed to the user 992. If the operated device 994 is located at a location away from the user 992 and there is a person other than the user 992 near the operated device 994, the user 992 can converse with the other person via the operated device 994. For example, if the user 992 inputs words in a first language using a first application, the input language may be translated into a language used by the other person located near the operated device 994 and output from the operated device 994.

[0124] Note that input to the input unit 102 may be character input using an input device such as a keyboard or a touch panel, or may be voice input. The support device 1 may also be provided with a device that outputs voice instead of the display unit 101. In such a case, the content that would be output by display on the display unit 101 is output by voice. Furthermore, multiple types of hardware can be applied as user interfaces, such as the input unit 102 and the display unit 101. The designer 991 can change from one type of hardware to another type of hardware during the work. For example, the designer 991 can initially input text using a keyboard, and then halfway through the work, the designer 991 can input text by speech.

[0125] In addition, since the interaction flow information is recorded in a storage device such as the storage unit 12 of the support device 1 or the storage unit 22 of the management device 2, it can be obtained by anyone other than the designer 991 who created the interaction flow information.

[0126] The conversation control device 3 may also transmit the history of interactions between the user 992 and the operated device 994 in the form of text data to the management device 2, which may then be recorded in the storage unit 22. The recorded history of interactions can be viewed by other users. In this case, even if different applications are used to view the history between the user 992 and the other users, both the user 992 and the other users can view the history of interactions. This is because the history of interactions is text data.

[0127] However, a password may be required for viewing. Once a password is entered, the person who entered the password may be able to view the content without entering the password thereafter.

[0128] The assistance system 100, the assistance device 1, the management device 2, and the conversation control device 3 may each be implemented using a plurality of information processing devices communicably connected via a network. All or part of the functions of the assistance system 100, the assistance device 1, the management device 2, and the conversation control device 3 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include portable media such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and storage devices such as a hard disk built into a computer system. The program may also be transmitted via a telecommunications line.

[0129] The support device 1 is an example of a computer program generation support device. The designer 991 is an example of an output source. The information does not necessarily need to be input to the input unit 102 by a person such as the designer 991, but may be input automatically by a device programmed with the information to be input in advance. The display unit 101 is an example of an output unit. The display control unit 111 is an example of an output control unit.

[0130] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0131] 100...support system, 1...support device, 2...management device, 3...conversation control device, 10...user interface, 11...control unit, 12...storage unit, 13...communication unit, 101...display unit, 102...input unit, 110...input control unit, 111...display control unit, 112...interaction flow information generation unit, 113...storage control unit, 114...communication control unit, 30...user interface, 31...control unit, 32...storage unit, 33...communication unit, 301...display unit, 302...input unit, 310...input control unit, 311...display control unit, 312...conversation control unit, 313...storage control unit, 314...communication control unit, 91...processor, 92...memory, 93...processor, 94...memory

Claims

1. A support device that supports implementation of an operation of a device to be controlled, an output control unit that controls the operation of an output unit that outputs information output by an output source and that outputs words input to an input unit that accepts input of words; Equipped with the output control unit, when information indicating an event that may occur in a human-device conversation that is an exchange between the device to be controlled and a user of the device to be controlled is input, causes the output unit to output the event indicated by the input information; the output control unit, when information indicating an event that may occur in the (N-1)th (N is an integer of 1 or more) order in the human-machine conversation is input to the input unit, causes the output unit to output information that prompts input of information indicating an event that may occur in the Nth order in the human-machine conversation; One of the events that may occur in the human-device conversation is a given word occurrence event, which is an event in which the device to be controlled outputs a word; the output unit displays information about candidates for response processing execution events in natural language. Support equipment.

2. Another event that may occur in the human-device conversation is a received speech occurrence event, which is an event in which the user inputs words into the device to be controlled. The support device according to claim 1 .

3. an interaction flow information generation unit that generates interaction flow information, which is text data information indicating the flow of events occurring in the human-machine conversation, based on the information input to the input unit; The assistance device according to claim 1 or 2, comprising:

4. a conversation control unit that controls the interaction between the user and the device to be controlled using interaction flow information, which is information generated by the assistance device, and is text data information indicating a flow of occurrence of events in the human-device conversation based on the information input to the input unit; and a conversation control unit that controls the interaction between the user and the device to be controlled using interaction flow information, which is information generated by the assistance device, and is text data information indicating a flow of occurrence of events in the human-device conversation based on the information input to the input unit. A conversation control device comprising:

5. A program that causes a computer to function as the support device according to claim 1 or 2.

Images