Information processing device, information processing system, information processing method, and program

Abstract

The present invention realizes a configuration for selecting, and then outputting, an optimal system utterance from among a plurality of system utterances generated by a plurality of dialogue execution modules for generating system utterances in accordance with algorithms different from each other. A data processing unit for generating and outputting a system utterance selects one system utterance from among a plurality of system utterances respectively generated by a plurality of dialogue execution modules and outputs the selected system utterance. The dialogue execution modules each generate, in accordance with an algorithm different from the others, a system utterance unique to the algorithm. The data processing unit selects one system utterance to be outputted according to confidence degrees set corresponding to the system utterances generated by the respective dialogue execution modules, or according to priorities corresponding to prescribed dialogue execution modules.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing system, an information processing method, and a program, and more particularly to an information processing device, an information processing system, an information processing method, and a program that execute processing based on a speech recognition result of a user's speech. [Background technology]

[0002] 2. Description of the Related Art In recent years, there has been an increase in the use of voice recognition systems that recognize speech uttered by a user and provide a response based on the recognition result. The voice recognition system analyzes user utterances input via a microphone and responds according to the analysis results. For example, when a user utters "Tell me the weather tomorrow," the weather information is acquired from a weather information server, a system response is generated based on the acquired information, and the generated response is output from a speaker. System utterance = "The weather will be sunny tomorrow, but there may be thunderstorms in the evening." The system outputs such an utterance.

[0003] Such a system speech output device has a data processing function for analyzing user utterances and generating responses based on the analysis results. The module that executes this data processing function is called a "dialogue execution module" or a "dialogue engine." There are various types of dialogue execution modules (dialogue engines).

[0004] For example, Patent Document 1 (JP Patent Publication 2003-280683 A) discloses a configuration for implementing a dialogue according to a specialized field using a dictionary for each field. By using the technology described in Patent Document 1, it becomes possible to have specialized conversations in the fields recorded in the dictionary. However, if the dictionary does not contain information for everyday conversation, it is possible that everyday conversation will not be carried out well.

[0005] Thus, depending on the type and functions of the dialogue execution module used by the device, there are cases where a smooth dialogue can be carried out, and cases where the dialogue is unnatural or does not take place at all. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2003-280683 A Summary of the Invention [Problem to be solved by the invention]

[0007] The present disclosure has been made in consideration of the above-mentioned problems, for example, and aims to provide an information processing device, an information processing system, an information processing method, and a program that enable optimal dialogue according to various situations by selectively using multiple different dialogue execution modules (dialogue engines). [Means for solving the problem]

[0008] A first aspect of the present disclosure is A data processing unit that generates and outputs a system utterance, The data processing unit includes: The information processing device selects and outputs one system utterance from a plurality of system utterances individually generated by a plurality of dialogue execution modules.

[0009] Furthermore, a second aspect of the present disclosure is An information processing system having a robot control device for controlling a dialogue robot and a server capable of communicating with the robot control device, The robot control device includes: outputting status information input via an input unit to the server; The server, a plurality of dialogue execution modules for generating system utterances according to different system utterance generation algorithms; Each of the plurality of dialogue execution modules generates an individual system utterance based on the situation information and transmits the individual system utterance to the robot control device; The robot control device includes: The information processing system selects and outputs one system utterance from a plurality of system utterances received from the server.

[0010] Furthermore, a third aspect of the present disclosure is An information processing method executed in an information processing device, The information processing device has a data processing unit that generates and outputs a system utterance, The data processing unit: The information processing method involves selecting and outputting one system utterance from a plurality of system utterances generated individually by a plurality of dialogue execution modules.

[0011] Furthermore, a fourth aspect of the present disclosure is An information processing method executed in an information processing system having a robot control device for controlling a dialogue robot and a server capable of communicating with the robot control device, The robot control device includes: outputting status information input via an input unit to the server; the server has a plurality of dialogue execution modules for generating system utterances according to different system utterance generation algorithms; Each of the plurality of dialogue execution modules generates an individual system utterance based on the situation information and transmits the individual system utterance to the robot control device; The robot control device, The information processing method includes selecting and outputting one system utterance from a plurality of system utterances received from the server.

[0012] Furthermore, a fifth aspect of the present disclosure is A program for causing an information processing device to execute information processing, The information processing device has a data processing unit that generates and outputs a system utterance, The program causes the data processing unit to The program selects and outputs one system utterance from multiple system utterances generated individually by multiple dialogue execution modules.

[0013] The program of the present disclosure is, for example, a program that can be provided by a storage medium or a communication medium in a computer-readable format to an information processing device or a computer system capable of executing various program codes. By providing such a program in a computer-readable format, processing according to the program is realized on the information processing device or the computer system.

[0014] Further objects, features, and advantages of the present disclosure will become apparent from the following detailed description of the embodiments of the present disclosure and the accompanying drawings. In this specification, a system refers to a logical collective configuration of multiple devices, and each device is not necessarily located in the same housing.

[0015] According to a configuration of an embodiment of the present disclosure, a configuration is realized in which an optimal system utterance is selected and output from a plurality of system utterances generated by a plurality of dialogue execution modules that generate system utterances according to different algorithms. Specifically, for example, a data processing unit that generates and outputs a system utterance selects and outputs one system utterance from multiple system utterances generated individually by multiple dialogue execution modules. Each of the multiple dialogue execution modules generates an algorithm-specific system utterance according to a different algorithm. The data processing unit selects one system utterance to be output according to a confidence level set corresponding to the system utterance generated by each of the multiple dialogue execution modules or a predetermined priority corresponding to the dialogue execution module. This configuration realizes a configuration in which an optimal system utterance is selected and output from a plurality of system utterances generated by a plurality of dialogue execution modules which generate system utterances according to different algorithms. It should be noted that the effects described in this specification are merely examples and are not limiting, and additional effects may also be provided. [Brief description of the drawings]

[0016] [Figure 1] 11 is a diagram for explaining a specific example of processing of a dialogue robot that responds to a user utterance. FIG. [Diagram 2] 11 is a diagram for explaining a specific example of processing of a dialogue robot that responds to a user utterance. FIG. [Diagram 3] FIG. 1 is a diagram illustrating an example of a configuration of an information processing device according to the present disclosure. [Figure 4] FIG. 1 is a diagram illustrating an example of a configuration of an information processing device according to the present disclosure. [Diagram 5] FIG. 2 is a diagram illustrating a process executed by an information processing device according to the present disclosure. [Figure 6] FIG. 2 is a diagram illustrating a process executed by an information processing device according to the present disclosure. [Figure 7] 2 is a diagram illustrating the configuration and processing of a processing decision unit (decision making unit) of the information processing device of the present disclosure. FIG. [Figure 8] 1 is a diagram showing a flowchart illustrating a sequence of processing executed by a processing decision unit (decision making unit) of an information processing device according to the present disclosure. [Figure 9] FIG. 2 is a diagram illustrating a process executed by a scenario-based dialogue execution module. [Figure 10] 2 is a diagram for explaining data stored in a scenario database to which a scenario-based dialogue execution module refers; FIG. [Figure 11] FIG. 11 is a flowchart illustrating the processing executed by a scenario-based dialogue execution module. [Figure 12] FIG. 2 is a diagram for explaining the processing executed by an episode knowledge base dialogue execution module. [Figure 13] 11 is a diagram for explaining data stored in an episode knowledge database to which an episode knowledge based dialogue execution module refers; FIG. [Figure 14] FIG. 13 is a flowchart illustrating the processing executed by an episode knowledge base dialogue execution module. [Figure 15] FIG. 13 is a diagram for explaining the processing executed by an RDF knowledge-based dialogue execution module. [Figure 16] 1 is a diagram for explaining data stored in an RDF knowledge database to which an RDF knowledge-based dialogue execution module refers; FIG. [Figure 17] FIG. 13 is a flowchart illustrating the processing executed by an RDF knowledge-based dialogue execution module. [Figure 18] FIG. 13 is a diagram for explaining the processing executed by a situation verbalization & RDF knowledge-based dialogue execution module. [Figure 19] FIG. 13 is a flowchart illustrating the processing executed by a situation verbalization & RDF knowledge-based dialogue execution module. [Figure 20] FIG. 11 is a diagram illustrating a process executed by a machine learning model-based dialogue execution module. [Figure 21] FIG. 11 is a flowchart illustrating the processing executed by a machine learning model-based dialogue execution module. [Figure 22] 11 is a diagram illustrating a process executed by an execution process determination unit. FIG. [Diagram 23] 11 is a diagram illustrating priority information corresponding to a dialogue execution module used by an execution process determination unit. FIG. [Figure 24] FIG. 11 is a flowchart illustrating a process executed by an execution process determination unit. [Diagram 25] 1 is a diagram illustrating a dialogue processing sequence executed by an information processing device according to the present disclosure. [Figure 26] 1 is a diagram illustrating a dialogue processing sequence executed by an information processing device according to the present disclosure. [Figure 27] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, the information processing device, the information processing system, the information processing method, and the program of the present disclosure will be described in detail with reference to the drawings. The description will be made according to the following items. 1. Overview of dialogue processing based on speech recognition of user utterances executed by the information processing device of the present disclosure 2. Configuration example of information processing device of the present disclosure 3. Specific configuration examples of the processing decision unit (decision making unit) and specific processing examples 4. Details of processing in the dialogue execution module (dialogue engine) 4-1. System utterance generation processing using the scenario-based dialogue execution module 4-2. System utterance generation processing using episode knowledge-based dialogue execution module 4-3. System utterance generation processing using the RDF knowledge-based dialogue execution module 4-4. System utterance generation processing using situation verbalization & RDF knowledge-based dialogue execution module 4-5. System utterance generation processing using machine learning model-based dialogue execution module 5. Details of the processing executed by the execution processing decision unit 6. Example of system speech output by the information processing device of the present disclosure 7. Examples of hardware configurations of information processing devices 8. Summary of the Disclosure

[0018] [1. Overview of dialogue processing based on speech recognition of user utterances executed by the information processing device of the present disclosure] First, an overview of dialogue processing based on speech recognition of user utterances executed by an information processing device of the present disclosure will be described with reference to FIG. 1 and subsequent drawings.

[0019] FIG. 1 is a diagram showing an example of processing of a dialogue robot 10, which is an example of an information processing device of the present disclosure, that recognizes a user utterance made by a user 1 and responds to the user utterance. The conversation robot 10 receives a user utterance, for example, User utterance = "I want to drink beer." A voice recognition process is executed on this user utterance. The data processing such as the voice recognition process may be executed by the interactive robot 10 itself, or by an external device capable of communicating with the interactive robot 10.

[0020] The interactive robot 10 executes a response process based on the result of voice recognition of the user's utterance. In the example shown in FIG. 1, data for responding to the user utterance "I want to drink beer" is acquired, a response is generated based on the acquired data, and the generated response is output via the speaker of the interactive robot 10.

[0021] In the example shown in FIG. 1, the interactive robot 10 performs the following system responses. System response = "Beer is Belgium." In this specification, utterances from a device such as a dialogue robot will be described as "system utterances" or "system responses."

[0022] The interactive robot 10 generates and outputs a response using knowledge data acquired from a storage unit within the device or knowledge data acquired via a network. That is, the system refers to a knowledge database and generates and outputs an optimal system response to the user's utterance.

[0023] In the example shown in Figure 1, Belgium is registered in the knowledge database as a region with delicious beer, and the system generates and outputs the optimal system response to the user's utterance by referring to the registered information in this knowledge database.

[0024] Figure 2 shows User utterance = "I want to go to Belgium and eat something delicious." In response to this user utterance, the interactive robot 10 performs the following system response. System response = "What's your favorite food?"

[0025] This system response differs from the system response in FIG. 1 described above in that it is not a system response that is generated and output as an optimal system response to a user utterance by referring to a knowledge database. The system response shown in FIG. 2 is a response process that utilizes a system response registered in the scenario database.

[0026] The scenario database stores optimal system utterances corresponding to various user utterances, and the interactive robot 10 searches the scenario database for registered data that matches or is similar to the user utterance, obtains system response data recorded in the searched registered data, and outputs the obtained system response. This results in a system response as shown in FIG.

[0027] In the dialogue processes shown in FIG. 1 and FIG. 2, the dialogue robot 10 performs processes according to different algorithms to generate and output system responses. For example, the user utterance shown in Fig. 2 User utterance = "I want to go to Belgium and eat something delicious", When a system utterance is generated for this user utterance by referring to a knowledge database in the same manner as in the process shown in FIG. 1, it is expected that the following system utterance will be generated, for example. System utterance = "The chocolate in Belgium is delicious."

[0028] In this way, if the algorithm for generating the system response executed on the interactive robot 10 side is different, there is a high possibility that the content of the response to the same user utterance will be completely different. Furthermore, if dialogue processing is performed using only one response generation algorithm, it may not be possible to generate an optimal system response, resulting in a system utterance that is completely unconsidered in response to a user utterance, or it may even be impossible for the system to respond.

[0029] The present disclosure is intended to solve such problems, and realizes optimal dialogue according to various situations by selectively using a plurality of different dialogue execution modules (dialogue engines). In other words, it is possible to change the response generation algorithm according to the situation, such as response generation processing using a knowledge database as shown in Figure 1, or response generation processing using a scenario database as shown in Figure 2, to enable the system to make optimal utterances.

[0030] [2. Configuration Example of Information Processing Device of the Present Disclosure] Next, a configuration example of the information processing device of the present disclosure will be described.

[0031] FIG. 3 is a diagram illustrating an example of the configuration of an information processing device according to the present disclosure. Figure 3 shows: (1) Information processing device configuration example 1 (2) Information processing device configuration example 2 Illustrated are configuration examples of these two information processing devices.

[0032] The information processing device configuration example 1 of (1) is a configuration of a single interactive robot 10. The interactive robot 10 is configured to execute all processes such as voice recognition processing of user utterances inputted via a microphone, and generation processing of system utterances.

[0033] The information processing device configuration example 2 of (2) is a device configured by a dialogue robot 10 and an external device connected to the dialogue robot 10. The external device is, for example, a server 21, a PC 22, a smartphone 23, or the like.

[0034] In this configuration, a user's utterance input from the microphone of the interactive robot 10 is transferred to an external device, which performs voice recognition of the user's utterance. The external device further generates a system utterance based on the voice recognition result. The external device transmits the generated system utterance to the interactive robot 10, which outputs it via a speaker.

[0035] In addition, in such a system configuration consisting of the interactive robot 10 and an external device, the division of the processing between the processing executed on the interactive robot 10 side and the processing executed on the external device side can be set in various ways.

[0036] Next, a specific configuration example of the information processing device of the present disclosure will be described with reference to FIG. FIG. 4 is a diagram showing an example of a configuration of the information processing device 100 of the present disclosure. The information processing device 100 is divided into a data input / output unit 110 and a robot control unit 150 .

[0037] The data input / output unit 110 is a component configured in the interactive robot shown in FIG. On the other hand, the robot control unit 150 is a component that can be configured in the interactive robot shown in Fig. 1 and other figures, but can also be configured in an external device that can communicate with the robot. The external device is, for example, a server on a cloud, a PC, a smartphone, or other device. The configuration may use one or more of these devices.

[0038] When the data input / output unit 110 and the robot control unit 150 are different devices, the data input / output unit 110 and the robot control unit 150 each have a communication unit, and input / output data to / from each other via the two communication units.

[0039] Note that only the main elements necessary to explain the processing of the present disclosure are shown in Fig. 4. Each of the data input / output unit 110 and the robot control unit 150 has, for example, a control unit that controls the respective execution processing, a storage unit that stores various data, a user operation unit, a communication unit, etc., but these configurations are not shown in the figure.

[0040] The main components of the data input / output unit 110 and the robot control unit 150 will be described below. The data input / output unit 110 has an input unit 120 and an output unit 130 . The input unit 120 includes a voice input unit (microphone) 121 , an image input unit (camera) 122 , and a sensor unit 123 . The output unit 130 includes an audio output unit (speaker) 131 and a drive control unit 132 .

[0041] A voice input unit (microphone) 121 of the input unit 120 inputs voice such as a user's speech. An image input unit (camera) 122 captures an image such as a face image of a user. The sensor unit 123 is configured with various sensors, such as a distance sensor, a temperature sensor, and an illuminance sensor. The data acquired by the input unit 120 is input to a state analysis unit 161 in a data processing unit 160 of the robot control unit 150.

[0042] When the data input / output unit 110 and the robot control unit 150 are configured as different devices, the data acquired by the input unit 120 is transmitted from the data input / output unit 110 to the robot control unit 150 via a communication unit.

[0043] Next, the output unit 130 of the data input / output unit 110 will be described. The audio output unit (speaker) 131 of the output unit 130 outputs the system utterance generated by the dialogue processing unit 164 in the data processing unit 160 of the robot control unit 150. The driving control unit 132 drives the interactive robot. For example, the interactive robot 10 shown in Fig. 1 has tires and other driving units and can move. For example, the robot can perform movement processing such as moving closer to the user. Such movement and other driving processing is executed in accordance with a driving command from the action processing unit 165 of the data processing unit 160 of the robot control unit 150.

[0044] Next, the configuration of the robot control unit 150 will be described. As described above, the robot control unit 150 can be configured within the interactive robot 10 shown in FIG. 1 and other figures, but can also be configured within an external device capable of communicating with the robot. The external device is, for example, a server on a cloud, a PC, a smartphone, etc. One or more of these devices may be used.

[0045] The robot control unit 150 has a data processing unit 160 and a communication unit 170. The communication unit 170 is configured to be able to communicate with an external server. The external server is a server that holds various databases, such as a knowledge database, that can be used to generate system utterances.

[0046] As explained above, although not shown in the figure, the robot control unit 150 also has a control unit that controls the processing of each part of the robot control unit 150, a memory unit, a communication unit that communicates with the data input / output unit 110, etc.

[0047] The data processing unit 160 includes a state analysis unit 161 , a situation analysis unit 162 , a process determination unit (decision making unit) 163 , a dialogue processing unit 164 , and an action processing unit 165 .

[0048] The state analysis unit 161 receives input information from the voice input unit (microphone) 121, image input unit (camera) 122, and sensor unit 123 of the input unit 120 of the data input / output unit 110, and performs state analysis based on the input information.

[0049] Specifically, the analysis unit 121 analyzes the user's speech inputted through the voice input unit (microphone) 121. The analysis unit 121 also analyzes image data inputted from the image input unit (camera) 122, and performs user identification processing based on the user's face image, user state analysis processing, and the like. The state analysis unit 161 executes the user identification process based on the user's face image by referring to a user DB in which the user's face image is registered in advance. The user DB is stored in a storage unit accessible by the data processing unit 160.

[0050] The state analysis unit 161 further analyzes the state, such as the distance to the user, the current temperature, and brightness, based on the sensor information input from the sensor unit 123 .

[0051] The status analysis unit 161 sequentially analyzes the information acquired by the voice input unit (microphone) 121, image input unit (camera) 122, and sensor unit 123 of the input unit 120, and outputs the analyzed status information to the situation analysis unit 152.

[0052] That is, the state analysis unit 161 outputs to the situation analysis unit 152, as necessary, time-series state information of the state acquired at time t1, the state acquired at time t2, and the state acquired at time t3. The state analysis unit 161 outputs state information, to which a time stamp indicating the acquisition time of the state information has been added, to the situation analysis unit 152 as needed.

[0053] The state information analyzed by the state analysis unit 161 includes information indicating each state of the device itself, the state of people, the state of things, and the state of a site. The status information of the device itself includes, for example, information that the device itself, i.e., an interactive robot having a data input / output unit 110, is charging, as well as various other status information such as the last action performed, remaining battery power, device temperature, falling, walking, and current emotional state.

[0054] The person's status information includes, for example, the name of the person included in the camera captured image, the person's facial expression, the person's position, angle, whether the person is speaking or not, the person's spoken text, and other status information.

[0055] The state information of an object includes, for example, the result of identifying the object contained in the image captured by the camera, the time when the object was last recognized, the location (angle, distance), and the like. The location status information includes information such as the brightness of the location, the temperature, and whether the location is indoors or outdoors.

[0056] The status analysis unit 161 sequentially generates status information composed of these various pieces of information based on the information acquired by the voice input unit (microphone) 121, the image input unit (camera) 122, and the sensor unit 123, and outputs the generated status information to the situation analysis unit 152 together with a timestamp indicating the time information at the time the information was acquired.

[0057] The status analysis unit 162 generates status information based on the status information for each time unit sequentially input from the status analysis unit 161, and outputs the generated status information to a process determination unit (decision making unit) 163. The situation analyzing unit 162 generates situation information having a data format that can be interpreted by a dialogue execution module (dialogue engine) in the process determining unit (decision making unit) 163.

[0058] The state analyzing unit 162 executes a voice recognition process for a user utterance input from the voice input unit (microphone) 121 via the state analyzing unit 161, for example. The voice recognition process of the user's speech in the situation analysis unit 162 includes a process of converting voice data to text data using ASR (Automatic Speech Recognition) or the like.

[0059] The process decision unit (decision making unit) 163 executes a process of selecting one system utterance from system utterances generated by a plurality of dialogue execution modules (dialogue engines) that generate system utterances according to a plurality of different algorithms.

[0060] Each of a plurality of dialogue execution modules (dialogue engines) that generates system utterances according to a plurality of different algorithms generates the system utterance based on the situation information generated by the situation analysis unit 162. The multiple dialogue execution modules (dialogue engines) may be configured inside the process determination unit (decision making unit) 163, or may be configured in an external server.

[0061] A specific example of the processing executed by the state analysis unit 161 and the situation analysis unit 162 will be described with reference to FIGS. FIG. 5 shows an example of state information generated by the state analysis unit 161 at a certain time t1.

[0062] That is, at time t1, the status analysis unit 161 inputs acquired information from the voice input unit (microphone) 121, image input unit (camera) 122, and sensor unit 123 of the input unit 120 of the data input / output unit 110, and generates the following status information based on the input information. State information = "Tanaka is facing you. Tanaka is talking. A stranger is in the distance. A plastic bottle is diagonally in front of you to the left..."

[0063] The state analysis unit 161 generates, for example, such state information. The state information generated by the state analysis unit 161 is sequentially input to the situation analysis unit 162 together with a time stamp.

[0064] A specific example of processing by the situation analysis unit 162 will be described with reference to Fig. 6. The situation analysis unit 162 generates situation information based on a plurality of pieces of state information generated by the state analysis unit 161, i.e., time-series state information. For example, the following situation information as shown in Fig. 6 is generated. Situation information = "Tanaka turned towards me. A stranger appeared. Tanaka said, 'I'm hungry.'"

[0065] The situation information generated by the situation analysis unit 162 is output to a process determination unit (decision making unit) 163. The process decision unit (decision making unit) 163 transfers this situation information to a number of dialogue execution modules (dialogue engines) which generate system utterances according to a number of different algorithms. Each of the multiple dialogue execution modules (dialogue engines) executes a system utterance generation algorithm specific to each module based on the situation information generated by the situation analysis unit 162, and individually generates a system utterance.

[0066] The process decision unit (decision making unit) 163 selects one system utterance to be output from among a plurality of system utterances generated by each of a plurality of dialogue execution modules (dialogue engines).

[0067] Each of the multiple dialogue execution modules (dialogue engines) applies a different algorithm to generate different system utterances, but the processing decision unit (decision making unit) 163 executes a process to select one system utterance to be output from these multiple system utterances. A specific example of the generation and selection process of the system utterance executed by the process decision unit (decision making unit) 163 will be described in detail later.

[0068] Furthermore, the process decision unit (decision making unit) 163 generates not only system utterances but also actions of the robot device, that is, drive control information.

[0069] The system utterance determined by the process determination unit (decision making unit) 163 is output to the dialogue processing unit 164. Moreover, the action of the robot apparatus determined by the process determination unit (decision making unit) 163 is output to an action processing unit 165.

[0070] The dialogue processor 164 generates an utterance text based on the system utterance determined by the process determiner (decision maker) 163, and controls the audio output unit (speaker) 131 of the output unit 130 to output the system utterance.

[0071] On the other hand, the action processing unit 165 generates drive information based on the action of the robot apparatus determined by the process determination unit (decision making unit) 163, and controls the drive control unit 132 of the output unit 130 to drive the robot.

[0072] [3. Specific configuration examples and specific processing examples of the processing decision unit (decision making unit)] Next, a specific configuration example and a specific processing example of the processing determination section (decision making section) 163 will be described.

[0073] As described above, the process determination unit (decision making unit) 163 selects one system utterance to be output from among a plurality of system utterances generated by each of a plurality of dialogue execution modules (dialogue engines).

[0074] Each of the multiple dialogue execution modules (dialogue engines), which generate system utterances according to multiple different algorithms, generates the next system utterance to be executed based on the situation information generated by the situation analysis unit 162, specifically, for example, a user utterance included in the situation information.

[0075] FIG. 7 shows a specific example of the configuration of the process determination unit (decision making unit) 163. In FIG. The example shown in FIG. 7 is a configuration example having the following five dialogue execution modules (dialogue engines) in the process determination section (decision making section) 163. (1) Scenario-based dialogue execution module 201 (2) Episode knowledge-based dialogue execution module 202 (3) RDF (Resource Description Framework) knowledge-based dialogue execution module 203 (4) Situation Verbalization & RDF Knowledge-Based Dialogue Execution Module 204 (5) Machine learning model-based dialogue execution module 205 These five dialogue execution modules (dialogue engines) execute parallel processing and each generate system responses using a different algorithm.

[0076] In addition, FIG. 7 shows an example in which five dialogue execution modules (dialogue engines) 201 to 205 are configured within the process determination unit (decision making unit) 163, but these five dialogue execution modules (dialogue engines) 201 to 205 may be configured separately in an external device such as an external server.

[0077] In this case, the processing decision unit (decision making unit) 163 communicates with an external device such as an external server via the communication unit 170. The processing decision unit (decision making unit) 163 transmits the situation information generated by the situation analysis unit 162, specifically, for example, situation information such as a user utterance included in the situation information, to the external device such as the external server via the communication unit 170. A dialogue execution module (dialogue engine) in an external device such as an external server generates a system utterance according to an algorithm specific to each module based on situation information such as received user utterances, and transmits it to a processing decision unit (decision making unit) 163.

[0078] The system utterances generated by the five dialogue execution modules (dialogue engines) 201 to 205 configured in the process decision unit (decision making unit) 163 or in an external device are input to the execution process decision unit 210 in the process decision unit (decision making unit) 163 shown in FIG. 7.

[0079] The execution process determination unit 210 inputs the system utterances generated by the five modules, and selects one system utterance to be output from the input system utterances. The selected system utterance is output to the dialogue processor 164, converted into text, and output via the voice output unit (speaker) 131.

[0080] Although the five modules 201 to 205 perform the process of generating the system utterance according to their respective algorithms, not all of the modules necessarily succeed in generating the system utterance. For example, all of the five modules may fail to generate the system utterance. In such a case, the execution process determination unit 210 determines the action of the robot and outputs the determined action to the action processing unit 165.

[0081] The action processing unit 165 generates drive information based on the action of the robot apparatus determined by the process determination unit (decision making unit) 163, and controls the drive control unit 132 of the output unit 130 to drive the robot.

[0082] Incidentally, the processing determination section (decision making section) 163 receives directly input of situation information generated by the situation analysis section 162, and may decide the action of the robot based on this situation information, for example, situation information other than user utterances.

[0083] Next, a processing sequence of the processing executed by the processing determination section (decision making section) 163 will be described with reference to FIG. FIG. 8 is a flowchart illustrating a sequence of processing executed by process determination section (decision making section) 163. As shown in FIG.

[0084] Processing according to this flow can be executed according to a program stored in a memory unit of the robot control unit 150 of the information processing device 100, and can be executed under the control of a control unit (data processing unit) having a processor such as a CPU with program execution function. The process of each step in the flow shown in FIG. 8 will be described below.

[0085] (Step S101) First, in step S101, the process determination unit (decision making unit) 163 determines whether or not the situation has been updated or whether or not a user utterance text has been input. Specifically, it is determined whether new situation information or a user utterance has been input from the situation analysis unit 162 to the process determination unit (decision making unit) 163.

[0086] When it is determined from the situation analysis unit 162 that new situation information or user utterance has not been input to the process determination unit (decision making unit) 163, the process remains at step S101. If it is determined that new situation information or a user utterance has been input to the process determination section (decision making section) 163 from the situation analysis section 162, the process proceeds to step S102.

[0087] (Step S102) If the situation analysis unit 162 determines that new situation information or user utterance has been input to the processing determination unit (decision making unit) 163, the processing determination unit (decision making unit) 163 determines in step S102, according to a predetermined algorithm, whether or not to execute a system utterance.

[0088] Specifically, the default algorithm is, for example, an algorithm that executes system utterances when a user utterance is input, and executes system utterances once every two times when a user utterance is not input, i.e., when there is only a change in the situation.

[0089] (Step S103) If it is determined in the process of determining whether or not to execute a system utterance in step S102 that a system utterance is to be executed, the processes of steps S111 to S115 are executed in parallel. The process of steps S111 to S115 is a process of generating a system utterance using different dialogue execution modules (dialogue engines).

[0090] On the other hand, if it is determined in the process of determining whether or not to execute a system utterance in step S102 that the system utterance is not to be executed, the process of step S104 is executed.

[0091] (Step S104) If it is determined in the process of determining whether or not to execute a system utterance in step S102 that the system utterance is not to be executed, the process proceeds to step S104, and no system utterance is output.

[0092] In this case, the process determination unit (decision making unit) 163 may output an instruction to the action processing unit 165 to cause the interactive robot to execute an action such as a movement process.

[0093] (Steps S111 to S115) If it is determined in the process of determining whether or not to execute a system utterance in step S102 that a system utterance is to be executed, the processes of steps S111 to S115 are executed in parallel. As described above, the processes in steps S111 to S115 are processes for generating system utterances using different dialogue execution modules (dialogue engines).

[0094] In steps S111 to S115, the following five processes are executed in parallel. (S111) Generation of system utterances by the scenario-based dialogue execution module (+utterance confidence) (executes processing by referring to the scenario DB) (S112) Generation of system utterances by episode knowledge-based dialogue execution module (+utterance confidence) (executes processing with reference to episode knowledge DB) (S113) Generation of system utterances by the RDF knowledge-based dialogue execution module (+utterance confidence) (performs processing by referencing the RDF knowledge DB) (S114) Generation of system utterances (+utterance confidence) by the RDF knowledge-based dialogue execution module with situation verbalization processing (performs processing by referring to the RDF knowledge DB) (S115) Generation of system utterances (+utterance confidence) by machine learning model-based dialogue execution module (performs processing with reference to the machine learning model)

[0095] These five processes are processes for generating system utterances using different dialogue execution modules (dialogue engines) 201-205. As mentioned above, the processing by these five dialogue execution modules (dialogue engines) 201 to 205 may be executed within the data processing unit 160 of the robot control unit 150 shown in FIG. 4, or may be executed using an external device such as an external server connected via the communication unit 170.

[0096] The five processes executed by the dialogue execution modules (dialogue engines) 201 to 205 will be described in detail later. In steps S111 to S115, a system utterance generation process is executed using five different dialogue execution modules (dialogue engines) 201 to 205, to which different algorithms are applied. Although each dialogue execution module (dialogue engine) generates a system utterance corresponding to one and the same situation information, for example, one and the same user utterance, each module generates a different system utterance because of different algorithms, and some modules may fail to generate a system utterance.

[0097] When generating a system utterance in steps S111 to S115, the five dialogue execution modules (dialogue engines) also generate a confidence value which is an index value indicating the confidence of the generated system utterance and output it to the execution process determination unit 210.

[0098] Each dialogue execution module (dialogue engine), for example, outputs a confidence level of 1.0 if it succeeds in generating a system utterance, and outputs a confidence level of 0.0 if it fails to generate a system utterance. However, in cases where the utterance has been repeated many times in the past, or where the accuracy of the created system utterance is low, the confidence level may be set to a value between 0.0 and 1.0, for example, 0.5.

[0099] (Step S121) After the processing of steps S111 to S115, the execution process decision unit 210 of the process decision unit (decision making unit) 163 shown in FIG. 7 inputs a plurality of different system utterances generated based on different algorithms from a plurality of dialogue execution modules (dialogue engines) 201 to 205.

[0100] In step S121, the execution process determination unit 210 selects one system utterance having the highest confidence value from the multiple system utterances input from the multiple dialogue execution modules (dialogue engines), and sets the selected system utterance as the system utterance to be output by the dialogue robot.

[0101] If the confidence levels input from multiple dialogue execution modules (dialogue engines) are equal, the system utterance to be output by the dialogue robot is determined according to the priority of each dialogue execution module (dialogue engine) that is set in advance. Details of this process will be explained later.

[0102] Each of the dialogue execution modules (dialogue engines) 201 to 205 may be configured to output only the system utterance, and not to output the confidence value. In this configuration, the execution process determination unit 210 executes the following processes. When a system utterance is input from the dialogue execution module (dialogue engine), the confidence level of the system utterance is set to 1.0. When no system utterance is input from the dialogue execution module (dialogue engine), the confidence level of the system utterance is set to 0.0.

[0103] In step S121, the execution process determination unit 210 selects one system utterance as a system utterance to be output from among a plurality of system utterances input from a plurality of dialogue execution modules (dialogue engines). This selection process is performed taking into consideration the confidence value associated with the system utterance generated by each module and the preset priority of each module. The details of this process will be explained later.

[0104] (Step S122) Finally, in step S122, the process determining unit (decision making unit) 163 causes the interactive robot to output the one system utterance selected in step S121.

[0105] Specifically, the system utterance determined by the processing determination unit (decision making unit) 163 is output to the dialogue processing unit 164. The dialogue processing unit 164 generates an utterance text based on the input system utterance, and controls the audio output unit (speaker) 131 of the output unit 130 to output the system utterance.

[0106] [4. Details of processing in the dialogue execution module (dialogue engine)] Next, the details of the system utterance generation process using the different dialogue execution modules (dialogue engines) 201 to 205 executed in steps S111 to S115 of the flow shown in FIG. 8 will be described.

[0107] As described above, in steps S111 to S115 of the flow shown in FIG. 8, the following five processes are executed in parallel. (S111) Generation of system utterances (+utterance confidence) by the scenario-based dialogue execution module 201 (executing processing by referring to the scenario DB) (S112) Generation of system utterances (+utterance confidence) by episode knowledge-based dialogue execution module 202 (executing processing with reference to episode knowledge DB) (S113) Generation of system utterances (+utterance confidence) by the RDF knowledge-based dialogue execution module 203 (executing processing by referring to the RDF knowledge DB) (S114) Generation of system utterances (+utterance confidence) by the RDF knowledge-based dialogue execution module 204 with situation verbalization processing (executing processing with reference to the RDF knowledge DB) (S115) Generation of system utterances (+utterance confidence) by the machine learning model-based dialogue execution module 205 (executing processing with reference to the machine learning model)

[0108] As described above, these five processes may be executed within the data processing unit 160 of the robot control unit 150 shown in FIG. 4, or may be executed by an external device such as an external server connected via the communication unit 170. For example, five external servers may each execute the five processes of steps S111 to S115, and the process results may be received by a process decision unit (decision making unit) 163 in a data processing unit 160 of the robot control unit 150 shown in FIG.

[0109] The processes executed by these five dialogue execution modules (dialogue engines) 201 to 205 will be described in detail below in turn. (4-1. System utterance generation processing by the scenario-based dialogue execution module) First, the process of generating a system utterance by the scenario-based dialogue execution module 201 executed in step S111 of the flow shown in FIG. 8 will be described.

[0110] The process of generating system utterances by the scenario-based dialogue execution module 201 will be described in detail with reference to FIG. 9 shows a scenario-based dialogue execution module 201. The scenario-based dialogue execution module 201 generates a system utterance by referring to scenario data stored in a scenario DB (database) 211 shown in FIG. A scenario DB (database) 211 is a database installed in the robot control unit 150 or in an external device such as an external server.

[0111] The scenario-based dialogue execution module 201 and the scenario DB (database) 211 may be configured within the robot control unit 150 of the information processing device 100 shown in FIG. 4, or may be configured to be included in an external server capable of communicating with the information processing device 100.

[0112] The scenario-based dialogue execution module 201 executes the processes in the order of steps S11 to S14 shown in Fig. 9. That is, the scenario-based system utterance generation algorithm is executed to generate a scenario-based system utterance. First, in step S11, a user utterance is input from the situation analysis unit 162. For example, the following user utterance is input: User utterance = "Good morning"

[0113] Next, in step S12, the scenario-based dialogue execution module 201 executes a matching process between the input user utterance and the data registered in the scenario DB.

[0114] A scenario DB (database) 211 is a database in which utterance pair data of user utterances and system utterances corresponding to various dialogue scenarios are registered. A specific example of data registered in the scenario DB (database) 211 is shown in FIG. As shown in FIG. 10, a scenario DB (database) 211 stores utterance pair data of user utterances and system utterances for each of various dialogue scenarios (scenario ID=1, 2, . . . ).

[0115] Each entry contains an optimal system utterance that the dialogue robot (system) should execute in response to a certain user utterance. This scenario DB is a database in which optimal system utterances corresponding to user utterances are registered in advance according to various dialogue scenarios.

[0116] In step S12, the scenario-based dialogue execution module 201 executes a search process to check whether a user utterance that matches or is similar to the input user utterance is registered in the scenario DB, i.e., a matching process between the input user utterance and DB registered data.

[0117] Next, in step S13, the scenario-based dialogue execution module 201 acquires the scenario DB registered data that has the highest matching rate with respect to the input user utterance. The scenario DB (database) 211 shown in FIG. The registered data for scenario ID = (S1) is as follows: User utterance = Good morning / System utterance = Good morning, let's do our best today is registered.

[0118] In step S13, the scenario-based dialogue execution module 201 acquires this database registration data. That is, the following system utterances are acquired from the scenario DB (database) 211. System utterance = "Good morning, let's do our best today too."

[0119] Next, in step S14, the scenario-based dialogue execution module 201 outputs the system utterance acquired from the scenario DB (database) 211 to the execution process determination unit 210 shown in FIG.

[0120] When outputting this system utterance, the scenario-based dialogue execution module 201 may be configured to generate a confidence value, which is an index value indicating the confidence of the output system utterance, for example, a confidence value = 0.0 to 1.0, and output it to the execution process determination unit 210 together with the system utterance. For example, if the system succeeds in generating a system utterance, it outputs a confidence level of 1.0, and if the system fails to generate a system utterance, it outputs a confidence level of 0.0. As described above, each dialogue execution module (dialogue engine) can be configured to output only the system utterance and not output the confidence value.

[0121] Next, the processing sequence executed by the scenario-based dialogue execution module 201 will be described with reference to the flowchart shown in FIG. The process of each step in the flow shown in FIG. 11 will be explained in order.

[0122] (Step S211) First, in step S211, it is determined whether or not a user utterance has been input from the situation analysis unit 162, and if it is determined that an utterance has been input, the process proceeds to step S212.

[0123] (Step S212) Next, in step S212, the scenario-based dialogue execution module 201 determines whether or not user utterance data that matches or is similar to the input user utterance is registered in the scenario DB 211.

[0124] As described above with reference to FIG. 10, the scenario DB (database) 211 is a database in which utterance pair data of user utterances and system utterances corresponding to various dialogue scenarios are registered. In step S212, the scenario-based dialogue execution module 201 executes a search process to check whether a user utterance that matches or is similar to the input user utterance is registered in the scenario DB 211, i.e., a matching process between the input user utterance and DB registered data.

[0125] If it is determined that a user utterance that matches or is similar to the input user utterance is registered in the scenario DB 211, the process proceeds to step S213. If it is determined that a user utterance that matches or is similar to the input user utterance is not registered in the scenario DB 211, the process proceeds to step S214.

[0126] (Step S213) If it is determined in step S212 that a user utterance that matches or is similar to the input user utterance is registered in the scenario DB 211, the process proceeds to step S213.

[0127] In step S213, the scenario-based dialogue execution module 201 acquires from the scenario DB 211 a system utterance recorded corresponding to the registered user utterance in the scenario DB that has the highest matching rate with the input user utterance, and outputs the acquired system utterance to the execution process decision unit 210 shown in Figure 7.

[0128] In addition to the output of the system utterance, a value of confidence, which is an index value indicating the degree of confidence of the acquired system utterance, may also be output to the execution process determination unit 210. In this case, since the system utterance was successfully generated (acquired), the confidence level is output as 1.0.

[0129] (Step S214) On the other hand, if it is determined in step S212 that a user utterance that matches or is similar to the input user utterance is not registered in the scenario DB 211, the process proceeds to step S214.

[0130] In step S214, the scenario-based dialogue execution module 201 does not output a system utterance to the execution process determination unit 210. When outputting a value of confidence, which is an index value indicating the confidence of a system utterance, a value of confidence=0.0 is output to the execution process determination unit 210 since generation (acquisition) of the system utterance has failed.

[0131] (4-2. System utterance generation processing using episode knowledge-based dialogue execution module) Next, a process of generating a system utterance by the episode knowledge base dialogue execution module 202 executed in step S112 of the flow shown in FIG. 8 will be described.

[0132] The details of the system utterance generation process performed by the episode knowledge base dialogue execution module 202 will be described with reference to FIG. 12 shows the episode knowledge based dialogue execution module 202. The episode knowledge based dialogue execution module 202 generates a system utterance by referring to episode knowledge data stored in an episode knowledge DB (database) 212 shown in FIG. The episode knowledge DB (database) 212 is a database installed in the robot control unit 150 or in an external device such as an external server.

[0133] The episode knowledge base dialogue execution module 202 and the episode knowledge DB (database) 212 may be configured within the robot control unit 150 of the information processing device 100 shown in FIG. 4, or may be configured to be included in an external server capable of communicating with the information processing device 100.

[0134] The episode knowledge-based dialogue execution module 202 executes the processes in the order of steps S21 to S24 shown in Fig. 12. That is, the episode knowledge-based system utterance generation algorithm is executed to generate the episode knowledge-based system utterance. First, in step S21, a user utterance is input from the situation analysis unit 162. For example, the following user utterance is input: User utterance = "What did Oda Nobunaga do at Okehazama?"

[0135] Next, in step S22, the episode knowledge base dialogue execution module 202 executes a search process for registered data in the episode knowledge DB 212 based on the input user utterance.

[0136] The episode knowledge DB (database) 212 is a database that records various episode information such as historical facts, news, and events related to the user. The episode knowledge DB 212 is updated from time to time. For example, it is updated based on information input via the input unit 120 of the data input / output unit 120 of the interactive robot.

[0137] A specific example of registered data in episode knowledge DB (database) 212 is shown in FIG. As shown in FIG. 13, the episode knowledge DB (database) 212 records data indicating episode details for each of various dialogue episodes (episode ID (Ep_id)=1, 2, . . . ).

[0138] Specifically, the following information is recorded for each episode: When, Who, Where Action, State = What was done and what state was it in Target=What / what with=with whom Why, How = Why, How, Purpose Cause = What happened as a result? A database in which this information is recorded on an episode-by-episode basis is the episode knowledge DB (database) 212.

[0139] By referring to the registered information in the episode knowledge DB (database) 212, detailed information on various episodes can be obtained.

[0140] In step S22, the episode knowledge base dialogue execution module 202 executes a search process of the episode knowledge DB registered data based on the input user utterance. The process when the following user utterance is input will be described. User utterance = "What did Oda Nobunaga do at Okehazama?"

[0141] In this case, in step S23, the episode knowledge based dialogue execution module 202 extracts the entry with episode ID (Ep_id)=Ep1 from the episode knowledge DB registration data shown in FIG. 13 as the episode that contains the most words that match the words included in the user utterance.

[0142] Next, in step S24, the episode knowledge based dialogue execution module 202 generates a system utterance based on the episode detailed information contained in the entry for episode ID (Ep_id) = Ep1 obtained from the episode knowledge DB (database) 212, and outputs it to the execution processing decision unit 210 shown in Figure 7. For example, the following system utterance is generated and output to the execution process determination unit 210: System speech = "I defeated Imagawa Yoshimoto with a surprise attack."

[0143] In addition, when outputting this system utterance, the episode knowledge based dialogue execution module 202 may be configured to generate a confidence value, which is an index value indicating the confidence of the output system utterance, for example, confidence = 0.0 to 1.0, and output it to the execution process determination unit 210 together with the system utterance. For example, if the system succeeds in generating a system utterance, it outputs a confidence level of 1.0, and if the system fails to generate a system utterance, it outputs a confidence level of 0.0. As described above, each dialogue execution module (dialogue engine) can be configured to output only the system utterance and not output the confidence value.

[0144] Next, the processing sequence executed by the episode knowledge base dialogue execution module 202 will be described with reference to the flowchart shown in FIG. The process of each step in the flow shown in FIG. 14 will be explained in order.

[0145] (Step S221) First, in step S221, it is determined whether or not a user utterance has been input from the situation analysis unit 162, and if it is determined that an utterance has been input, the process proceeds to step S222.

[0146] (Step S222) Next, in step S222, the episode knowledge based dialogue execution module 202 determines whether episode data including a phrase that matches or is similar to a phrase included in the input user utterance is registered in the episode knowledge DB 212 or not.

[0147] The episode knowledge DB (database) 212, as described above with reference to FIG. 13, is a database in which detailed information on various dialogue episodes is registered. In step S222, the episode knowledge based dialogue execution module 202 determines whether episode data including a phrase that matches or is similar to a phrase included in the input user utterance is registered in the episode knowledge DB 212 or not.

[0148] If it is determined that episode data including a phrase that matches or is similar to a phrase included in the input user utterance is registered in the episode knowledge DB 212, the process proceeds to step S223. If it is determined that episode data including a phrase that matches or is similar to a phrase included in the input user utterance is not registered in the episode knowledge DB 212, the process proceeds to step S224.

[0149] (Step S223) If it is determined in step S222 that episode data including a phrase that matches or is similar to a phrase included in the input user utterance is registered in the episode knowledge DB 212, the process proceeds to step S223.

[0150] In step S223, the episode knowledge based dialogue execution module 202 generates a system utterance based on the episode detail information included in the episode acquired from the episode knowledge DB 212, and outputs the system utterance to the execution process determination unit 210 shown in FIG.

[0151] In addition to the output of the system utterance, a value of confidence, which is an index value indicating the degree of confidence of the acquired system utterance, may also be output to the execution process determination unit 210. In this case, since the system utterance was successfully generated (acquired), the confidence level is output as 1.0.

[0152] (Step S224) On the other hand, if it is determined in step S222 that episode data including a phrase that matches or is similar to the phrase included in the input user utterance is not registered in the episode knowledge DB 212, the process proceeds to step S224.

[0153] In step S224, the episode knowledge base dialogue execution module 202 does not output a system utterance to the execution process determination unit 210. When outputting a value of confidence, which is an index value indicating the confidence of a system utterance, a value of confidence=0.0 is output to the execution process determination unit 210 since generation (acquisition) of the system utterance has failed.

[0154] (4-3. System utterance generation processing using the RDF knowledge-based dialogue execution module) Next, a process of generating a system utterance by the RDF (Resource Description Framework) knowledge-based dialogue execution module 203 executed in step S113 of the flow shown in FIG. 8 will be described.

[0155] The details of the system utterance generation process performed by the RDF knowledge-based dialogue execution module 203 will be described with reference to FIG. 15 shows the RDF knowledge-based dialogue execution module 203. The RDF knowledge-based dialogue execution module 203 generates a system utterance by referring to the RDF knowledge data stored in the RDF knowledge DB (database) 213 shown in FIG. The RDF knowledge DB (database) 213 is a database installed in the robot control unit 150 or in an external device such as an external server.

[0156] The RDF knowledge-based dialogue execution module 203 and the RDF knowledge DB (database) 213 may be configured within the robot control unit 150 of the information processing device 100 shown in FIG. 4, or may be configured as being possessed by an external server capable of communicating with the information processing device 100.

[0157] The RDF knowledge-based dialogue execution module 203 executes the processes in the order of steps S31 to S34 shown in Fig. 15. That is, the RDF knowledge-based system utterance generation algorithm is executed to generate the RDF knowledge-based system utterance.

[0158] RDF stands for Resource Description Framework, a framework for describing information (resources) primarily on the Web, and is a framework standardized by the W3C. RDF is a framework for describing the relationships between elements, and describes relationship information about information (resources) using three elements: subject, predicate, and object.

[0159] For example, the information (resource) "Dachshunds are dogs" is Subject = Dachshund Predicate = ia a Object = dog It is described as information classified into these three elements, with the relationship between the three elements determined.

[0160] Data recording such relationships between elements is recorded in the RDF knowledge database 213. An example of data stored in the RDF knowledge database 213 is shown in FIG. As shown in FIG. 16, the RDF knowledge database 213 contains the following information: (a) Predicate (b) Subject (c) Object These three elements are divided and recorded. By referring to the registered information in the RDF knowledge DB (database) 213, it is possible to know the elements contained in various information and the relationships between the elements.

[0161] The RDF knowledge-based dialogue execution module 203 refers to the registered data in the RDF knowledge DB (database) 213, which records the elements contained in various information and the relationships between the elements, and generates the optimal system utterance in response to the user utterance. The RDF knowledge-based dialogue execution module 203 executes the processes in the order of steps S31 to S34 shown in Fig. 15. That is, the RDF knowledge-based system utterance generation algorithm is executed to generate the RDF knowledge-based system utterance.

[0162] First, in step S31, a user utterance is input from the situation analysis unit 162. For example, the following user utterance is input: User utterance = "What is a dachshund?"

[0163] Next, in step S32, the RDF knowledge base dialogue execution module 203 executes a search process of the RDF knowledge DB registered data based on the input user utterance.

[0164] As described above with reference to FIG. 16, the RDF knowledge DB (database) 213 stores various types of information. (a) Predicate (b) Subject (c) Object This is a database that records information divided into these three elements. By referring to the registered information in the RDF knowledge DB (database) 213, it is possible to know the elements contained in various information and the relationships between the elements.

[0165] In step S32, the RDF knowledge base dialogue execution module 203 executes a search process of the RDF knowledge DB registered data based on the input user utterance. The process when the following user utterance is input will be described. User utterance = "What is a dachshund?"

[0166] In this case, in step S33, the RDF knowledge-based dialogue execution module 203 extracts information (resource) with resource ID = (R1) from the RDF knowledge DB registration data shown in Figure 16 as the information (resource) that contains the most words that match the words included in the user utterance.

[0167] Next, in step S34, the RDF knowledge-based dialogue execution module 203 retrieves information included in the entry of the resource ID (R1) obtained from the RDF knowledge DB (database) 213, i.e., Subject = Dachshund Predicate = ia a Object = dog A system utterance is generated based on these elements and inter-element information, and is output to the execution process determination unit 210 shown in FIG. For example, the following system utterance is generated and output to the execution process determination unit 210: System utterance = "Dachshunds are dogs."

[0168] When outputting this system utterance, the RDF knowledge-based dialogue execution module 203 may be configured to generate a confidence value, which is an index value indicating the confidence of the output system utterance, for example, a confidence value = 0.0 to 1.0, and output it to the execution process determination unit 210 together with the system utterance. For example, if the system succeeds in generating a system utterance, it outputs a confidence level of 1.0, and if the system fails to generate a system utterance, it outputs a confidence level of 0.0. As described above, each dialogue execution module (dialogue engine) can be configured to output only the system utterance and not output the confidence value.

[0169] Next, the processing sequence executed by the RDF knowledge base dialogue execution module 203 will be described with reference to the flowchart shown in FIG. The process of each step in the flow shown in FIG. 17 will be explained in order.

[0170] (Step S231) First, in step S231, it is determined whether or not a user utterance has been input from the situation analysis unit 162, and if it is determined that an utterance has been input, the process proceeds to step S232.

[0171] (Step S232) Next, in step S232, the RDF knowledge-based dialogue execution module 203 determines whether or not resource data including a phrase that matches or is similar to a phrase included in the input user utterance is registered in the RDF knowledge DB 213.

[0172] As described above with reference to FIG. 16, the RDF knowledge DB (database) 213 is a database that records elements constituting various information (resources) and relationships between the elements. In step S232, the RDF knowledge-based dialogue execution module 203 determines whether or not information (resource) including a phrase that matches or is similar to a phrase included in the input user utterance is registered in the RDF knowledge DB 213.

[0173] If it is determined that information (resource) including a phrase that matches or is similar to the phrase included in the input user utterance is registered in the RDF knowledge DB 213, the process proceeds to step S233. If it is determined that information (resource) including a phrase that matches or is similar to the phrase included in the input user utterance is not registered in the RDF knowledge DB 213, the process proceeds to step S234.

[0174] (Step S233) If it is determined in step S232 that information (resource) including a phrase that matches or is similar to the phrase included in the input user utterance is registered in the RDF knowledge DB 213, the process proceeds to step S233.

[0175] In step S233, the RDF knowledge-based dialogue execution module 203 acquires information (resources) including phrases that match or are similar to the phrases included in the input user utterance from the RDF knowledge DB 213, generates a system utterance based on the acquired information, and outputs it to the execution process determination unit 210 shown in Figure 7.

[0176] In addition to the output of the system utterance, a value of confidence, which is an index value indicating the degree of confidence of the acquired system utterance, may also be output to the execution process determination unit 210. In this case, since the system utterance was successfully generated (acquired), the confidence level is output as 1.0.

[0177] (Step S234) On the other hand, if it is determined in step S232 that information (resource) including a phrase that matches or is similar to the phrase included in the input user utterance is not registered in the RDF knowledge DB 213, the process proceeds to step S234.

[0178] In step S 234 , the RDF knowledge base dialogue execution module 203 does not output a system utterance to the execution process determination unit 210 . When outputting a value of confidence, which is an index value indicating the confidence of a system utterance, a value of confidence=0.0 is output to the execution process determination unit 210 since generation (acquisition) of the system utterance has failed.

[0179] (4-4. System utterance generation processing using situation verbalization & RDF knowledge-based dialogue execution module) Next, a process of generating a system utterance by the situation verbalization & RDF (Resource Description Framework) knowledge-based dialogue execution module 204 executed in step S114 of the flow shown in FIG. 8 will be described.

[0180] The details of the process of generating system utterances by the situation verbalization & RDF knowledge-based dialogue execution module 204 will be described with reference to FIG. 18 shows the situation verbalization & RDF knowledge-based dialogue execution module 204. The situation verbalization & RDF knowledge-based dialogue execution module 204 generates system utterances by referring to RDF knowledge data stored in the RDF knowledge DB (database) 213 shown in FIG. The RDF knowledge DB (database) 213 is a database installed in the robot control unit 150 or in an external device such as an external server.

[0181] The RDF knowledge DB (database) 213 shown in Fig. 18 is a database similar to the RDF knowledge DB (database) 213 previously described with reference to Fig. 15 and Fig. 16. That is, it is a database in which various pieces of information (resources) are classified into three elements, namely, subject, predicate, and object, and the relationships between the elements are recorded.

[0182] The situation verbalization & RDF knowledge-based dialogue execution module 204 and the RDF knowledge DB (database) 213 may be configured within the robot control unit 150 of the information processing device 100 shown in Figure 4, or may be configured to be possessed by an external server capable of communicating with the information processing device 100.

[0183] The situation verbalization & RDF knowledge-based dialogue execution module 204 executes the processes in the order of steps S41 to S45 shown in Fig. 15. That is, it executes the situation verbalization & RDF knowledge-based system utterance generation algorithm to generate the situation verbalization & RDF knowledge-based system utterance.

[0184] First, in step S41, the situation verbalization & RDF knowledge-based dialogue execution module 204 inputs situation information from the situation analysis unit 162. Here, situation information based on, for example, an image captured by a camera is input, rather than an utterance from a user. For example, enter the following status information: Status information = "Taro has just appeared"

[0185] Next, in step S42, the situation verbalization & RDF knowledge-based dialogue execution module 204 executes verbalization processing of the input situation information. This is a process of describing the observed situation as text information similar to user utterances. For example, the following situation verbalization information is generated: Situational verbalization information = Taro has just appeared

[0186] Next, in step S43, the situation verbalization & RDF knowledge base dialogue execution module 204 executes a search process for registered data in the RDF knowledge DB 213 based on the generated situation verbalization information.

[0187] As described above with reference to FIG. 16, the RDF knowledge DB (database) 213 stores various types of information. (a) Predicate (b) Subject (c) Object This is a database that records information divided into these three elements. By referring to the registered information in the RDF knowledge DB (database) 213, it is possible to know the elements contained in various information and the relationships between the elements.

[0188] In step S43, the situation verbalization & RDF knowledge base dialogue execution module 204 executes a search process of the RDF knowledge DB registered data based on the generated situation verbalization information. The processing for the situation verbalization information will be described below. Situational verbalization information = Taro has just appeared

[0189] In this case, in step S44, the situation verbalization & RDF knowledge-based dialogue execution module 204 extracts information (resources) that contains the most words matching the words contained in the situation verbalization information from the RDF knowledge DB registration data.

[0190] Next, in step S45, the situation verbalization & RDF knowledge-based dialogue execution module 204 generates a system utterance based on the information acquired from the RDF knowledge DB (database) 213, and outputs it to the execution process determination unit 210 shown in FIG. For example, the following system utterance is generated and output to the execution process determination unit 210: System utterance = "Oh, Taro has just arrived."

[0191] In addition, when outputting this system utterance, the situation verbalization & RDF knowledge-based dialogue execution module 204 may be configured to generate a confidence value, which is an index value indicating the confidence of the output system utterance, for example, confidence = 0.0 to 1.0, and output it to the execution process decision unit 210 together with the system utterance. For example, if the system succeeds in generating a system utterance, it outputs a confidence level of 1.0, and if the system fails to generate a system utterance, it outputs a confidence level of 0.0. As described above, each dialogue execution module (dialogue engine) can be configured to output only the system utterance and not output the confidence value.

[0192] Next, the processing sequence executed by the situation verbalization & RDF knowledge base dialogue execution module 204 will be described with reference to the flowchart shown in FIG. The process of each step in the flow shown in FIG. 19 will be described in order.

[0193] (Step S241) First, in step S241, it is determined whether or not situation information has been input from the situation analysis unit 162, and if it is determined that situation information has been input, the process proceeds to step S242.

[0194] (Step S242) Next, in step S242, the situation verbalization & RDF knowledge-based dialogue execution module 204 executes verbalization processing of the input situation information.

[0195] (Step S243) Next, in step S243, the situation verbalization & RDF knowledge-based dialogue execution module 204 determines whether or not resource data containing phrases that match or are similar to the phrases contained in the situation verbalization data generated in step S242 is registered in the RDF knowledge DB 213.

[0196] As described above with reference to FIG. 16, the RDF knowledge DB (database) 213 is a database that records elements constituting various information (resources) and relationships between the elements. In step S243, the situation verbalization & RDF knowledge-based dialogue execution module 204 determines whether information (resource) including a phrase that matches or is similar to a phrase included in the generated situation verbalization data is registered in the RDF knowledge DB 213.

[0197] If it is determined that information (resource) including a phrase that matches or is similar to the phrase included in the generated situation verbalization data is registered in the RDF knowledge DB 213, the process proceeds to step S244. If it is determined that information (resource) including a phrase that matches or is similar to the phrase included in the generated situation verbalization data is not registered in the RDF knowledge DB 213, the process proceeds to step S245.

[0198] (Step S244) If it is determined in step S243 that information (resource) including a phrase that matches or is similar to the phrase included in the generated situation verbalization data is registered in the RDF knowledge DB 213, the process proceeds to step S244.

[0199] In step S244, the situation verbalization & RDF knowledge-based dialogue execution module 204 acquires information (resources) including phrases that match or are similar to the phrases included in the generated situation verbalization data from the RDF knowledge DB 213, generates a system utterance based on the acquired information, and outputs it to the execution processing decision unit 210 shown in Figure 7.

[0200] In addition to the output of the system utterance, a value of confidence, which is an index value indicating the degree of confidence of the acquired system utterance, may also be output to the execution process determination unit 210. In this case, since the system utterance was successfully generated (acquired), the confidence level is output as 1.0.

[0201] (Step S245) On the other hand, if it is determined in step S243 that information (resources) including phrases that match or are similar to the phrases included in the generated situation verbalization data is not registered in the RDF knowledge DB 213, the process proceeds to step S245.

[0202] In step S245, the situation verbalization & RDF knowledge base dialogue execution module 204 does not output a system utterance to the execution process determination unit 210. When outputting a value of confidence, which is an index value indicating the confidence of a system utterance, a value of confidence=0.0 is output to the execution process determination unit 210 since generation (acquisition) of the system utterance has failed.

[0203] (4-5. System utterance generation processing using machine learning model-based dialogue execution module) Next, a process of generating a system utterance by the machine learning model-based dialogue execution module 205 executed in step S115 of the flow shown in FIG. 8 will be described.

[0204] The process of generating system utterances by the machine learning model-based dialogue execution module 205 will be described in detail with reference to FIG. 20 shows the machine learning model-based dialogue execution module 205. The machine learning model-based dialogue execution module 205 inputs a user utterance to a machine learning model 215 shown in FIG. The machine learning model 215 is installed in the robot control unit 150 or in an external device such as an external server.

[0205] The machine learning model 215 shown in Fig. 20 is a learning model that inputs a user utterance and outputs a system utterance as an output. This machine learning model is a learning model generated by machine learning processing of a large number of different sets of input sentences and response sentences, that is, data consisting of sets of user utterances and output utterances (system utterances). This learning model is, for example, a learning model for each user, and is updated sequentially.

[0206] The machine learning model-based dialogue execution module 205 and the machine learning model 215 may be configured within the robot control unit 150 of the information processing device 100 shown in FIG. 4, or may be configured to be included in an external server capable of communicating with the information processing device 100.

[0207] The machine learning model-based dialogue execution module 205 executes the processes in the order of steps S51 to S52 shown in Fig. 20. That is, the machine learning model-based system utterance generation algorithm using the machine learning model is executed to generate a machine learning model-based system utterance.

[0208] First, the machine learning model-based dialogue execution module 205 inputs a user utterance from the situation analysis unit 162 in step S51. For example, the following user utterance is input: User utterance = "Yesterday's game was awesome."

[0209] Next, in step S52, the machine learning model-based dialogue execution module 204 inputs the input user utterance “Yesterday's game was seriously the best” to the machine learning model 215.

[0210] The machine learning model 215 is a learning model that outputs a system utterance when a user utterance is input.

[0211] In step S52, the machine learning model 215 User utterance: "Yesterday's game was awesome." When the user inputs a command, the system outputs the system utterance as the output for that command.

[0212] In step S53, the machine learning model-based dialogue execution module 205 acquires the output from the machine learning model 215. The acquired data is, for example, the following data. Data obtained: "I know, I know, I was moved."

[0213] Next, in step S54, the machine learning model-based dialogue execution module 205 outputs the data acquired from the machine learning model 215 as a system utterance to the execution process determination unit 210 shown in FIG. For example, the following system utterance is output to the execution process determination unit 210: System utterance: "I understand, I understand, I was moved."

[0214] When outputting this system utterance, the machine learning model-based dialogue execution module 205 may be configured to generate a confidence value, which is an index value indicating the confidence of the output system utterance, for example, a confidence value of 0.0 to 1.0, and output the confidence value to the execution process determination unit 210 together with the system utterance. For example, if the system succeeds in generating a system utterance, it outputs a confidence level of 1.0, and if the system fails to generate a system utterance, it outputs a confidence level of 0.0. As described above, each dialogue execution module (dialogue engine) can be configured to output only the system utterance and not output the confidence value.

[0215] Next, the processing sequence executed by the machine learning model-based dialogue execution module 205 will be described with reference to the flowchart shown in FIG. The process of each step in the flow shown in FIG. 21 will be explained in order.

[0216] (Step S251) First, in step S251, it is determined whether or not a user utterance has been input from the situation analysis unit 162, and if it is determined that an utterance has been input, the process proceeds to step S252.

[0217] (Step S252) Next, in step S252, the machine learning model-based dialogue execution module 205 inputs the user utterance input in step S251 to the machine learning model, obtains the output of the machine learning model, and outputs this output to the execution process determination unit as a system utterance.

[0218] In addition to the output of the system utterance, a value of confidence, which is an index value indicating the degree of confidence of the acquired system utterance, may also be output to the execution process determination unit 210. In this case, since the system utterance was successfully generated (acquired), the confidence level is output as 1.0.

[0219] In this way, in steps S111 to S115 of the flow shown in FIG. 8, the following five processes are executed in parallel. (S111) Generation of system utterances by the scenario-based dialogue execution module (+utterance confidence) (executes processing by referring to the scenario DB) (S112) Generation of system utterances by episode knowledge-based dialogue execution module (+utterance confidence) (executes processing with reference to episode knowledge DB) (S113) Generation of system utterances by the RDF knowledge-based dialogue execution module (+utterance confidence) (performs processing by referencing the RDF knowledge DB) (S114) Generation of system utterances (+utterance confidence) by the RDF knowledge-based dialogue execution module with situation verbalization processing (performs processing by referring to the RDF knowledge DB) (S115) Generation of system utterances (+utterance confidence) by machine learning model-based dialogue execution module (performs processing with reference to the machine learning model)

[0220] As described above, these five processes may be executed within the data processing unit 160 of the robot control unit 150 shown in FIG. 4, or may be executed as distributed processes using an external device such as an external server connected via the communication unit 170. For example, five external servers may each execute the five processes of steps S111 to S115, and the process results may be received by a process decision unit (decision making unit) 163 in a data processing unit 160 of the robot control unit 150 shown in FIG.

[0221] The processing results of steps S111 to S115 of the flow shown in FIG. 8, ie, the system utterances generated by the five dialogue execution modules (dialogue engines) 201 to 205 shown in FIG. 7, are input to the execution process determination unit 210 shown in FIG.

[0222] [5. Details of the processing executed by the execution processing decision unit] Next, the process executed by the execution process determination unit 210 will be described in detail.

[0223] As described above with reference to FIG. 7, the execution process determination unit 210 inputs system utterances generated by the five dialogue execution modules (dialogue engines) 201 to 205, and selects one system utterance to be output from the input system utterances. The selected system utterance is output to the dialogue processor 164, converted into text, and output via the voice output unit (speaker) 131.

[0224] The process executed by the process execution determination unit 210 will be described with reference to FIG. As shown in FIG. 22, the execution process determining unit 210 receives the processing results from each of the following five dialogue execution modules. (1) Scenario-based dialogue execution module 201 (2) Episode knowledge-based dialogue execution module 202 (3) RDF (Resource Description Framework) knowledge-based dialogue execution module 203 (4) Situation Verbalization & RDF Knowledge-Based Dialogue Execution Module 204 (5) Machine learning model-based dialogue execution module 205

[0225] These five dialogue execution modules (dialogue engines) 201 to 205 execute parallel processing and generate system responses using different algorithms, respectively. The system utterances generated by these five modules are input to the execution process determination unit 210.

[0226] The five dialogue execution modules (dialogue engines) 201 to 205 input the system utterance generated by each module and its confidence level (0.0 to 1.0) to the execution process determination unit 210.

[0227] The execution process determination unit 210 selects one system utterance having the highest confidence value from the multiple system utterances input from the five dialogue execution modules (dialogue engines) 201-205, and determines the system utterance to be output from the output unit 130 of the data input / output unit 110. In other words, it determines the system utterance to be output by the interactive robot 10.

[0228] When the confidence levels set corresponding to the system utterances input from the multiple dialogue execution modules (dialogue engines) 201 to 205 are equal, the execution process determination unit 210 determines the system utterance to be output by the interactive robot according to a preset priority for each dialogue execution module (dialogue engine).

[0229] An example of preset priority for each dialogue execution module (dialogue engine) will be described with reference to FIG. FIG. 23 is a diagram showing an example of preset priorities for each dialogue execution module (dialogue engine). A priority of 1 is the highest priority and 5 is the lowest priority.

[0230] In the example shown in Figure 23, Priority 1 = Scenario-based dialogue execution module 201 Priority 2 = Episode knowledge-based dialogue execution module 202 Priority 3 = RDF (Resource Description Framework) knowledge-based dialogue execution module 203 Priority 4 = Situation Verbalization & RDF Knowledge-Based Dialogue Execution Module 204 Priority 5 = Machine Learning Model-Based Dialogue Execution Module 205 This is the priority setting for the dialogue execution module.

[0231] The execution process determining unit 210 first selects, based on the confidence levels input from a plurality of dialogue execution modules (dialogue engines), a process for selecting a system utterance having the highest confidence level as a system utterance to be output. However, when there are multiple system utterances with the highest confidence level, the system utterance to be output by the interactive robot is determined according to the priority of each dialogue execution module (dialogue engine) that is preset as shown in FIG.

[0232] Next, the sequence of the processing executed by the execution processing decision unit 210 will be described with reference to the flowchart shown in FIG. The processing of each step will be explained in turn.

[0233] (Step S301) First, in step S301, the execution process determination unit 210 selects five dialogue execution modules (dialogue engines), namely: Scenario-based dialogue execution module 201 Episode knowledge-based dialogue execution module 202 RDF (Resource Description Framework) knowledge-based dialogue execution module 203 Situation Verbalization & RDF Knowledge-Based Dialogue Execution Module 204 Machine learning model-based dialogue execution module 205 It is determined whether or not there is an input from these five dialogue execution modules (dialogue engines) 201 to 205.

[0234] That is, it is determined whether or not data on the system utterance generated according to the algorithm executed in each module and its confidence level (0.0 to 1.0) have been input. If there is an input, the process proceeds to step S302.

[0235] (Step S302) Next, in step S302, the execution process determination unit 210 determines whether or not there is data with a confidence level of 1.0 among the input data from the five dialogue execution modules (dialogue engines) 201 to 205. If there is, the process proceeds to step S303. If not, the process proceeds to step S311.

[0236] (Step S303) If it is determined in step S302 that there is data with a confidence level of 1.0 among the input data from the five dialogue execution modules (dialogue engines) 201-205, then in step S303, the execution process determination unit 210 determines whether there is a plurality of pieces of data with a confidence level of 1.0 among the input data from the five dialogue execution modules (dialogue engines) 201-205. If there are multiple, proceed to step S304. If there is only one, not multiple, the process proceeds to step S305.

[0237] (Step S304) In step S303, if there is a plurality of pieces of data with a confidence level of 1.0 among the input data from the five dialogue execution modules (dialogue engines) 201 to 205, the process of step S304 is executed.

[0238] In step S304, the execution process determination unit 210 selects, from among the multiple system utterances with confidence level=1.0, the system utterance output by the module with the highest priority according to a predefined module-based priority, as the system utterance to be finally output by the interactive robot. The execution process determining unit 210 outputs the selected system utterance to the dialogue processing unit 164 .

[0239] (Step S305) On the other hand, in step S303, if there is only one piece of data with a confidence level of 1.0 among the input data from the five dialogue execution modules (dialogue engines) 201 to 205, the process goes to step S305.

[0240] In step S305, the execution process determination unit 210 selects one system utterance with a confidence level of 1.0 as the system utterance to be finally output by the interactive robot. The execution process determining unit 210 outputs the selected system utterance to the dialogue processing unit 164 .

[0241] (Step S311) If it is determined in the judgment process of step S302 that there is no data with a confidence level of 1.0 in the input data from the five dialogue execution modules (dialogue engines) 201 to 205, then in step S311, the execution process determination unit 210 determines whether there is data with a confidence level of >0.0 in the input data from the five dialogue execution modules (dialogue engines) 201 to 205. If there is, proceed to step S312. If not, the process ends. In this case, no system utterance is output.

[0242] (Step S312) If it is determined in step S311 that there is data with a confidence level > 0.0 in the input data from the five dialogue execution modules (dialogue engines) 201 to 205, then in step S312, the execution process determination unit 210 determines whether there is a plurality of data with the highest confidence level of > 0.0 in the input data from the five dialogue execution modules (dialogue engines) 201 to 205. If there are multiple, the process proceeds to step S313. If there is only one, not multiple, the process proceeds to step S314.

[0243] (Step S313) In step S312, if there is a plurality of pieces of data with the highest confidence level (confidence level>0.0) among the input data from the five dialogue execution modules (dialogue engines) 201 to 205, the process of step S313 is executed.

[0244] In step S313, the execution process decision unit 210 selects, from among the multiple system utterances with the highest confidence level among the data with confidence level > 0.0, the system utterance output by the module with the highest priority level according to a predefined module-based priority level, as the system utterance to be finally output by the interactive robot. The execution process determining unit 210 outputs the selected system utterance to the dialogue processing unit 164 .

[0245] (Step S314) On the other hand, in step S312, if there is only one piece of data with the highest confidence level of confidence > 0.0 among the input data from the five dialogue execution modules (dialogue engines) 201 to 205, the process of step S314 is executed.

[0246] In step S314, the execution process determination unit 210 selects the system utterance with the highest confidence level (confidence level>1.0) as the system utterance to be finally output by the interactive robot. The execution process determining unit 210 outputs the selected system utterance to the dialogue processing unit 164 .

[0247] In this way, the execution process determination unit 210 selects one system utterance with the highest confidence value from the multiple system utterances input from the five dialogue execution modules (dialogue engines) 201 to 205, and sets this as the system utterance to be output by the dialogue robot. When the confidence levels input from multiple dialogue execution modules (dialogue engines) are equal, the system utterance to be output by the dialogue robot is determined according to the preset priority of each dialogue execution module (dialogue engine).

[0248] In this way, the information processing device of the present disclosure generates a plurality of system utterances by operating in parallel a plurality of dialogue execution modules that generate system utterances according to different algorithms, and selects and outputs the most appropriate system utterance from among them. By performing such processing, it becomes possible to output optimal system utterances according to various situations, enabling dialogue with the user to be conducted more naturally and smoothly.

[0249] [6. Example of system speech output by the information processing device of the present disclosure] Next, a specific example of a system utterance performed by the processing of the information processing device of the present disclosure described above will be described.

[0250] 25 and 26 show an example of a dialogue sequence between the user 1 and the interactive robot 10. In FIG. The interactive robot 10 executes the system utterance by performing the system utterance generation process according to the process disclosed hereinabove. That is, a plurality of dialogue execution modules, which generate system utterances according to different algorithms, are operated in parallel to generate a plurality of system utterances, and the most appropriate system utterance is selected and output from among them.

[0251] 26 and 27, the user 1 and the interactive robot 10 alternately utter, from a system utterance 01, to a user utterance 02, a system utterance 03, . . . , a user utterance 18. In FIG. That is, the following speech sequence:

[0252] (The user returns) System Speech = "Welcome back. Where have you been?" User utterance = "I went to work at the office." System utterance = "That's right. I go there every day." User utterance = "That's what a job is." System utterance = "Please do your best for me too." User speaks = "Sure, sure. I'll grab a beer." System utterance = "When you think of beer, you think of Belgium." User utterance = "I want to go to Belgium and eat something." System utterance = "By the way, what's your favorite food?" User utterance = "Yeah, but maybe yakitori for now." System utterance = "You like yakitori? Chicken is good." User utterance = "Meat is good, meat is good." System utterance = "Tanaka said he likes meat too." User utterance = "Uh, yeah." System utterance = "You're going to a barbecue with Tanaka this weekend, right?" User speaks = "Yeah. It's cold already." System utterance = "Kotatsu and oranges are good." (Beer and food are on the table) System speech = "Oh. What a feast. Enjoy."

[0253] In these utterance sequences, each system utterance output by the interactive robot 10 is one system utterance selected each time from the system utterances generated by the following five dialogue execution modules. (1) Scenario-based dialogue execution module 201 (2) Episode knowledge-based dialogue execution module 202 (3) RDF (Resource Description Framework) knowledge-based dialogue execution module 203 (4) Situation Verbalization & RDF Knowledge-Based Dialogue Execution Module 204 (5) Machine learning model-based dialogue execution module 205

[0254] For example, the first system utterance: "Welcome back. Where have you been?" This system utterance is based on the user's situation, i.e. (The user returns) This is a system utterance generated by the situation verbalization & RDF knowledge-based dialogue execution module 204 based on the situation information when the user returned.

[0255] Next system utterance = "Oh yeah, I go there every day." This system utterance is the previous user utterance, i.e. User utterance = "I went to work at the office." This is a system utterance generated by the episode knowledge-based dialogue execution module 202 based on this user utterance.

[0256] Next system utterance = "Please do your best for me too." This system utterance is the previous user utterance, i.e. User utterance = "That's what a job is." This is a system utterance generated by the machine learning model-based dialogue execution module 205 based on this user utterance.

[0257] Next system utterance = "When you think of beer, you think of Belgium." This system utterance is the previous user utterance, i.e. User speaks = "Sure, sure. I'll grab a beer." This is a system utterance generated by the RDF (Resource Description Framework) knowledge-based dialogue execution module 203 based on the user utterance.

[0258] Next system utterance = "By the way, what's your favorite food?" This system utterance is the previous user utterance, i.e. User utterance = "I want to go to Belgium and eat something." This is a system utterance generated by the scenario-based dialogue execution module 201 based on the user utterance.

[0259] The same applies to the following system utterances, where multiple dialogue execution modules that generate system utterances according to different algorithms are operated in parallel to generate multiple system utterances, and the most appropriate system utterance is selected and output.

[0260] In this way, the information processing device of the present disclosure generates a plurality of system utterances by operating in parallel a plurality of dialogue execution modules that generate system utterances according to different algorithms, and selects and outputs the most appropriate system utterance from among them. By performing such processing, it becomes possible to output optimal system utterances according to various situations, enabling dialogue with the user to be conducted more naturally and smoothly.

[0261] [7. Examples of hardware configurations of information processing devices] Next, with reference to FIG. 27, an example of a hardware configuration of an information processing device will be described. The hardware described with reference to FIG. 27 is an example of a hardware configuration common to the information processing device described above with reference to FIG. 4 and an external device such as an external server equipped with a dialogue execution module (dialogue engine).

[0262] A CPU (Central Processing Unit) 501 functions as a control unit or data processing unit that executes various processes according to programs stored in a ROM (Read Only Memory) 502 or a storage unit 508. For example, it executes processes according to the sequences described in the above-mentioned embodiments. A RAM (Random Access Memory) 503 stores programs and data executed by the CPU 501. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504.

[0263] CPU 501 is connected to input / output interface 505 via bus 504, and input / output interface 505 is connected to input unit 506 including various switches, a keyboard, a mouse, a microphone, a sensor, etc., and output unit 507 including a display, a speaker, etc. CPU 501 executes various processes in response to commands input from input unit 506, and outputs the process results to output unit 507, for example.

[0264] A storage unit 508 connected to the input / output interface 505 is formed of, for example, a hard disk, and stores programs executed by the CPU 501 and various data. A communication unit 509 functions as a transmission / reception unit for Wi-Fi communication, Bluetooth (registered trademark) (BT) communication, and other data communication via a network such as the Internet or a local area network, and communicates with external devices.

[0265] A drive 510 connected to the input / output interface 505 drives removable media 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory such as a memory card, and executes recording or reading of data.

[0266] [8. Summary of the Configuration of the Present Disclosure] The embodiments of the present disclosure have been described in detail above with reference to specific embodiments. However, it is obvious that a person skilled in the art can modify or substitute the embodiments without departing from the gist of the present disclosure. In other words, the present invention has been disclosed in the form of an example, and should not be interpreted as being limited. In order to judge the gist of the present disclosure, the claims should be taken into consideration.

[0267] The technology disclosed in this specification can have the following configurations. (1) a data processing unit that generates and outputs system utterances; The data processing unit includes: An information processing device that selects and outputs one system utterance from multiple system utterances generated individually by multiple dialogue execution modules.

[0268] (2) each of the plurality of dialogue execution modules The information processing device according to (1) generates algorithm-specific system utterances according to different system utterance generation algorithms.

[0269] (3) The data processing unit inputting a user utterance, and inputting a speech recognition result of the input user utterance to the plurality of dialogue execution modules; The information processing device according to (1) or (2), wherein the plurality of dialogue execution modules select one system utterance from system utterances generated based on a user utterance.

[0270] (4) The data processing unit inputting situation information, which is observation information, into the plurality of dialogue execution modules; The information processing device according to any one of (1) to (3), wherein the plurality of dialogue execution modules select one system utterance from among system utterances generated based on situation information.

[0271] (5) The data processing unit The information processing device according to any one of (1) to (4), further comprising: a confidence level for a system utterance that is set in correspondence with the system utterance generated by each of the plurality of dialogue execution modules; and a system utterance having a high confidence level value is selected as an output system utterance.

[0272] (6) The data processing unit includes: If there are multiple system utterances with the highest confidence value, The information processing device according to (5), further comprising: selecting, as an output system utterance, a system utterance generated by a dialogue execution module having a high priority according to a predetermined priority corresponding to the dialogue execution module.

[0273] (7) Each of the plurality of dialogue execution modules Generate a generated system utterance and a confidence measure corresponding to the generated system utterance; The data processing unit includes: The information processing device according to any one of (1) to (6), wherein the system utterance having a high confidence value is selected as an output system utterance.

[0274] (8) The plurality of dialogue execution modules include An information processing device described in any one of (1) to (7) including a scenario-based dialogue execution module that generates system utterances by referring to a scenario database in which utterance combination data of user utterances and system utterances according to various dialogue scenarios is registered.

[0275] (9) The plurality of dialogue execution modules include The information processing device according to any one of (1) to (8), further comprising an episode knowledge based dialogue execution module that generates a system utterance by referring to an episode knowledge database in which various episode information is recorded.

[0276] (10) The plurality of dialogue execution modules include: An information processing device described in any one of (1) to (9), which includes an RDF knowledge-based dialogue execution module that generates system utterances by referring to an RDF (Resource Description Framework) knowledge database that records elements contained in various information and the relationships between the elements.

[0277] (11) The plurality of dialogue execution modules include An information processing device as described in any one of (1) to (10), which includes a situation verbalization & RDF knowledge-based dialogue execution module that executes a verbalization process of situation information, and based on the situation verbalization data generated by the verbalization process, searches an RDF (Resource Description Framework) knowledge database that records elements contained in various information and the relationships between the elements, to generate system utterances.

[0278] (12) The plurality of dialogue execution modules include: The information processing device according to any one of (1) to (11), further comprising a machine learning model-based dialogue execution module that generates system utterances using a machine learning model generated by machine learning processing of pair data of an input sentence and a response sentence.

[0279] (13) The data processing unit includes: a state analysis unit that receives external information including voice information from an input unit and generates time-unit state information that is external state analysis information for each time unit; a situation analysis unit that continuously receives the state information and generates external situation information based on the received state information; a process determination unit that receives the situation information generated by the situation analysis unit and determines a process to be executed by the information processing device; The processing determination unit, inputting the situation information into a plurality of dialogue execution modules; acquiring a plurality of system utterances that are individually generated by the plurality of dialogue execution modules based on the situation information; The information processing device according to any one of (1) to (12), wherein one system utterance to be output is selected from a plurality of acquired system utterances.

[0280] (14) An information processing system having a robot control device that controls a dialogue robot and a server that can communicate with the robot control device, The robot control device includes: outputting status information input via an input unit to the server; The server, a plurality of dialogue execution modules for generating system utterances according to different system utterance generation algorithms; Each of the plurality of dialogue execution modules generates an individual system utterance based on the situation information and transmits the individual system utterance to the robot control device; The robot control device includes: An information processing system that selects and outputs one system utterance from the multiple system utterances received from the server.

[0281] (15) The robot control device The information processing system according to (14), further comprising: a system utterance generating unit that generates a system utterance corresponding to the system utterance;

[0282] (16) The robot control device If there are multiple system utterances with the highest confidence value, According to a predetermined priority for each dialogue execution module, a system utterance generated by a dialogue execution module with a high priority is selected as an output system utterance (15).

[0283] (17) An information processing method executed in an information processing device, The information processing device has a data processing unit that generates and outputs a system utterance, The data processing unit: An information processing method for selecting and outputting one system utterance from multiple system utterances generated individually by multiple dialogue execution modules.

[0284] (18) An information processing method executed in an information processing system having a robot control device for controlling a dialogue robot and a server capable of communicating with the robot control device, The robot control device includes: outputting status information input via an input unit to the server; the server has a plurality of dialogue execution modules for generating system utterances according to different system utterance generation algorithms; Each of the plurality of dialogue execution modules generates an individual system utterance based on the situation information and transmits the individual system utterance to the robot control device; The robot control device, An information processing method for selecting and outputting one system utterance from a plurality of system utterances received from the server.

[0285] (19) A program for causing an information processing device to execute information processing, The information processing device has a data processing unit that generates and outputs a system utterance, The program causes the data processing unit to A program that selects and outputs one system utterance from multiple system utterances generated individually by multiple dialogue execution modules.

[0286] In addition, the series of processes described in the specification can be executed by hardware, software, or a combination of both. When executing processes by software, a program recording the processing sequence can be installed and executed in a memory in a computer built into dedicated hardware, or the program can be installed and executed in a general-purpose computer capable of executing various processes. For example, the program can be pre-recorded on a recording medium. In addition to installing the program from the recording medium to the computer, the program can be received via a network such as a LAN (Local Area Network) or the Internet, and installed on a recording medium such as a built-in hard disk.

[0287] In addition, the various processes described in the specification may not only be executed in chronological order as described, but may also be executed in parallel or individually depending on the processing capacity of the device executing the processes or as necessary. Furthermore, in this specification, a system refers to a logical collective configuration of multiple devices, and is not limited to devices in the same housing. [Industrial Applicability]

[0288] As described above, according to the configuration of one embodiment of the present disclosure, a configuration is realized in which an optimal system utterance is selected and output from multiple system utterances generated by multiple dialogue execution modules that generate system utterances according to different algorithms. Specifically, for example, a data processing unit that generates and outputs a system utterance selects and outputs one system utterance from multiple system utterances generated individually by multiple dialogue execution modules. Each of the multiple dialogue execution modules generates an algorithm-specific system utterance according to a different algorithm. The data processing unit selects one system utterance to be output according to a confidence level set corresponding to the system utterance generated by each of the multiple dialogue execution modules or a predetermined priority corresponding to the dialogue execution module. This configuration realizes a configuration in which an optimal system utterance is selected and output from a plurality of system utterances generated by a plurality of dialogue execution modules which generate system utterances according to different algorithms. [Explanation of symbols]

[0289] 10. Conversational Robot 21 Server 22 Smartphone 23 PC 100 Information processing device 110 Data input / output section 120 Input section 121 Audio input section 122 Image input unit 123 Sensors 130 Output section 131 Audio output section 132 Drive control unit 150 Robot control unit 160 Data Processing Unit 161 Status Analysis Unit 162 Situation Analysis Department 163 Processing decision-making unit (decision-making unit) 164 Dialogue Processing Unit 165 Action Processing Unit 170 Communications Department 201 Scenario-based Dialogue Execution Module 202 Episode Knowledge-Based Dialogue Execution Module 203 RDF Knowledge-Based Dialogue Execution Module 204 Situation Verbalization & RDF Knowledge-Based Dialogue Execution Module 205 Machine Learning Model-Based Dialogue Execution Module 210 Execution process decision unit 211 Scenario Database 212 Episode Knowledge Database 213 RDF Knowledge Database 215 Machine Learning Models 501 CPU 502 ROM 503 RAM 504 Bus 505 Input / Output Interface 506 Input section 507 Output section 508 Storage section 509 Communications Department 510 Drive 511 Removable Media

Claims

1. A data processing unit that generates and outputs a system utterance, The data processing unit includes: A configuration in which a process of selecting and outputting one system utterance from a plurality of system utterances individually generated by a plurality of dialogue execution modules is executed, The plurality of dialogue execution modules include (a) a scenario-based dialogue execution module that generates system utterances by referring to a scenario database in which utterance pair data of user utterances and system utterances corresponding to various dialogue scenarios is registered; (b) an episode knowledge-based dialogue execution module that generates system utterances by referring to an episode knowledge database that records various episode information; (c) an RDF knowledge-based dialogue execution module that generates system utterances by referring to an RDF (Resource Description Framework) knowledge database that records elements contained in various information and relationships between the elements; At least the above three types of dialogue execution modules (a) to (c), Each of the three types of dialogue execution modules (a) to (c) is If the generation of the system utterance is successful, output the generated system utterance and a confidence level of the generated system utterance that is greater than 0 and indicates the confidence level of the generated system utterance; If the generation of the system utterance fails, the system utterance is not output, and the confidence level is output as 0. The data processing unit includes: An information processing device that selects and outputs one system utterance with the highest confidence value from a plurality of system utterances including a plurality of system utterances generated individually by the three types of dialogue execution modules (a) to (c) above.

2. Each of the plurality of dialogue execution modules The information processing apparatus according to claim 1 , wherein algorithm-specific system utterances are generated according to different system utterance generation algorithms.

3. The data processing unit includes: inputting a user utterance, and inputting a speech recognition result of the input user utterance to the plurality of dialogue execution modules; The information processing apparatus according to claim 1 , wherein the plurality of dialogue execution modules select one system utterance from system utterances generated based on user utterances.

4. The data processing unit includes: inputting situation information, which is observation information, into the plurality of dialogue execution modules; The information processing apparatus according to claim 1 , wherein the plurality of dialogue execution modules select one system utterance from among system utterances generated based on situation information.

5. The data processing unit includes: If there are multiple system utterances with the highest confidence value, 2. The information processing apparatus according to claim 1, wherein a system utterance generated by a dialogue execution module having a high priority is selected as the output system utterance according to a predetermined priority corresponding to the dialogue execution module.

6. The plurality of dialogue execution modules include:

2. The information processing device according to claim 1, further comprising a situation verbalization & RDF knowledge-based dialogue execution module that executes a verbalization process of situation information, and based on the situation verbalization data generated by the verbalization process, searches an RDF (Resource Description Framework) knowledge database that records elements contained in various information and relationships between the elements, to generate system utterances.

7. The plurality of dialogue execution modules include: The information processing device according to claim 1 , further comprising a machine learning model-based dialogue execution module that generates a system utterance using a machine learning model generated by machine learning processing of paired data of an input sentence and a response sentence.

8. The data processing unit includes: a state analysis unit that receives external information including voice information from an input unit and generates time-unit state information that is external state analysis information for each time unit; a situation analysis unit that continuously receives the state information and generates external situation information based on the received state information; a process determination unit that receives the situation information generated by the situation analysis unit and determines a process to be executed by the information processing device; The processing determination unit, inputting the situation information into a plurality of dialogue execution modules; acquiring a plurality of system utterances that are individually generated by the plurality of dialogue execution modules based on the situation information; The information processing apparatus according to claim 1 , further comprising: a processor for selecting one system utterance to be output from the plurality of acquired system utterances.

9. An information processing system having a robot control device for controlling a dialogue robot and a server capable of communicating with the robot control device, The robot control device includes: outputting status information input via an input unit to the server; The server, a plurality of dialogue execution modules for generating system utterances according to different system utterance generation algorithms; Each of the plurality of dialogue execution modules generates an individual system utterance based on the situation information and transmits the individual system utterance to the robot control device; The robot control device includes: execute a process of selecting and outputting one system utterance from a plurality of system utterances received from the server; The plurality of dialogue execution modules included in the server include (a) a scenario-based dialogue execution module that generates system utterances by referring to a scenario database in which utterance pair data of user utterances and system utterances corresponding to various dialogue scenarios is registered; (b) an episode knowledge-based dialogue execution module that generates system utterances by referring to an episode knowledge database that records various episode information; (c) an RDF knowledge-based dialogue execution module that generates system utterances by referring to an RDF (Resource Description Framework) knowledge database that records elements contained in various information and relationships between the elements; At least the above three types of dialogue execution modules (a) to (c), Each of the three types of dialogue execution modules (a) to (c) is If the generation of the system utterance is successful, output the generated system utterance and a confidence level of the generated system utterance that is greater than 0 and indicates the confidence level of the generated system utterance; If the generation of the system utterance fails, the system utterance is not output, and the confidence level is output as 0. The robot control device includes: An information processing system that selects and outputs one system utterance with the highest confidence value from a plurality of system utterances including a plurality of system utterances generated individually by the three types of dialogue execution modules (a) to (c) received from the server.

10. The robot control device includes: If there are multiple system utterances with the highest confidence value, 10. The information processing system according to claim 9, wherein a system utterance generated by a dialogue execution module with a high priority is selected as the output system utterance according to a predefined priority for each dialogue execution module.

11. An information processing method executed in an information processing device, The information processing device has a data processing unit that generates and outputs a system utterance, The data processing unit: executing a process of selecting and outputting one system utterance from a plurality of system utterances individually generated by a plurality of dialogue execution modules; The plurality of dialogue execution modules include (a) a scenario-based dialogue execution module that generates system utterances by referring to a scenario database in which utterance pair data of user utterances and system utterances corresponding to various dialogue scenarios is registered; (b) an episode knowledge-based dialogue execution module that generates system utterances by referring to an episode knowledge database that records various episode information; (c) an RDF knowledge-based dialogue execution module that generates system utterances by referring to an RDF (Resource Description Framework) knowledge database that records elements contained in various information and relationships between the elements; At least the above three types of dialogue execution modules (a) to (c), Each of the three types of dialogue execution modules (a) to (c) is If the generation of the system utterance is successful, output the generated system utterance and a confidence level of the generated system utterance that is greater than 0 and indicates the confidence level of the generated system utterance; If the generation of the system utterance fails, a process of outputting 0 as the confidence level is executed without outputting the system utterance; The data processing unit includes: An information processing method for selecting and outputting one system utterance with the highest confidence value from a plurality of system utterances including a plurality of system utterances generated individually by the three types of dialogue execution modules (a) to (c) above.

12. An information processing method executed in an information processing system having a robot control device for controlling a dialogue robot and a server capable of communicating with the robot control device, The robot control device includes: outputting status information input via an input unit to the server; the server has a plurality of dialogue execution modules for generating system utterances according to different system utterance generation algorithms; Each of the plurality of dialogue execution modules generates an individual system utterance based on the situation information and transmits the individual system utterance to the robot control device; The robot control device, execute a process of selecting and outputting one system utterance from a plurality of system utterances received from the server; The plurality of dialogue execution modules included in the server include (a) a scenario-based dialogue execution module that generates system utterances by referring to a scenario database in which utterance pair data of user utterances and system utterances corresponding to various dialogue scenarios is registered; (b) an episode knowledge-based dialogue execution module that generates system utterances by referring to an episode knowledge database that records various episode information; (c) an RDF knowledge-based dialogue execution module that generates system utterances by referring to an RDF (Resource Description Framework) knowledge database that records elements contained in various information and relationships between the elements; At least the above three types of dialogue execution modules (a) to (c), Each of the three types of dialogue execution modules (a) to (c) is If the generation of the system utterance is successful, output the generated system utterance and a confidence level of the generated system utterance that is greater than 0 and indicates the confidence level of the generated system utterance; If the generation of the system utterance fails, a process of outputting 0 as the confidence level is executed without outputting the system utterance; The robot control device includes: an information processing method for selecting and outputting one system utterance having a maximum confidence value from a plurality of system utterances including a plurality of system utterances generated individually by the three types of dialogue execution modules (a) to (c) received from the server.

13. A program for causing an information processing device to execute information processing, The information processing device has a data processing unit that generates and outputs a system utterance, The program causes the data processing unit to selecting and outputting one system utterance from a plurality of system utterances generated individually by a plurality of dialogue execution modules; The plurality of dialogue execution modules include (a) a scenario-based dialogue execution module that generates system utterances by referring to a scenario database in which utterance pair data of user utterances and system utterances corresponding to various dialogue scenarios is registered; (b) an episode knowledge-based dialogue execution module that generates system utterances by referring to an episode knowledge database that records various episode information; (c) an RDF knowledge-based dialogue execution module that generates system utterances by referring to an RDF (Resource Description Framework) knowledge database that records elements contained in various information and relationships between the elements; At least the above three types of dialogue execution modules (a) to (c), Each of the three types of dialogue execution modules (a) to (c) is If the generation of the system utterance is successful, output the generated system utterance and a confidence level of the generated system utterance that is greater than 0 and indicates the confidence level of the generated system utterance; If the generation of the system utterance fails, a process of outputting 0 as the confidence level is executed without outputting the system utterance; The program causes the data processing unit to A program for selecting and outputting one system utterance with the highest confidence value from a plurality of system utterances, including a plurality of system utterances generated individually by the three types of dialogue execution modules (a) to (c) above.

Images