Recognition control device, recognition control method, and recognition control program

Abstract

A recognition control device (10) according to an embodiment executes a plurality of processes that generate responses to input information at mutually different speeds. The recognition control device (10) outputs the responses generated in the respective plurality of processes. The plurality of processes include a first lower-layer recognition process that generates a reflexive response to the information; a second lower-layer recognition process that is slower than the first lower-layer recognition process and generates a response based on a context indicated in the information; and a middle-layer recognition process that is slower than the second lower-layer recognition process and generates a response based on the context and a set scenario.

Description

Cognitive Control Apparatus, Cognitive Control Method, and Cognitive Control Program

[0001] The present invention relates to a cognitive control apparatus, a cognitive control method, and a cognitive control program.

[0002] Conventionally, systems using digital humans (DH: Digital Human) are known. A digital human is a method of causing a computer to produce human-like responses.

[0003] For example, Non-Patent Document 1 proposes a hybrid architecture that integrates planning and reactive control for the movement and manipulation of physical robots.

[0004] Furthermore, for example, Non-Patent Document 2 proposes robot control based on reinforcement learning using a spiking neural network (SNN: Spiking Neural Network).

[0005] RONALD C. ARKIN and DOUGLAS C. MACKENZIE, "Planning to Behave:A Hybrid Deliberative / Reactive Robot Control Architecture for Mobile Manipulation"Vahid Azimirad, S. Yaser Khodkam, Amir Bolouri,"A new hybrid learning control system for robots based on spiking neural networks"

[0006] However, conventional techniques have a problem that response generation in DH is not optimized. For example, optimizing the dialog response speed of DH and responses based on recognition of external information enables the generation of real-time human-like responses.

[0007] For example, the techniques described in Non-Patent Documents 1 and 2 mainly focus on physical tasks of robots or control of physical robots, and do not achieve optimization of the dialog response speed of DH.

[0008] Therefore, the object of the present invention is to optimize response generation in DH.

[0009] To solve the problem, the cognitive control device of the present invention is characterized by having a cognitive processing unit that executes a plurality of processes that generate responses to input information at different speeds, and an output processing unit that outputs the responses generated in each of the plurality of processes.

[0010] According to the present invention, response generation in DH can be optimized.

[0011] Figure 1 is a diagram showing an example configuration of a cognitive control device according to the first embodiment. Figure 2 is a diagram illustrating the processing of the cognitive control device. Figure 3 is a diagram illustrating the first lower-level cognitive processing. Figure 4 is a diagram illustrating the second lower-level cognitive processing. Figure 5 is a diagram illustrating the middle-level cognitive processing. Figure 6 is a diagram showing an example of dialogue. Figure 7 is a flowchart showing the processing flow of the cognitive control device. Figure 8 is a diagram showing an example configuration of a computer that executes the cognitive control program.

[0012] The embodiments for carrying out the present invention will be described below with reference to the drawings. The present invention is not limited to these embodiments.

[0013] The embodiment optimizes the DH's dialogue response speed and responses based on the recognition of external information. As a result, response generation in the DH is optimized.

[0014] [Configuration of the First Embodiment] The configuration of the cognitive control device will be explained using Figure 1. Figure 1 is a diagram showing an example of the configuration of the cognitive control device of the first embodiment.

[0015] As shown in Figure 1, the cognitive control device 10 has an input unit 11, an output unit 12, a storage unit 13, and a control unit 14.

[0016] The input unit 11 is an interface connected to input devices such as a mouse and a keyboard. The output unit 12 is an interface connected to output devices such as a speaker and a display.

[0017] The storage unit 13 stores data, programs, etc., that are referenced when the control unit 14 performs various processes. The storage unit 13 is implemented by semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as hard disks and optical discs. The storage unit 13 stores the user database 131 and model information 132.

[0018] The user database 131 stores information about the user. For example, the user database 131 stores the user's gender, age, personality (e.g., Big 5), etc.

[0019] Model information 132 consists of the model's parameters. For example, if the model is a neural network, then model information 132 consists of parameters such as weights and biases.

[0020] The control unit 14 is responsible for controlling the entire cognitive control device 10. The functions of the control unit 14 are realized, for example, by the CPU (Central Processing Unit) executing a program stored in the memory unit 13. The control unit 14 includes a management unit 141, a task execution unit 142, a reaction prediction processing unit 143, an initial reaction processing unit 144, an execution processing unit 145, and an output processing unit 146.

[0021] The management unit 141 performs upper-level cognitive processing. The task execution unit 142 performs middle-level cognitive processing. The reaction prediction processing unit 143 performs second lower-level cognitive processing. The initial reaction processing unit 144 performs first lower-level cognitive processing. The execution processing unit 145 performs execution processing. The output processing unit 146 performs output processing.

[0022] The task execution unit 142, the response prediction processing unit 143, and the initial response processing unit 144 are examples of cognitive processing units. Specifically, the cognitive processing unit performs intermediate cognitive processing, a second lower-level cognitive processing, and a first lower-level cognitive processing.

[0023] Figure 2 will be used to explain the details of each process. Figure 2 is a diagram illustrating the processing of the cognitive control device. As shown in Figure 2, the cognitive control device 10 executes multiple cognitive processes with different processing speeds by combining them.

[0024] The cognitive control device 10 constructs a model based on the model information 132, corresponding to the upper-level cognitive processing (step S1), the middle-level cognitive processing (step S2), the second lower-level cognitive processing (step S3), and the first lower-level cognitive processing (step S4). The model may also be referred to as AI. Each cognitive processing may be a process that inputs information into each AI, which is a language model, and outputs a response.

[0025] The cognitive control device 10 performs processing by combining multiple AIs. Here, the upper-level cognitive processing is the process of formulating and managing the overall strategy. The upper-level cognitive processing monitors and controls the progress of the overall scenario and tasks. Specifically, the upper-level cognitive processing generates scenarios based on the input goals and scenario settings, monitors the state of dialogue between the DH and the user, gives instructions to the subsequent middle-level cognitive processing, resets the scenario, and enables each model to grow autonomously through feedback. For example, the goal and scenario setting input is to make the user understand the information about the exhibits. In response, the upper-level cognitive processing generates a scenario that includes the order of explanations to make the user understand the information about the exhibits, whether or not to engage in small talk, and the timing of question and answer sessions.

[0026] The intermediate cognitive processing, the second lower cognitive processing, and the first lower cognitive processing are processes that generate responses to external information. The generated responses are output as text, images, audio, or robot actions, etc.

[0027] For example, intermediate cognitive processing is performed by a prompt-based heavy AI. Second lower cognitive processing is performed by a normal-speed lightweight AI (hereinafter referred to as the second lightweight AI). First lower cognitive processing is performed by the fastest lightweight AI (hereinafter referred to as the first lightweight AI).

[0028] Note that "fastest" here means the fastest among the AIs performing the response generation process in the first embodiment. Also, the lighter the AI, the faster the processing tends to be. The second lightweight AI is lighter than the heavier AI. The first lightweight AI is lighter than the second lightweight AI. For example, if the AI ​​is a neural network, the lighter the AI, the smaller the model size (e.g., number of nodes and layers). On the other hand, the heavier the AI, the more informative the response that takes context into account.

[0029] For example, when the cognitive control device 10 receives external information (e.g., sensing information), it executes a first lower-level cognitive process, a second lower-level cognitive process, and an intermediate-level cognitive process. The first lower-level cognitive process, the second lower-level cognitive process, and the intermediate-level cognitive process generate responses at different speeds. The cognitive control device 10 then performs output according to the results of each cognitive process, according to priority and interrupt rules (execution process (step S5) and output process (step S6)).

[0030] The cognitive control device 10 receives output instructions from each cognitive layer during execution and instructs output processing. At that time, the cognitive control device 10 can perform output selection and interrupt control. For example, the cognitive control device 10 can interrupt the response of one cognitive layer with the response of another cognitive layer. In output selection, the cognitive control device 10 selects the output mode from speech, motion (robot movement), and screen display and issues an instruction.

[0031] In output processing, the cognitive control device 10 outputs according to the mode selected in the execution process. For example, the cognitive control device 10 outputs a response through voice output using a speech synthesis module, robot control using a motion / facial expression generation module, or screen display using a display operation module.

[0032] The robot can output voice and screen content in response to the control of the cognitive control device 10. The robot can also move its face, hands, etc., in response to the control of the cognitive control device 10. The cognitive control device 10 can interact with the user through the robot. The interaction takes place between the DH (which includes the cognitive control device 10 and the robot) and the user. The cognitive control device 10 also receives user video, audio, etc., via sensors.

[0033] Figure 3 will be used to explain the details of the first underlying cognitive processing. Figure 3 is a diagram illustrating the first underlying cognitive processing. In the first underlying cognitive processing, the cognitive control device 10 immediately generates a spinal reflex reaction (response) using the first lightweight AI (fastest). The first underlying cognitive processing is also called the initial reaction.

[0034] As shown in Figure 3, in the initial response, the cognitive control device 10 generates a reaction based on external information. The external information includes video information, dialogue situation recognition information, voice rule information, slide information, and voice information. Video information is information obtained by analyzing video and includes the user's gender and age, gestures, emotions, what they are looking at, and person ID (information that identifies the user). Dialogue situation recognition information is information based on the history of the dialogue and includes the user's willingness to speak, motivation, level of understanding, level of trust, type of interjection, and speaker change estimation (whether or not the speaker has changed). Voice rule information is information on standard phrases spoken by the user. Slide information includes the scenario state (progress of the scenario) based on the slides presented to the user. Voice information is information obtained by analyzing voice and includes the user's gender and age, speech, and emotions.

[0035] The lightweight AI (fastest) provides the optimal reaction based on the user's emotions, state, and situation, taking into account the input external information. The reaction may be a reflexive nod of agreement, or it may be an agreement that corresponds to the recognized emotions of the user or the positivity of the user's statements. The reaction is not limited to nodding, but may also include blinking, changing the direction of the face, expressing emotions (e.g., a change in display), or continuing the current state.

[0036] By outputting based on the reactions generated by the first lower-level cognitive processing, the cognitive control device 10 maintains smooth dialogue with the user and appropriately adjusts its reactions based on the user's emotions and statements, providing natural and consistent responses.

[0037] Figure 4 will be used to explain the details of the second lower-level cognitive processing. Figure 4 is a diagram illustrating the second lower-level cognitive processing. In the second lower-level cognitive processing, the cognitive control device 10 uses a second lightweight AI (normal speed) to predict a response (Prompt) and generate a reaction based on the relationship between the user and the DH, the emotional state of the DH, and the dialogue history between the DH and the user.

[0038] Furthermore, in this scenario, the DH (Decorative Human) engages in dialogue to help the user understand the information about the exhibits. The cognitive control device 10 integrates real-time information such as the user's situation, the dialogue situation, and the DH's state to determine the appropriate turn to speak (whether it is appropriate for the user or the DH to speak), understanding of the exhibits, and understanding of the environment, and provides the optimal response. For example, the cognitive control device 10 can understand the exhibits based on an exhibit summary that summarizes the information about the exhibits in advance.

[0039] Furthermore, the cognitive control device 10 considers social factors such as the user's impression of the DH and the user's personality (e.g., Big 5) in order to understand the relationship between the user and the DH, as well as the DH's own personality, and generates a consistent response based on the dialogue history. The cognitive control device 10 also considers information input to the AI ​​as a prompt.

[0040] Furthermore, in the second lower-level cognitive processing, external information is reflected in the generation of responses. As a result, the DH can accurately understand the user's intentions and engage in natural dialogue. This allows the cognitive control device 10 to smoothly conduct dialogue with the user and provide reactions at the appropriate time.

[0041] Details of the middle-level cognitive processing will be described with reference to FIG. 5. FIG. 5 is a diagram explaining the middle-level cognitive processing. In the middle-level cognitive processing, the cognitive control device 10 makes a judgment based on task progress and scenario prediction by weighted AI, selects an appropriate task, and outputs a signal for causing the task to be executed. In the middle-level cognitive processing, the cognitive control device 10 further receives, as an input, reactions generated in a first lower-level cognitive processing and a second lower-level cognitive processing.

[0042] The cognitive control device 10 understands a situation based on the relationship between a user and a DH and the DH's own personality, and generates a reaction based on the understanding. The cognitive control device 10 generates a scenario state, a user reaction prediction, or a response to casual conversation or a request for repetition.

[0043] Furthermore, in order to understand the relationship between the user and the DH and the DH's own personality, the cognitive control device 10 generates a response for deepening the relationship with the user in consideration of social factors such as the user's impression of the DH and the user's personality (e.g., BIG5).

[0044] Furthermore, the cognitive control device 10 generates a response matching the current context through past dialogue history and exhibition summaries. At this time, the cognitive control device 10 receives a lower-level reaction prediction as an input. This enables real-time response. Furthermore, by integrating the response with the understanding of the environment, smooth dialogue with the user is maintained.

[0045] In the middle-level cognitive processing, the cognitive control device 10 analyzes information in real time based on various elements (external information) such as the user's age, gestures, and emotions, thereby realizing more natural and consistent communication.

[0046] [Example of Dialogue] FIG. 6 is a diagram showing an example of dialogue. The system is a DH executed by the cognitive control device 10. FIG. 6 shows operations and utterances of the system and the user in chronological order. Utterances are performed by voice or text display. Furthermore, the system's operations are performed by a robot.

[0047] A solid-line box indicates an utterance. A dotted-line box indicates an action. "(Looks interesting)", "(Looks unconvinced)", and "(Looks convinced)" indicate content recognized by the system from external information (such as video of the user's gestures). For example, "(Looks unconvinced)" means that the system has recognized from external information that the user is not convinced by the explanation.

[0048] Assume that a scenario for explaining information of technology-related exhibits to a user is set. The scenario includes tasks such as explaining an outline of the exhibit, checking the user's degree of understanding of the exhibit, and giving a detailed explanation of the exhibit.

[0049] First, a response from the first lower-layer cognitive processing will be described. For example, in response to the user's voice input "Oh right, I'm actually having trouble with ΔΔ right now.", the cognitive control device 10 outputs an action "(nod)" as a response.

[0050] As described above, in the first lower-layer cognitive processing, external information is used as input, and a simple action is output as a response. Note that in the first lower-layer cognitive processing, the dialogue context, the personalities of the user and DH, the relationship between the user and DH, and the progress status of the scenario do not need to be considered.

[0051] Next, a response from the second lower-layer cognitive processing will be described. For example, in response to an input of the recognized content "(Looks interesting)" based on the user's gesture, the cognitive control device 10 outputs an utterance "Shall I explain it to you?" as a response. In addition, in the second lower-layer cognitive processing, short utterance responses such as "Thank you" and "Oh, I see" are generated.

[0052] As described above, in the second lower-layer cognitive processing, external information, the dialogue context, the personalities of the user and DH, and the relationship between the user and DH are used as inputs, and in addition to simple actions, short utterances are output as responses. Note that in the second lower-layer cognitive processing, the progress status of the scenario does not need to be considered.

[0053] Next, we will explain the response based on the intermediate cognitive processing. For example, the cognitive control device 10 outputs an utterance as a response that takes into account the progression of the scenario, based on the various input information. For example, at the beginning of the scenario, the cognitive control device 10 outputs an utterance that explains the overview of the exhibit, such as, "Yes, that's right. This is a technology that does XX with XX." Also, for example, if the cognitive control device 10 recognizes that the user is requesting an explanation of the exhibit, it will proceed to a detailed explanation of the exhibit, and will output an utterance that explains the exhibit in detail, such as, "XX is..." and "Therefore, △△ can be solved by XX..."

[0054] Typically, the first lower-level cognitive processing generates a response faster than the second lower-level cognitive processing, and the second lower-level cognitive processing generates a response faster than the middle-level cognitive processing. The cognitive control device 10 outputs responses in the order they are generated, in principle. However, the cognitive control device 10 may interrupt responses according to predetermined rules.

[0055] For example, in the example in Figure 6, suppose the user utters, "That's right. And I'm having trouble with △△," and the second lower-level cognitive processing generates the utterance "That's a problem." as a response. And suppose that before the utterance "That's a problem." is output, the middle-level cognitive processing generates the utterance "I see. This is a technology that solves △△ using ××." In this case, as shown in Figure 6, the cognitive control device 10 interrupts the response generated by the second lower-level cognitive processing and outputs the response generated by the middle-level cognitive processing.

[0056] The cognitive control device 10 may discard responses that are not adopted due to an interrupt without outputting them. For example, if the cognitive control device 10 receives an interruption to a response generated by the intermediate cognitive processing, it discards the responses generated by the first lower-level cognitive processing and the second lower-level cognitive processing.

[0057] For example, the cognitive control device 10 calculates a naturalness score for each response generated by a cognitive process. If a response with a higher score is generated later, before the previously generated response is output, the later-generated response is inserted into the previously generated response and output.

[0058] [Processing of the First Embodiment] The processing flow of the cognitive control device 10 will be explained using Figure 7. Figure 7 is a flowchart showing the processing flow of the cognitive control device.

[0059] As shown in Figure 7, first, the cognitive control device 10 performs upper-level cognitive processing (step S101). The cognitive control device 10 also accepts information input (step S102). The input information includes external information, dialogue history, information about the user (which may include dialogue history), and information about the object of the scenario (for example, a summary of the exhibit).

[0060] Next, the cognitive control device 10 executes the first lower-level cognitive treatment, the second lower-level cognitive processing, and the intermediate-level cognitive processing (step S103).

[0061] The cognitive control device 10 outputs according to the results of each process in accordance with priority and interrupt rules (step S104).

[0062] The cognitive control device 10 then determines whether the scenario has ended (step S105). If the scenario has not ended (step S105; No), the cognitive control device 10 returns to step S102 and repeats the process. If the scenario has ended (step S105; Yes), the cognitive control device 10 terminates the process.

[0063] [Effects of the First Embodiment] The cognitive control device 10 executes multiple processes that generate responses to input information at different speeds. The cognitive control device 10 also outputs the responses generated in each of the multiple processes.

[0064] For example, the cognitive control device 10 performs a first lower-level cognitive process that generates a reflexive response to information, a second lower-level cognitive process that is slower than the first lower-level cognitive process and generates a response based on the context indicated in the information, and an intermediate-level cognitive process that is slower than the second lower-level cognitive process and generates a response based on the context and a set scenario.

[0065] For example, the cognitive control device 10 generates an action of a robot that interacts with the user as a response through a first lower-level cognitive processing, generates a sentence as a response through a second lower-level cognitive processing, and generates a sentence with more information than the sentence generated by the second lower-level cognitive processing as a response through an intermediate-level cognitive processing.

[0066] Responses generated by multiple cognitive processes differ in their generation speed and the amount of information they can represent. Mid-level cognitive processing using heavy AI takes longer to generate responses compared to lower-level cognitive processing using light AI, but it can generate responses with a greater amount of information. The cognitive control device 10 achieves smooth responses in real-time dialogue with the user by combining and outputting responses of multiple different types. In particular, in mid-level cognitive processing, the cognitive control device 10 can provide consistently seamless responses by referring to the dialogue history, situation, user status, etc. from the database and deepening its understanding of the user and the DH itself. As a result, the cognitive control device 10 can optimize response generation in the DH.

[0067] The cognitive control device 10 outputs responses generated by each of the multiple processes using priority control and interrupt control. Furthermore, if the cognitive control device 10 interrupts the first response generated by the first process among the multiple processes and outputs a second response generated after the first response by a second process that is slower than the first process, the cognitive control device 10 discards the first response.

[0068] In this way, the cognitive control device 10 does not simply output responses in the order they are generated, but rather performs interrupt control as appropriate, thereby enabling a more natural dialogue between the DH and the user.

[0069] [System Configuration, etc.] Furthermore, the components of each part shown in the diagram are functional concepts and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown in the diagram, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. In addition, all or any part of the processing functions performed by each device can be realized by a CPU and the program executed on that CPU, or by hardware using wired logic.

[0070] Furthermore, among the processes described in the embodiments described above, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

[0071] [Program] The cognitive control device 10 described above can be implemented by installing a program (cognitive control program) as packaged software or online software on a desired computer. For example, by having the computer run the above program, the computer can function as the cognitive control device 10. The term "computer" here includes smartphones, mobile phones, mobile communication terminals such as PHS (Personal Handyphone System), and terminals such as PDA (Personal Digital Assistant).

[0072] Figure 8 shows an example configuration of a computer that executes a cognitive control program. Computer 1000 has, for example, memory 1010 and CPU 1020. Computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0073] Memory 1010 includes ROM (Read Only Memory) 1011 and RAM (Random Access Memory) 1012. ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0074] The hard disk drive 1090 stores, for example, the OS 1091, application program 1092, program module 1093, and program data 1094. That is, the program that defines each process executed by the cognitive control device 10 is implemented as a program module 1093 in which executable code is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing a process similar to the functional configuration in the cognitive control device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0075] Furthermore, the data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, memory 1010 or hard disk drive 1090. The CPU 1020 then reads the program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as needed and executes them.

[0076] Furthermore, the program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090; for example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via a network interface 1070.

[0077] 10 Cognitive control unit 11 Input unit 12 Output unit 13 Storage unit 14 Control unit 131 User database 132 Model information 141 Management unit 142 Task execution unit 143 Response prediction processing unit 144 Initial response processing unit 145 Execution processing unit 146 Output processing unit

Claims

1. A cognitive control device comprising: a cognitive processing unit that executes a plurality of processes that generate responses to input information at different speeds; and an output processing unit that outputs the responses generated by each of the plurality of processes.

2. The cognitive control device according to claim 1, characterized in that the cognitive processing unit performs a first lower-level cognitive processing that generates a reflexive response to the information, a second lower-level cognitive processing that is slower than the first lower-level cognitive processing and generates a response based on the context indicated in the information, and an intermediate-level cognitive processing that is slower than the second lower-level cognitive processing and generates a response based on the context and a set scenario.

3. The cognitive control device according to claim 2, characterized in that the cognitive processing unit generates an action of a robot that interacts with a user as a response by the first lower-level cognitive processing, the second lower-level cognitive processing generates a sentence as a response, and the middle-level cognitive processing generates a sentence with a larger amount of information as a response than the sentence generated by the second lower-level cognitive processing.

4. The cognitive control device according to claim 1, characterized in that the output processing unit outputs the responses generated by each of the plurality of processes by priority control and interrupt control.

5. The cognitive control device according to claim 1, characterized in that the output processing unit interrupts the first response generated by the first process among the plurality of processes and outputs a second response generated after the first response by a second process which is slower than the first process, and then discards the first response.

6. A cognitive control method performed by a cognitive control device, comprising: a cognitive processing step that executes a plurality of processes that generate responses to input information at different speeds; and an output processing step that outputs the responses generated in each of the plurality of processes.

7. A cognitive control program characterized by causing a computer to execute a cognitive processing step that executes a plurality of processes that generate responses to input information at different speeds, and an output processing step that outputs the responses generated in each of the plurality of processes.

Images