Robot, response method and program

Abstract

To speedily and properly respond to human utterance to a robot.SOLUTION: The present invention relates to a controller for a robot capable of responding to an object, and the controller comprises: a voice acquisition unit 21 which acquires utterance of the object; a partial analysis unit 23 which partially analyzes the utterance acquired by the voice acquisition unit 21 when the object makes the utterance; a gesture response control unit 24 which controls a first response by the robot to the object by use of non-linguistic behavior according to an analysis result of the partial analysis unit 23; an utterance analysis unit 25 which analyzes the utterance of the object acquired by the voice acquisition unit 21 with a section longer than the section of the utterance analyzed by the partial analysis unit 23; a response sentence generation unit 26 which generates a response sentence to the utterance of the object according to an analysis result obtained by the utterance analysis unit 25; and a linguistic response control unit 27 which controls a second response using a language by the robot based upon the response sentence generated by the response sentence generation unit 26.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a robot , How to respond and programs. [Background technology]

[0002] A robot that operates autonomously and converses with humans has been proposed. For example, Patent Document 1 describes a natural language processing device that enables appropriate dialogue with humans. The natural language processing device of Patent Document 1 includes a sequential analysis processing unit that executes analysis processing sequentially and in parallel in each analysis processing unit every time a part of a natural language sentence in an analyzable unit is input, and an output unit that obtains an output such as a dialogue response sentence based on the analysis result in each analysis processing unit of the sequential analysis processing unit. Each processing unit prepared in the sequential analysis processing unit obtains the previous analysis result immediately before or before that in its own processing unit and the previous analysis result immediately before or before that in other processing units, and obtains the analysis result while looking ahead with reference to the obtained analysis result. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-102771 A Summary of the Invention [Problem to be solved by the invention]

[0004] Generally, a robot that converses with a person generates and speaks a response sentence after listening to what the person has said, so if the robot does not respond in the middle of an utterance, the speaker feels as if the robot is not listening at all. In the natural language processing device of Patent Document 1, each time a part of a sentence is input, each analysis processing unit performs analysis processing sequentially and in parallel in order to return a response as quickly as possible.

[0005] However, when this natural language processing device is applied to a robot configured to be able to converse with a user, the analysis process is performed using the results predicted by the look-ahead process that looks ahead at the next input data, and the results of this analysis process may be incorrect. If the analysis result is incorrect, an inappropriate response sentence will be generated for the sentence input by the user, and as a result, it will be impossible to respond appropriately to the user's utterance.

[0006] The present invention has been made in consideration of the above circumstances, To provide a response that can prevent the speaker from feeling that the robot is not listening at all even when the response in the response sentence is suppressed. The purpose of this document is to: [Means for solving the problem]

[0007] In order to achieve the above object, a robot according to the present invention is a robot capable of responding by voice and by gesture, and includes: a first identification means for starting a predetermined identification process in response to a start of an utterance by a speaker to identify a category of a content that the speaker is trying to convey through the utterance; and a control means for controlling the robot to execute a gesture that is preset in association with the identified category as soon as the category is identified by the identification process regardless of whether the utterance by the speaker has ended or not. Even in a case where the control means controls the robot to execute the gesture response before the utterance by the speaker ends, the control means controls the robot to wait for the utterance by the speaker to end before executing the voice response. and controlling the robot so that the robot remains silent until the speaker finishes speaking. The robot is also configured to be controllable to perform different gestures for each category. Further, a response method according to the present invention is a response method executed by a robot capable of executing voice responses and gesture responses, and includes a specifying step of starting a predetermined specification process in response to a start of speech by a speaker to specify a category of a content that the speaker is trying to convey through the speech, and a control step of controlling the robot to execute a gesture that is preset in association with the specified category as soon as the category is specified by the specification process regardless of whether the speech by the speaker has been completed or not, and the control step controls the robot to execute the voice response after waiting for the speech by the speaker to be completed, even in a case where the control step controls the robot to execute the gesture response before the speech by the speaker is completed. and controlling the robot so that the robot remains silent until the speaker finishes speaking. The robot is also configured to be controllable to perform different gestures for each category. The program according to the present invention causes a computer of a robot capable of responding by voice and by gesture to function as: a specifying means for starting a predetermined specifying process in response to the start of an utterance by a speaker to specify a category of the contents that the speaker is trying to convey by the utterance; and a control means for controlling the robot to execute a gesture that is preset in association with the specified category as soon as the category is specified by the specifying process regardless of whether the utterance by the speaker has ended or not. Even in the case where the control means controls the robot to execute the gesture response before the utterance by the speaker ends, the control means controls the robot to execute the voice response after waiting for the utterance by the speaker to end. and controlling the robot so that the robot remains silent until the speaker finishes speaking. The robot is also configured to be controllable to perform different gestures for each category. Effect of the Invention

[0008] According to the present invention, Even when responses in response sentences are suppressed, responses can be made that can prevent the speaker from feeling that the robot is not listening at all. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing a schematic configuration of a robot to which a control device according to an embodiment of the present invention is applied; [Diagram 2] 2 is a block diagram showing a functional configuration of a control device of a robot according to an embodiment. FIG. [Figure 3A] FIG. 13 is a front view showing the motion of gesture number 2 that the control device in the embodiment causes the robot to perform. [Figure 3B] FIG. 13 is a front view showing the motion of gesture number 3 that the control device in the embodiment causes the robot to perform. [Figure 3C] FIG. 13 is a front view showing the motion of gesture number 4 that the control device in the embodiment causes the robot to perform. [Figure 3D] FIG. 13 is a front view showing the motion of gesture number 5 that the control device in the embodiment causes the robot to perform. [Figure 3E] FIG. 13 is a front view showing the motion of gesture number 6 that the control device in the embodiment causes the robot to perform. [Figure 4] 5 is a flowchart showing a conversation recording process of the control device according to the embodiment. [Diagram 5] 5 is a diagram showing an example of a conversation record recorded in the conversation recording process shown in FIG. 4. [Figure 6] 4 is a flowchart showing an analytical learning process of the control device according to the embodiment. [Figure 7] FIG. 13 is a diagram showing an example in which all spoken sentences corresponding to a predetermined target are read out from a conversation record. [Figure 8] FIG. 13 is a diagram illustrating an example of a unique phoneme sequence table. [Figure 9] FIG. 13 is a diagram showing an example of correspondence between sentences and gestures. [Figure 10] 10 is a flowchart showing a response gesture database registration process of the control device according to the embodiment. [Figure 11] 11 is a timing chart showing an example of the time when a speech sentence is uttered. [Figure 12]11 is a diagram showing an example of a response gesture database used in the response gesture database registration process shown in FIG. 10. FIG. [Figure 13] 4 is a flowchart showing a predicted response control process of the control device according to the embodiment. [Figure 14] FIG. 13 is a diagram illustrating an example of a response time list. [Figure 15] 4 is a flowchart showing a linguistic response control process of the control device according to the embodiment. [Figure 16] 10 is a timing chart showing an example of a response of a robot according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

[0011] Embodiment FIG. 1 is a diagram showing a schematic configuration of a robot 1 to which a control device 2 according to an embodiment of the present invention is applied. The robot 1 has a three-dimensional shape that resembles a human (child) in appearance. The robot 1 includes a head 101, a body 102, and arms 103. The head 101 and the arms 103 are parts that can be moved by a gesture operating unit 7, which is a drive device built into the robot 1. The head 101 is attached to the body 102 by a neck joint 5 so as to be capable of bending, extending, rotating, and lateral bending. The arms 103 are attached to the body 102 by a shoulder joint 6 so as to be capable of bending, extending, and adduction and abduction.

[0012] The robot 1 includes a microphone 3 for collecting voice, a speaker 4 for outputting voice, a gesture operating unit 7 for moving a head 101 and an arm 103, and a control device 2. The robot 1 can converse with a person by performing voice recognition of an utterance of a predetermined target, generating a response sentence to the utterance, and uttering the response sentence by voice synthesis. The robot 1 can also respond with non-verbal behavior, that is, by moving the head 101 and the arm 103, when conversing with a predetermined target. In the robot 1, a plurality of different gesture actions are set as such non-verbal behavior, and these plurality of gesture actions include actions such as nodding and raising and lowering the arms.

[0013] The robot 1 performs various actions in response to external stimuli such as a call or contact from a predetermined object that exists outside the robot 1. This allows the robot 1 to communicate with and interact with the predetermined object. The predetermined object is an object that exists outside the robot 1 and that communicates and interacts with the robot 1. The predetermined object is, for example, the user who owns the robot 1, people around the user (such as the user's close relatives or friends), and other robots that can speak. The predetermined object can also be called a communication object, a communication partner, an exchange object, an exchange partner, etc.

[0014] 2 is a block diagram showing a functional configuration of the control device 2. The control device 2 is electrically connected to a microphone 3 and a speaker 4, receives a voice signal from the microphone 3, and utters a response sentence from the speaker 4. The control device 2 also controls a gesture operation unit 7 to cause the robot 1 to execute the above gesture actions. The gesture operation unit 7 includes an actuator, and drives, for example, the head 101 and the arm 103 of the robot 1 shown in FIG. 1.

[0015] The robot 1 is provided with sensors (potentiometers) at the joints 5 that detect the rotation angles of flexion / extension, rotation, and lateral bending of the head 101, and the gesture actuator 7 causes the head 101 to perform a predetermined movement by feedback control using the detection values ​​of the sensors at the joints 5. Similarly, the robot 1 is provided with sensors (potentiometers) at the joints 6 that detect the rotation angles of flexion / extension, adduction / abduction of the arm 103, and the gesture actuator 7 causes the arm 103 to perform a predetermined movement by feedback control using the detection values ​​of the sensors at the joints 6.

[0016] 3A to 3E are front views showing examples of gesture actions to be executed by the robot 1. Fig. 3A shows the gesture action of gesture number 2 for responding to an utterance of "Good morning" from a predetermined target. The gesture action of gesture number 2 is an action in which the head 101 is turned forward (or slightly above the front) by the joints 5 and the gesture actuator 7, and the left and right arms 103 are raised above the shoulders by the joints 6 and the gesture actuator 7.

[0017] 3B shows a gesture action of gesture number 3 for responding to a predetermined target's utterance "Hello." The gesture action of gesture number 3 is an action of directing the head 101 to the front (or slightly above the front) by the joints 5 and the gesture actuator 7, and raising the left and right arms 103 in front of the head 101 by the joints 6 and the gesture actuator 7.

[0018] 3C shows a gesture action of gesture number 4 for responding to a predetermined target's speech of "Bye-bye," "Goodbye," or "Goodbye." The gesture action of gesture number 4 is an action of tilting the head 101 slightly to the left by the joint 5 and the gesture actuator 7, and raising the right arm 103 close to the head 101 while keeping the left arm 103 down by the joint 6 and the gesture actuator 7. In the gesture action of gesture number 4, the right arm 103 may be raised and swung from side to side.

[0019] 3D shows a gesture action of gesture number 5 for responding to a predetermined target's utterance "I'm home." The gesture action of gesture number 5 is an action of directing the head 101 to the front (or slightly above the front) by the joints 5 and the gesture actuator 7, and raising the left and right arms 103 to shoulder height by the joints 6 and the gesture actuator 7.

[0020] 3E shows a gesture action of gesture number 6 for responding to a predetermined target's utterance "good night." The gesture action of gesture number 6 is an action of directing the head 101 downward by the joint 5 and the gesture actuator 7 while keeping the left and right arms 103 downward by the joint 6 and the gesture actuator 7.

[0021] As shown in FIG. 2, the control device 2 includes a control unit 20, a storage unit 30, and an I / O interface for inputting and outputting signals to and from the microphone 3, the speaker 4, and the gesture operation unit 7. The control unit 20 is composed of a CPU (Central Processing Unit) and the like, and by executing a program stored in the storage unit 30, it realizes the functions of each unit (voice acquisition unit 21, identification unit 22, partial analysis unit 23, gesture response control unit 24, speech analysis unit 25, response sentence generation unit 26, language response control unit 27, learning unit 28, and identification unit 29) described later, and controls the operation of the robot 1. The storage unit 30 is composed of a ROM (Read Only Memory) and a RAM (Random Access Memory), and the like, and a part or the whole of the ROM is composed of an electrically rewritable memory (such as a flash memory). The robot 1 may include, for example, an imaging device for recognizing the face of a predetermined target, and the control unit 20 may communicate with the imaging device via the I / O interface to acquire image data and the like.

[0022] The voice acquisition unit 21, which functions as an acquisition means, performs A / D conversion on a voice signal input from the microphone 3 by a predetermined subject by sampling at a predetermined frequency, and generates digital data by, for example, linear PCM (Pulse Code Modulation). The voice acquisition unit 21 further converts the digital data by short-time Fourier transform (STFT) to acquire a spectrogram. The voice acquisition unit 21 sends the acquired spectrogram to the identification unit 22.

[0023] The speech acquisition unit 21 also sequentially analyzes the spectrogram to acquire a string of phonemes that indicate the contents of speech. The phonemes are acquired by segmental phoneme labeling. Segmental phoneme labeling is to decompose the speech signal into components such as consonants and vowels that are considered to constitute the speech signal, and to assign labels that represent each component. The speech acquisition unit 21 decomposes the speech signal into components based on the transition of formants acquired from the spectrogram, and the pattern and changes of the spectrogram, and selects and assigns labels that match the patterns of the components. The process of acquiring the string of phonemes (labeling the speech data in phoneme units) can be performed, for example, using the open source Julius phoneme segmentation kit. The speech acquisition unit 21 converts the segmental phoneme labels into phonemes, and sequentially sends the acquired phonemes and the time between phonemes or the time when the phonemes were uttered to the learning unit 28 and the speech analysis unit 25.

[0024] The identification unit 22 extracts voice feature data, such as i-vector, from the spectrogram sent from the voice acquisition unit 21, and identifies the predetermined target by determining which of the voice feature data of multiple predetermined targets stored in the storage unit 30 the extracted voice feature data matches, and acquires the target ID of the identified predetermined target. The identification unit 22 sends the acquired target ID to the learning unit 28 and the utterance analysis unit 25.

[0025] The learning unit 28, which functions as a learning means, learns, for each predetermined target, the shortest phoneme string that can identify the content of the utterance input from the microphone 3, from the phoneme string sent from the voice acquisition unit 21 and the sentence of the voice recognition result by the utterance analysis unit 25. The partial analysis unit 23, which functions as a first analysis means, partially analyzes the utterance input from the predetermined target, and if a part of the phoneme string in the utterance matches the phoneme string learned by the learning unit 28 (the shortest phoneme string described later), sends that information to the gesture response control unit 24. In addition, the identification unit 29, which functions as an identification means, identifies a gesture action corresponding to the input utterance according to the voice recognition result by the utterance analysis unit 25. If a part of the phoneme string in the input utterance matches the shortest phoneme string, the gesture response control unit 24 selects a gesture action corresponding to the matching shortest phoneme string, and causes the gesture operation unit 7 to perform the response.

[0026] 3A to 3E, the gesture response control unit 24 sends to the gesture operation unit 7 an action sequence describing the action start angle, action angular velocity, action angular acceleration, stop angle, and stop time of each of the flexion / extension, rotation, and lateral bending of the head 101, and the flexion / extension and adduction / abduction of the arm 103. The gesture operation unit 7 generates control signals for driving the head 101 and the arm 103 in accordance with the detection values ​​of the sensors of the joints 5 and 6 in accordance with the action sequence sent from the gesture response control unit 24, and drives the head 101 and the arm 103 by inputting the generated signals to actuators.

[0027] The speech analysis unit 25, which functions as the second analysis means, uses the sequence of phonemes sent from the speech acquisition unit 21 to analyze the input speech for a section longer than the section of the speech analyzed by the partial analysis unit 23, which functions as the first analysis means, and performs speech recognition. At this time, the speech analysis unit 25 performs morphological analysis and the like by referring to a dictionary database stored in the storage unit 30, and performs speech recognition on the speech data sent from the speech acquisition unit 21. The speech analysis unit 25 sends the analysis result to the response sentence generation unit 26. The response sentence generation unit 26 generates a response sentence suitable for the content of the speech for each speaker who has spoken. The response sentence generation unit 26 sends the generated response sentence to the language response control unit 27, and the language response control unit 27 causes the response sentence to be spoken from the speaker 4 by voice synthesis.

[0028] The control unit 20 executes a conversation recording process, an analysis learning process, and a prediction response control process in order to return a non-verbal response by a gesture action as soon as possible in response to the user's utterance. The conversation recording process, the analysis learning process, and the prediction response control process will be described below in order.

[0029] 4 is a flowchart showing a conversation recording process according to the embodiment. In the conversation recording process, the control device 2 records the contents of the conversation between the robot 1 and a specific target and the recognized short sentences for each specific target registered in the robot 1. The conversation recording process is repeated every time a series of conversations is held, and conversation record data is accumulated.

[0030] The control unit 20 starts a conversation recording process at the same time as a conversation between the robot 1 and a predetermined target is started. For example, when the voice acquisition unit 21 detects a voice with a voice level exceeding a predetermined threshold, the control unit 20 starts a voice recognition process, and if the voice recognition is successful, it determines that a conversation has started. Also, if there is a lot of noise in the voice, it may be arranged to determine that a conversation has started if the voice recognition result is segmented and one word is recognized.

[0031] First, the identification unit 22 identifies a predetermined object as described above (step S401).

[0032] Next, the speech analysis unit 25 determines whether the conversation has ended (step S402). The end of the conversation can be determined, for example, by determining that the length of the silent period exceeds a predetermined length, by determining that the context has ended, or by determining that the face of the speaker cannot be recognized in the case of face recognition using a camera, or by a combination of these. The end of the context can be determined, for example, when the speech analysis unit 25 detects that there is no next utterance from the predetermined target for a predetermined time after an answer to a question is uttered, or when an utterance declaring the end of the conversation such as "bye-bye" or "see you later" is made by the predetermined target.

[0033] When the conversation has not ended (step S402; N), the speech analysis unit 25 performs speech recognition on the speech data acquired by the speech acquisition unit 21 from the speech of the predetermined target by performing morphological analysis or the like with reference to the dictionary database stored in the storage unit 30 (step S403). Then, the speech analysis unit 25 records a conversation record in which the recognition result and the speech data are associated with each other as a conversation record database in the storage unit 30 (step S404), and returns to step S402. As shown in FIG. 5, the conversation record includes a time, a conversation partner (target ID), a sentence uttered by the predetermined target during the conversation, a phoneme string included in the utterance of the predetermined target during the conversation, and speech data. Here, the time is the time when the conversation started or ended. The uttered sentence is a character string that represents the content of the uttered sentence. The phoneme string is a string of phonemes of the uttered sentence. The voice data column is information that specifies the voice data itself or a file in which the voice data is recorded. The voice data is used later to analyze the length of the utterance of the phoneme string.

[0034] On the other hand, when the conversation has ended (step S402; Y), the utterance analysis unit 25 ends the conversation recording process. In this manner, the utterance analysis unit 25 stores the conversation record in which the recognition result of the utterance of the predetermined target is associated with the voice data as a conversation record database in the storage unit 30.

[0035] Next, the analysis learning process will be described with reference to Fig. 6. This analysis learning process is a process for generating a response gesture database used when controlling gesture actions and recording it in the storage unit 30. The analysis learning process is executed, for example, when a predetermined number of conversations are newly stored in the conversation record database, or each time a predetermined period of time has elapsed.

[0036] The learning unit 28 first clears the response gesture database generated in the previous analysis and learning process (S600). The analysis and learning is performed for each of all predetermined targets registered in the robot 1. Next, from the conversation record database shown in Fig. 5, as shown in Fig. 7, all of the multiple utterance sentences corresponding to the target ID (ID=1) of the first registered predetermined target are read out, and the multiple utterance sentences that have been read out are stored in a predetermined storage area of ​​the RAM (step S601).

[0037] Next, the learning unit 28 determines whether or not analysis learning for controlling gesture motions, which will be described later, has been completed for all the predetermined targets (step S602). If analysis learning has not been completed for all the predetermined targets (step S602; N), a table (unique phoneme string table) is created by deleting the first one (e.g., "Good morning" at 9:01 on 9 / 9 / 2018) of the overlapping utterance sentences (e.g., "Good morning") from the multiple utterance sentences (as shown in FIG. 7) stored in step S601, and the other utterance sentences (e.g., "Good morning" at 8:00 on 9 / 12 / 2018, 8:00 on 9 / 14 / 2018, and 9:00 on 9 / 14 / 2018) and storing the table in a predetermined storage area of ​​the RAM (step S603). This unique phoneme sequence table is temporary for analytical learning of one target ID, and does not require time and voice data, and since the target ID is numbered for each predetermined target, it is sufficient to have a correspondence between the spoken sentence and the phoneme sequence, as shown in Fig. 8. Then, the learning unit 28 reads out the first spoken sentence from the unique phoneme sequence table (step S604).

[0038] Next, it is determined whether or not all the utterance sentences have been read from the unique phoneme sequence table (step S605). When all the utterance sentences have not been read from the unique phoneme sequence table (step S605; N), the identification unit 29 identifies a gesture action corresponding to the utterance sentence read in step S604 by using a gesture action database shown in FIG. 9 stored in the storage unit 30 (step S606). As shown in FIG. 9, this gesture action database stores gesture corresponding sentences and gesture action numbers in association with each other, and in step S606, the number of the gesture action corresponding to the gesture corresponding sentence that matches the utterance sentence read in step S604 is identified as the number representing the corresponding gesture action. For example, for the gesture corresponding sentence "Good morning", the gesture number "2" representing the gesture action shown in FIG. 3A is identified.

[0039] Next, the learning unit 28 determines whether or not the corresponding gesture action has been identified in the above step S606 (step S607). If the gesture action has been identified in step S606 (step S607; Y), the learning unit 28 executes a response gesture database registration process (described later, shown in FIG. 10) (step S608). Then, the learning unit 28 reads out the speech sentence following the speech sentence read out in step S604 from the unique phoneme sequence table (step S609), and executes the above steps S605 to S608 again. On the other hand, if the corresponding gesture action cannot be identified in the above step S606 (step S607; N), the learning unit 28 skips the above step S608, does not execute the response gesture database registration process, and executes steps S609 and onward.

[0040] Then, when all the utterance sentences have been read out from the unique phoneme sequence table as a result of repeatedly executing steps S605 to S609 (step S605; Y), all the utterance sentences corresponding to the target ID next to the first target ID that was the target read out in step S601 are read out from the utterance sentences stored in the conversation record database shown in Fig. 5, and the read out utterance sentences are stored in a predetermined storage area of ​​the RAM (step S610). Next, steps S602 and after are executed again. Thus, when the analytical learning by the above-mentioned steps S603 to S609 is completed for all the predetermined targets (step S602; Y), the analytical learning process is terminated.

[0041] Next, the response gesture database registration process in step S608 in Fig. 6 will be described with reference to Fig. 10. Through this response gesture database registration process, the learning unit 28 specifies the shortest phoneme string as the minimum (shortest) phoneme string capable of identifying the spoken sentence read out in step S604 or S609 in Fig. 6. For example, for the sentence "good morning," the phoneme string "oha" is specified.

[0042] First, the learning unit 28 sets a counter N as a local variable to 1 (step S1000), and reads out the phoneme string from the beginning to the Nth of the phoneme string of the uttered sentence read out in step S604 or S609 in Fig. 6 (step S1001). Then, it is determined whether or not the length of the phoneme string of the uttered sentence of interest is equal to the length N of the read phoneme string (step S1002). If the length of the phoneme string of the uttered sentence is equal to the length N of the read phoneme string (step S1002; Y), it is determined that there is no shortest phoneme string that can identify the uttered sentence, and the process returns to the flowchart of Fig. 6 (step S609) without storing anything in the response gesture database.

[0043] When the length of the phoneme sequence of the spoken sentence is not equal to the length N of the read phoneme sequence (step S1002; N), the learning unit 28 searches the unique phoneme sequence table for a spoken sentence other than the targeted sentence whose phoneme sequence from the beginning matches the read phoneme sequence (step S1003).

[0044] Then, the learning unit 28 determines whether or not a matching phoneme string was found in step S1003 (step S1004). If a matching phoneme string was found (step S1004; Y), the counter N is incremented by 1 (step S1005), and the phoneme string from the beginning to the Nth phoneme string of the utterance sentence of interest is read (step S1006). Then, the process returns to step S1002, and the process from step S1002 is executed again.

[0045] If, as a result of repeatedly executing steps S1002 to S1006, there is no utterance other than the target sentence in which the phoneme sequence from the beginning matches the read-out phoneme sequence in the unique phoneme sequence table (step S1004; N), the read-out phoneme sequence up to the Nth phoneme sequence is recorded in the RAM of memory unit 30 as the shortest phoneme sequence capable of identifying the content of the target utterance (hereinafter referred to as the "shortest phoneme sequence") (step S1007).

[0046] Next, the learning unit 28 refers to a predetermined area in which a plurality of utterance sentences (as shown in FIG. 7) corresponding to the target ID of a predetermined target read out in step S601 or step S610 in FIG. 6 are stored, and measures the average length of the corresponding phoneme strings (shortest phoneme strings) up to the Nth utterance sentence for all utterance sentences that are the same as the utterance sentence of interest (step S1008). At this time, the number of occurrences of the same utterance sentence is counted.

[0047] In step S1008, all the voice data of the same utterance sentence as the utterance sentence of interest is extracted, and the lengths of the sections of the phonemes of the shortest phoneme sequence identified in step S1007 (the time from the beginning to the end of the shortest phoneme sequence) are extracted, and the average time is calculated. For example, assume that the shortest phoneme sequence of the utterance sentence "Good morning" is "oha". FIG. 11 shows voice data of the same utterance sentence "Good morning" of the same predetermined target uttered at different times, arranged vertically with the same start timing. The learning unit 28 extracts the voice data of the utterance sentence "Good morning" from the voice data of multiple utterance sentences corresponding to the target ID of the predetermined target shown in FIG. 7, as shown in FIG. 11, and measures the average length of the utterance of the shortest phoneme sequence "oha" by averaging the time from the start of the voice data to the end of the phoneme of "a", for example, t1 and t2 in FIG. 11.

[0048] Then, the learning unit 28 stores the spoken sentence, the shortest phoneme string identified in step S1007, the time when the shortest phoneme string was spoken measured in step S1008 (which may be the time when the shortest phoneme string was spoken plus the detection time (e.g., 20 ms)), the number of the gesture action identified in step S606 of FIG. 6, and the number of occurrences of the spoken sentence in the response gesture database shown in FIG. 12 (step S1009), and returns to the flowchart of FIG. 6 (step S609).

[0049] 5, 9, and 12 show examples of greetings such as "Good morning," but the spoken sentences and gesture-enabled sentences may each include interjections, interjections, or exclamations such as "Um," "Well," "Oh," and "Oh," respectively.

[0050] Next, the predictive response control process will be described with reference to Fig. 13. The predictive response control process is started, for example, when the voice acquisition unit 21 detects a voice with a voice level exceeding a predetermined threshold. When the control unit 20 starts the predictive response control, first, the identification unit 22 identifies a predetermined target (step S1300). Next, the partial analysis unit 23 reads out the shortest phoneme string length for the identified predetermined target from the response gesture database, and generates a response time list as shown in Fig. 14 using the read out shortest phoneme string length (step S1301). In the response time list shown in Fig. 14, the response times are listed in order from shortest to oldest.

[0051] Then, the control unit 20 judges whether the conversation between the predetermined target and the robot 1 has ended (step S1302). The end of the conversation can be judged in the same manner as step S402 of the conversation recording process (FIG. 4). If the conversation has not ended (step S1302; N), it judges whether the predetermined target has started speaking (step S1303) and waits until the target starts speaking (step S1303; N). The start of speaking is detected, for example, when the voice level reaches or exceeds a threshold value, or by facial recognition of the predetermined target when facial recognition is performed using a camera. When the predetermined target has started speaking (step S1303; Y), the process from step S1304 onwards is executed. On the other hand, when the conversation ends (step S1302; Y), the prediction response control process ends. As described above, the process from step S1304 onwards is executed every time a speech by a predetermined target starts. After step S1304, the control unit 20 performs the following process the same number of times as the number of predicted response times recorded in the predicted response time list.

[0052] The partial analysis unit 23 reads out the first response time from the response time list generated in step S1301 (step S1304), determines whether the response time has elapsed since it was determined in step S1303 that the target speech started (step S1305), and waits until the response time has elapsed (step S1305; N). When the response time has elapsed (step S1305; Y), the partial analysis unit 23 cuts out the target speech input from the microphone 3 from the target speech started until the response time has elapsed (step S1306). Then, the partial analysis unit 23 determines whether silence is detected in the cut-out speech (step S1307). If silence, for example, with a level below a threshold, is included in the cut-out target speech for a certain period of time (for example, 100 ms) or more (step S1307; Y), the process returns to step S1302.

[0053] On the other hand, when the extracted voice of the predetermined target does not include a non-voice portion (step S1307; N), the partial analysis unit 23 converts the voice extracted in step S1306 into a phoneme string (step S1308). Then, the partial analysis unit 23 determines whether or not a phoneme string that matches the phoneme string converted in step S1308 exists among the multiple shortest phoneme strings stored in the response gesture database corresponding to the predetermined target identified in step S1300 (step S1309). When a shortest phoneme string that matches the converted phoneme string exists (step S1309; Y), the robot 1 is made to execute a gesture action corresponding to the shortest phoneme string that matches (step S1312), and the process returns to step S1302.

[0054] For example, assume that the phoneme string converted in step S1308 is "oha." When the partial analysis unit 23 searches for the phoneme string "oha" among the shortest phoneme strings in the response gesture database shown in Fig. 12, the partial analysis unit 23 finds a match with the shortest phoneme string "oha," and obtains the corresponding gesture number "2" and sends it to the gesture response control unit 24. Then, the gesture response control unit 24 causes the robot 1 to execute a gesture action corresponding to gesture number 2 shown in Fig. 3A.

[0055] On the other hand, if there is no shortest phoneme string that matches the converted phoneme string (step S1309; N), it is determined whether or not all response times have been read from the response time list (step S1310). If all response times have not been read from the response time list (step S1310; N), the partial analysis unit 23 reads the next response time from the response time list (step S1311) and executes steps S1305 and after again. Then, if all response times have been read from the response time list (step S1310; Y), the process returns to step S1302.

[0056] The predictive response control process has been described above. The control unit 20 performs the following linguistic response control process in parallel with performing the gesture action in the predictive response control process. The linguistic response control process will be described with reference to FIG. 15. The linguistic response control process is started, for example, when the voice acquisition unit 21 of the control unit 20 detects a voice with a voice level exceeding a predetermined threshold, similar to the predictive response control process. When the control unit 20 starts the linguistic response control process, first, the recognition unit 22 recognizes a predetermined target (step S1501). Next, the control unit 20 determines whether or not the conversation between the predetermined target and the robot 1 has ended, similar to step S402 of the conversation recording process (FIG. 4) (step S1502).

[0057] If the conversation has not ended (step S1502; N), similar to step S1303 of the prediction response control process (FIG. 13), it is determined whether or not the predetermined target has started speaking (step S1503), and the process waits until the target starts speaking (step S1503; N). On the other hand, if the conversation has ended (step S1502; Y), the linguistic response control process is terminated.

[0058] When the predetermined target starts speaking (step S1503; Y), the speech analysis unit 25 acquires the target's speech phonemes from the speech acquisition unit 21 (step S1504) and determines whether the target's speech has ended (step S1505). Whether the speech has ended can be determined, for example, by whether the voice level of the voice data acquired by the speech acquisition unit 21 has remained below a predetermined threshold for a predetermined period of time (e.g., 600 ms). While the speech has not ended (step S1505; N), acquisition of the speech phonemes (step S1504) is repeated.

[0059] When the speech of the specified target is completed (step S1505; Y), the speech analysis unit 25 converts the acquired sequence of phonemes into an utterance sentence (step S1506), and refers to the dictionary database stored in the memory unit 30 from the converted utterance sentence, performs syntactic analysis, and obtains the words and syntax contained in the utterance sentence (step S1507).

[0060] Next, the response sentence generation unit 26 generates a response sentence to the utterance of the predetermined target based on the words and syntax of the utterance sentence by referring to the response sentence database stored in the storage unit 30 (step S1508). Then, the language response control unit 27 vocalizes the response sentence from the speaker 4 by voice synthesis (step S1509), and returns to step S1502.

[0061] The above describes the linguistic response control process. Next, an example of the operation of the predictive response control process will be described with reference to Fig. 16. Fig. 16 shows an example of the operation when a phoneme string of a voice extracted according to a response time read from the response time list in Fig. 14 matches the shortest phoneme string in the response gesture database in Fig. 12, and a gesture action corresponding to the matching shortest phoneme string is executed. In this example, the utterance "Good morning" is input by a predetermined target.

[0062] Then, it is assumed that the speech of the target input from when the target starts speaking (time point: T0~) until the first response time (100 ms) in the response time list has elapsed (time point: T1) is extracted (step S1306 in FIG. 13) and the extracted speech is converted into a phoneme string (step S1308 in FIG. 13), which is the word "oha." This converted phoneme string of "oha" matches the shortest phoneme string "oha" of the spoken sentence "Good morning" in the response gesture database in FIG. 12 (step S1309: Y). Therefore, the robot 1 is caused to execute a gesture action of gesture number "2" corresponding to this shortest phoneme string "oha" (step S1312).

[0063] After that, when a certain time (e.g., 600 ms) has elapsed without any voice from the predetermined target (a state in which the voice level of the voice data acquired by the voice acquisition unit 21 is equal to or lower than a predetermined threshold) (step S1505 in FIG. 15; Y), the language response control unit 27 controls the robot 1 to make the robot 1 execute a speech response using language (steps S1506 to S1509). The certain time (e.g., 600 ms) is a time for confirming that the predetermined target has finished speaking and generating a response sentence. In this way, before making the robot 1 execute a speech response, the speech sentence of the predetermined target, i.e., the content of the speech, is predicted using the shortest phoneme string, and the robot 1 is made to execute a gesture motion accordingly, so that the predetermined target can have the feeling that the robot 1 is listening to his / her speech before the speech response of the robot 1 is executed.

[0064] The non-verbal response that the control device 2 causes the robot 1 to perform is not limited to the movement of the head 101 and the arms 103. As a non-verbal response, gesture actions include not only the movement of the head 101 and the arms 103 but also facial expressions, such as opening and closing the eyelids, raising and lowering the eyebrows, and the movement of the eyes, nose, or mouth, or waving the hands or changing the shape of the hands, such as fisting the hands or opening and raising the hands. Another non-verbal response may be a change in the display mode of the eyes of a display type provided in the robot 1.

[0065] The conversation record subject to analytical learning includes at least the most recently recorded speech sentence, but the period during which the conversation record was recorded does not need to be a fixed period. For example, the most recent predetermined period when analytical learning is performed may be any period of conversation record, such as the most recent day, the most recent week, the most recent month, etc. When the period of the conversation record subject to analytical learning is changed for each analytical learning, the conversation record subject to the previous analytical learning and the conversation record subject to the new analytical learning may overlap in part or not at all in the target period.

[0066] As described above, according to this embodiment, by partially matching an utterance of a specific target, the robot can be made to perform non-verbal behavior corresponding to the utterance. Therefore, it is possible to respond without spending time on at least detecting the end of the utterance, recognizing the voice, and generating a response sentence, and it is possible to quickly and appropriately respond to the utterance of the specific target.

[0067] In addition, since the robot 1 returns a predetermined response to a predetermined target using non-verbal behavior, it does not interfere with the conversation of the target (the conversation itself proceeds normally). Therefore, there is a smaller possibility of an incorrect response than when the robot 1 returns a quick response to an utterance with a response sentence. In addition, even if the non-verbal behavior (gestures, etc.) of the robot 1 is not appropriate as a response to a sentence spoken by a predetermined target, it does not have a significant effect on the conversation.

[0068] In the analysis learning process of the embodiment, for a spoken sentence associated with a gesture movement, if the shortest phoneme sequence that does not match the phoneme sequence from the beginning in a unique phoneme sequence table from which duplicates in the conversation record have been removed is shorter than the length of the spoken sentence, it is recorded as the shortest phoneme sequence that identifies the spoken sentence, allowing a response to be made in a shorter time than if the entire spoken sentence was analyzed and then responded to.

[0069] The control device 2 includes an identification unit 22 that identifies a predetermined target, associates a target ID of the identified predetermined target with a spoken sentence in the speech record, identifies and records the shortest phoneme sequence that identifies the spoken sentence associated with the gesture action for each predetermined target in an analysis and learning process, and creates a response gesture database for each predetermined target. Then, in a conversation with a predetermined target, the predetermined target is identified and the speech sentence is identified by the shortest phoneme sequence using the response gesture database for the predetermined target, so that a gesture response that matches the speech of the predetermined target is possible and a gesture response can be returned quickly.

[0070] For example, in the embodiment, as shown in FIG. 8, when a spoken sentence contains "good morning" and "good night," the shortest phoneme sequence for identifying "good morning" is "oha," and therefore "oha" has been mainly used as an example of the shortest phoneme sequence. However, when the identified target is a person who speaks "hayaenassu" in a dialect instead of the standard "good morning," the shortest phoneme sequence for identifying "hayaenassu" may be "ha," "haya," "hayae," or the like. If the shortest phoneme sequence is "ha," the person can respond more quickly than a person whose shortest phoneme sequence is "oha." In this way, the control device 2 performs a conversation recording process and an analysis and learning process for each identification target, thereby enabling it to perform a predictive response control process that is optimal for the identification target.

[0071] In the embodiment, the voice is cut out at the time of the shortest phoneme string and the phoneme strings are compared, but various modifications are possible. For example, the voice may be cut out sequentially, and the predictive response control process may determine whether or not there is a matching phoneme string based on whether the phoneme string from the beginning of speech matches the shortest phoneme string. Alternatively, the voice from the beginning of speech may be directly compared with a reference voice stored in association with a gesture action without converting it into a phoneme string, and if the voices are similar, a gesture corresponding to the reference voice may be performed.

[0072] In the embodiment, the response gesture database is registered by calculating the average time of the length of the phoneme string, but if the aggregated time varies greatly, it is not necessary to register the response gesture. In the embodiment, the registration is performed without considering the number of occurrences and the variation in the length of the phoneme string, but it is also possible to register the response gesture database when the number of occurrences and the variation in the length of the phoneme string become statistically significant.

[0073] In the embodiment, the learning unit 28 learns the correspondence between the shortest phoneme sequence that identifies a speech sentence for each predetermined target and a gesture action. However, instead of learning a response gesture database for each predetermined target, for example, a response gesture database created before shipping from a factory may be stored in the ROM (or non-volatile RAM) of the robot 1. To create a response gesture database in advance, a voice conversation database that collects various speech sentences or a database that collects voice conversations with the robot (a database that collects a lot of voice conversations) may be prepared, and the analysis learning of FIG. 6 may be performed using this prepared database. In this case, the length of the shortest phoneme sequence registered in the response gesture database may be a time including the average or standard deviation of expected general targets. In addition, the response gesture database may include a general occurrence probability of the speech sentence instead of the number of occurrences. Alternatively, the response gesture database may be stored in a rewritable ROM or non-volatile RAM, and the number of occurrences may be counted from the utterance of the predetermined target while the robot 1 is in operation, and the item of the number of occurrences in the response gesture database may be updated.

[0074] In the embodiment, the content of the utterance is defined as an "utterance sentence", but the content of the utterance is not limited to a sentence. For example, "utterance categories" (here, "purposes of utterance") such as "greetings" ("good morning", "hello", etc.), "thanks" ("thank you", "I'm grateful", etc.), "questions" ("tell me a bit", "can I ask you something", etc.), and "evaluations" ("good", "you understand well", etc.) may be defined as the content of the utterance. In this case, the control device 2 can determine a non-verbal behavior response of the robot 1 for each of these "utterance categories" and learn the correspondence between the shortest phoneme string that identifies the "utterance category" and the non-verbal behavior response. The non-verbal behavior can be, for example, a bow gesture for a "greeting", a hand-waving gesture for a "thank you" or "evaluation", and a head-tilting gesture for a "question".

[0075] The spoken sentences (gesture-corresponding sentences) associated with the gestures are not limited to Japanese, and may be in a foreign language. The control device 2 can perform the analysis learning process and the predictive response control process using a phoneme set for each language. For example, English spoken sentences and gesture movements are associated with each other, and the above-mentioned analysis learning process and predictive response control process can be performed using the English phoneme set.

[0076] In addition, the number of non-verbal behaviors corresponding to one utterance sentence is not limited to one. For example, a gesture action group including a plurality of gesture actions may be associated with one utterance sentence (gesture-corresponding sentence), and when the shortest phoneme string of the utterance sentence is detected, one gesture action may be selected from the corresponding gesture action group and executed by the robot 1. In this case, the selection of the gesture action from the gesture action group may be performed with a fixed probability or randomly, or the gesture action may be selected from the gesture action group according to a change in utterance such as the pitch of the sound when the shortest phoneme string is uttered, the volume of the voice of the utterance, the position of the accent in the phoneme string, and the difference in intonation of the phoneme string. Furthermore, the probability of selecting a gesture action from the gesture action group may be changed according to a change in utterance, and a gesture action may be selected with the changed probability.

[0077] In addition to the above-mentioned changes and modifications, various modifications and derivations are possible. For example, the shape of the robot 1 is not limited to the shape shown in Fig. 1. For example, the robot 1 may have a shape that imitates a pet, such as a dog or a cat. The robot 1 may also have a shape of a stuffed toy or an anime character.

[0078] Alternatively, the robot 1 may be an avatar displayed on the screen of a smartphone or tablet. When the robot 1 is an avatar, the control device 2 can be realized by an application program installed on the smartphone or tablet. The control device 2 acquires an audio signal from a microphone 3 provided in the smartphone or tablet on which the avatar is displayed, causes the avatar displayed on the screen to respond non-verbally, and causes the avatar to speak a response sentence from a speaker 4 provided in the smartphone or tablet.

[0079] The control device 2 may include dedicated hardware such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or various control circuits as the control unit 20 instead of a CPU, and the dedicated hardware may function as each unit shown in Fig. 2. In this case, the functions of each unit may be realized by individual hardware, or the functions of each unit may be realized together by a single piece of hardware. Also, some of the functions of each unit may be realized by dedicated hardware, and the other parts may be realized by software or firmware.

[0080] The programs for realizing the functions of the control device 2 can be applied by being stored in a computer-readable storage medium, such as a flexible disk, a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disc)-ROM, or a memory card. Furthermore, the programs can be superimposed on a carrier wave and applied via a communication medium such as the Internet. For example, the programs can be posted and distributed on a bulletin board (BBS: Bulletin Board System) on a communication network. The programs can then be started and executed under the control of an OS (Operating System) in the same way as other application programs, thereby enabling the above-mentioned processing to be performed.

[0081] Although the preferred embodiment of the present invention has been described above, the present invention is not limited to such a specific embodiment, and the present invention includes the inventions described in the claims and their equivalents. The inventions described in the original claims of this application are listed below.

[0082] (Appendix 1) A control device for a robot capable of responding to an object, comprising: An acquisition means for acquiring a speech of the target; a first analysis means for partially analyzing the speech acquired by the acquisition means while the subject is speaking; a first control means for controlling a first response by the robot using a non-verbal behavior toward the target in accordance with an analysis result by the first analysis means; a second analysis means for analyzing the target speech acquired by the acquisition means in a section longer than the section of the speech analyzed by the first analysis means; a generation means for generating a response sentence to the utterance of the target in response to an analysis result by the second analysis means; A second control means for controlling a second response using language by the robot based on the response sentence generated by the generation means; A robot control device comprising:

[0083] (Appendix 2) The robot control device according to claim 1, characterized in that the length of the phoneme sequence of the utterance analyzed by the first analysis means is set shorter than the length of the phoneme sequence of the utterance analyzed by the second analysis means.

[0084] (Appendix 3) the first response comprises a plurality of first responses; a storage means for storing a plurality of reference phoneme sequences corresponding to the plurality of first responses; the first analysis means determines whether or not a phoneme sequence of a part of the utterance acquired by the acquisition means while the target is speaking matches any one of the plurality of reference phoneme sequences; The control device for a robot according to claim 1 or 2, wherein, when the first analysis means determines that the part of the phoneme sequence matches any one of the plurality of reference phoneme sequences, the first control means causes the robot to execute a first response, among the plurality of first responses, that corresponds to the any one of the plurality of reference phoneme sequences that has been determined to match.

[0085] (Appendix 4) A robot control device as described in Appendix 3, characterized in that each of the multiple reference phoneme sequences is set to the shortest phoneme sequence capable of identifying the content of the target's speech.

[0086] (Appendix 5) The robot is capable of executing the first response and the second response to a plurality of targets different from each other as the target, further comprising an identification means for identifying each of the plurality of objects; The acquisition means acquires, for each of the identified targets, an utterance of the target, an analysis means for analyzing the content of the acquired target utterance; A determination means for determining a first response corresponding to an analysis result by the analysis means from the plurality of first responses; and a learning means for learning, for each of the identified targets, a shortest phoneme sequence capable of identifying the content of the target's utterance corresponding to the analysis result based on the analysis result by the analysis means, by associating the shortest phoneme sequence with the identified first response as the reference phoneme sequence.

[0087] (Appendix 6) The robot has a drivable movable part, The control device for a robot described in any one of appendices 1 to 5, characterized in that the first response is a response by a gesture movement realized by driving the movable part of the robot, and the second response is a response by speaking the response sentence to the target.

[0088] (Appendix 7) A robot configured to be able to execute the first response and the second response to a target, comprising the robot control device described in any one of appendices 1 to 6.

[0089] (Appendix 8) A method for controlling a robot that is responsive to an object, the method comprising: acquiring an utterance of the target; a first analysis step of partially analyzing the utterance acquired by the acquisition step while the subject is speaking; a first control step of controlling a first response by the robot using a non-verbal behavior toward the target in accordance with an analysis result by the first analysis step; a second analysis step of analyzing the target utterance acquired by the acquisition step for a section longer than the section of the utterance analyzed by the first analysis step; a generation step of generating a response sentence to the utterance of the target according to an analysis result in the second analysis step; a second control step of controlling a second response using language by the robot based on the response sentence generated in the generation step; A robot control method comprising:

[0090] (Appendix 9) A computer that controls a robot that can respond to an object, An acquisition means for acquiring a speech of the target; a first analysis means for partially analyzing the speech acquired by the acquisition means while the subject is speaking; a first control means for controlling a first response by the robot using a non-verbal behavior toward the target in accordance with a result of the analysis by the first analysis means; a second analysis means for analyzing the target utterance acquired by the acquisition means over a section longer than the section of the utterance analyzed by the first analysis means; a generation means for generating a response sentence to the utterance of the target in response to an analysis result by the second analysis means; and a second control means for controlling a second response using language by the robot based on the response sentence generated by the generation means; A program to function as a [Explanation of symbols]

[0091] 1...robot, 2...control device, 3...microphone, 4...speaker, 5, 6...joints, 7...gesture actuation unit, 20...control unit, 21...voice acquisition unit, 22...recognition unit, 23...partial analysis unit, 24...gesture response control unit, 25...speech analysis unit, 26...response sentence generation unit, 27...language response control unit, 28...learning unit, 29...identification unit, 30...memory unit, 101...head, 102...torso, 103...arm

Claims

1. A robot capable of responding by voice and by gesture, a first identification means for starting a predetermined identification process in response to a start of an utterance by a speaker in order to identify a category of a content that the speaker is trying to convey through the utterance; a control means for controlling the robot to execute a gesture that is preset in association with the identified category as soon as the category is identified by the identification process, regardless of whether the speaker has finished speaking or not; Equipped with The control means Even when controlling the robot to execute the gesture response before the speaker finishes speaking, the robot is controlled to execute the voice response after waiting for the speaker to finish speaking, thereby controlling the robot to be silent until the speaker finishes speaking, The robot is configured to be controllable to perform different gestures for each category. A robot characterized by:

2. the identification process is a process of identifying the category by performing a forward match search of a predetermined database with a phoneme string, which is a time series of phonemes that are sequentially pronounced as the utterance, each time the phoneme string extends by a predetermined elapsed time due to the pronunciation; 2. The robot according to claim 1 .

3. A second identification means for identifying the speaker is provided, the identification process performs a forward match search for the phoneme string in the database associated with the speaker identified by the second identification means; 3. The robot according to claim 2 .

4. A response method executed by a robot capable of responding by voice and by gesture, comprising: a step of starting a predetermined identification process in response to a start of speech by a speaker in order to identify a category of a content that the speaker intends to convey by the speech; a control step of controlling the robot to execute a gesture that is preset in association with the identified category as soon as the category is identified by the identification process, regardless of whether the speaker has finished speaking or not; having The control step includes: Even when controlling the robot to execute the gesture response before the speaker finishes speaking, the robot is controlled to execute the voice response after waiting for the speaker to finish speaking, thereby controlling the robot to be silent until the speaker finishes speaking, The robot is configured to be controllable to perform different gestures for each category. A response method comprising:

5. A robot computer capable of responding by voice and by gestures, a determination means for starting a predetermined determination process in response to a start of speech by a speaker in order to determine a category of the content that the speaker is trying to convey through the speech; a control means for controlling the robot to execute a gesture that is preset in association with the identified category as soon as the category is identified by the identification process, regardless of whether the speaker has finished speaking or not; Function as a The control means Even when controlling the robot to execute the gesture response before the speaker finishes speaking, the robot is controlled to execute the voice response after waiting for the speaker to finish speaking, thereby controlling the robot to be silent until the speaker finishes speaking, The robot is configured to be controllable to perform different gestures for each category. A program characterized by:

Images