Behavior control system, information presentation system, program, and robot

Abstract

The behavior control system includes: an answer generation unit that generates an answer to a user's question on the basis of a dialogue function that causes the user to dialogue with a robot; an emotion determination unit that determines the emotion of the user to the answer on the basis of an emotion estimation function that estimates the emotion of the user; and a behavior determination unit that determines a behavior of the robot corresponding to the emotion.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a behavior control system, an information presentation system, a program, and a robot. BACKGROUND

[0002] In Patent Literature 1, a technology of determining an appropriate behavior of a robot for a state of a user is disclosed. The prior art of Patent Literature 1 identifies a reaction of the user when the robot performs a specific behavior, and in a case where a behavior of the robot for the identified reaction of the user cannot be determined, the behavior of the robot is updated by receiving information related to a behavior appropriate for the identified state of the user from a server.

[0003] PATENT LITERATURE

[0004] Patent Literature 1: Japanese Patent No. 6053847 SUMMARY

[0005] However, in the prior art, there is room for improvement in causing the robot to perform an appropriate behavior in response to a question from the user.

[0006] According to a first aspect of the present application, there is provided a behavior control system. The behavior control system includes: an answer generation section that generates an answer to a question of a user based on a dialogue function that causes the user to dialogue with a robot; a feeling determination section that determines a feeling of the user with respect to the answer based on a feeling estimation function that estimates a feeling of the user; and a behavior determination section that determines a behavior of the robot corresponding to the feeling.

[0007] The behavior determination section determines a first specific behavior in a case where the feeling indicates a first feeling of the user, and determines a second specific behavior different from the first specific behavior in a case where the feeling indicates a second feeling of the user different from the first feeling.

[0008] The behavior determination section determines, as the first specific behavior, an operation of contacting a hand of the user by causing a hand of the robot to be linked with an arm motion of the user in a case where the first feeling is that the user is happy, and determines, as the second specific behavior, an operation of raising the hand of the robot toward an upper portion of the robot without linking the hand of the robot with the arm motion of the user in a case where the second feeling is that the user is angry.

[0009] According to a second aspect of the present application, there is provided a program. The program causes a computer to function as the above-described behavior control system.

[0010] According to a third aspect of the present application, there is provided an information presentation system. The information presentation system includes: a search section that searches based on text information indicating content of utterance of a user; and a presentation section that presents a result of the search by the search section to the user through at least one of voice output, text display, and image display.

[0011] The information presentation system can further include an order section that, in a case where the utterance of the user with respect to the result of the search presented by the presentation section is content of an order to a store included in the result of the search presented by the presentation section, places an order to the store by generating voice data for placing an order to the store and outputting the voice data.

[0012] According to a fourth aspect of the present application, there is provided a program. The program causes a computer to function as the above-described information presentation system.

[0013] According to a fifth aspect of the present application, there is provided a robot. The robot includes the above-described information presentation system and an output device that outputs at least one of voice, text, and image presented by the above-described presentation section.

[0014] In addition, the above-described disclosure does not recite all the features required for the present disclosure. Furthermore, sub-combinations of these feature groups can also constitute the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a schematic diagram showing one example of a system 1 to which each embodiment relates.

[0016] Figure 2 is a schematic diagram showing a functional structure of a robot 100 to which a first embodiment relates.

[0017] Figure 3 is a schematic diagram showing one example of an operation flow related to an operation of determining a behavior in the robot 100 to which the first embodiment relates.

[0018] Figure 4 is a schematic diagram showing one example of a hardware structure of a computer 1200 to which each embodiment relates.

[0019] Figure 5 shows a functional structure of a robot 100 to which a second embodiment relates.

[0020] Figure 6 shows one example of an operation flow of the robot 100 to which the second embodiment relates.

[0021] REFERENCE NUMERALS

[0022] 1: system; 2: behavior control system; 10a, 10b, 10c, 11a, 11b, 11c, 12a, 12b: user; 20: communication network; 100, 101, 102: robot; 200: sensor section; 201: microphone; 202: depth sensor; 203: camera; 204: distance sensor; 210: sensor module section; 211: voice emotion recognition section; 212: speech understanding section; 213: expression recognition section; 214: face recognition section; 220: storage section; 221: reaction rule; 230: user state recognition section; 231: user behavior recognition section; 232: answer generation section; 233: emotion determination section; 234: behavior determination section; 235: behavior control section; 236: control object; 237: communication processing section; 260: output device; 262: display; 264: speaker; 300: server; 330: search section; 332: presentation section; 334: ordering section; 1200: computer; 1210: main controller; 1212: CPU; 1214: RAM; 1216: graphics controller; 1218: display device; 1220: input / output controller; 1222: communication interface; 1224: storage device; 1226: DVD drive; 1227: DVD-ROM; 1230: ROM; 1240: input / output chip. DETAILED DESCRIPTION

[0023] The present disclosure will be described below by disclosed embodiments, but the following embodiments are not intended to limit the disclosure involved in the claims. In addition, not all combinations of features described in the embodiments are necessary for the technical solution of the present disclosure.

[0024] [First Embodiment]

[0025] Figure 1 An example of a system 1 involved in the present embodiment is schematically shown. The system 1 includes a robot 100, a robot 101, a robot 102, and a server 300. A user 10a, a user 10b, a user 10c, and a user 10d are users of the robot 100. A user 11a, a user 11b, and a user 11c are users of the robot 101. A user 12a and a user 12b are users of the robot 102. In addition, in the description of the present embodiment, the user 10a, the user 10b, the user 10c, and the user 10d are sometimes collectively referred to as the user 10. In addition, the user 11a, the user 11b, and the user 11c are sometimes collectively referred to as the user 11. In addition, the user 12a and the user 12b are sometimes collectively referred to as the user 12. The robot 101 and the robot 102 have substantially the same functions as the robot 100. Therefore, the functions of the robot 100 are mainly used to describe the system 1.

[0026] The robot 100 carries out a conversation with the user 10 or provides the user 10 with an image. At this time, the robot 100 cooperates with the server 300 or the like capable of communicating via the communication network 20 to carry out a conversation with the user 10, provide the user 10 with an image, or the like. For example, the robot 100 not only learns an appropriate conversation by itself but also cooperates with the server 300 to learn so as to more appropriately carry out a conversation with the user 10. In addition, the robot 100 causes the server 300 to record image data or the like of the user 10 taken, requests the server 300 for the image data or the like as needed, and provides the user 10 with the image data or the like.

[0027] In addition, the robot 100 has an emotion value indicating a kind of emotion of itself. For example, the robot 100 has an emotion value indicating an emotion intensity of each of "joy", "anger", "sorrow", "cheerfulness", "happiness", "unhappiness", "relief", "unease", "sadness", "excitement", "worry", "relief", "feeling of fulfillment", "feeling of emptiness", and "ordinary". The robot 100, for example, emits a voice at a faster speed when the user 10 is in a state in which the emotion value of excitement is large. In this way, the robot 100 is able to express its own emotion with a behavior.

[0028] In addition, the robot 100 can also be configured to determine a behavior of the robot 100 corresponding to an emotion of the user 10 by matching an AI (Artificial Intelligence) chat engine with an emotion engine. Specifically, the robot 100 can be configured to accept a question from the user 10, determine an emotion of the user 10 with respect to the question, and determine a behavior of the robot 100 corresponding to the determined emotion.

[0029] More specifically, the robot 100 automatically generates an answer to the question using a chat engine set in advance in a case where the question from the user 10 is received. The chat engine can be interpreted as an algorithm and an operation for automatically conversing using a text. The chat engine is known as disclosed in, for example, Japanese Patent Application Publication No. 2018-081444, and thus a detailed description thereof is omitted.

[0030] Further, the robot 100 matches what kind of emotion of the user 10 the content of the answer corresponds to using an emotion engine set in advance, and carries out a specific operation based on the matched emotion, that is, a specific behavior. The emotion engine has a prescribed emotion generation algorithm. As the emotion generation algorithm, a known technique (see, for example, Japanese Patent No. 6351528) can be applied. For example, the matching of the answer to the question with the emotion of the user 10 can use a model generated in advance using machine learning.

[0031] The specific behavior can include a limb movement and a gesture of the robot 100. Specifically, the specific behavior is a hand-clapping gesture, a fist-pumping gesture, and the like. More specifically, the robot 100 is capable of a hand-clapping gesture as the specific behavior in a case where the emotion of the user 10 is "happy". In addition, the robot 100 is capable of a fist-pumping gesture as the specific behavior in a case where the emotion of the user 10 is "angry". In addition, the specific behavior can include a finger movement of the robot 100, a body movement of the robot 100, a head movement of the robot 100, a change in an expression of the robot 100, and the like in addition to the gesture of the robot 100.

[0032] In this way, the robot 100 has the ability to express the emotion of the user 10 with a gesture by using a mechanical hand and by combining the emotion engine and the like in the chat engine. In addition, the robot 100 is capable of taking a behavior by feeling an atmosphere of an unexpressed situation and the like by matching the answer to the question with the emotion of the user 10.

[0033] In addition, the robot 100 has a function of recognizing a behavior of the user 10. The robot 100 recognizes the behavior of the user 10 by analyzing a face image of the user 10 acquired by a camera function and a voice of the user 10 acquired by a microphone function. The robot 100 determines a behavior of the robot 100 to be executed on the basis of the recognized behavior of the user 10 and the like.

[0034] The robot 100 stores a rule of stipulating a behavior of the robot 100 to be executed on the basis of a behavior of the user 10, and performs various behaviors in accordance with the rule.

[0035] Specifically, the robot 100 has a reaction rule for determining a behavior of the robot 100 on the basis of a behavior of the user 10. For example, in the reaction rule, in a case where the behavior of the user 10 is "laughing", a behavior of "laughing" is stipulated as the behavior of the robot 100. In addition, in the reaction rule, in a case where the behavior of the user 10 is "angry", a behavior of "apologizing" is stipulated as the behavior of the robot 100. In addition, in the reaction rule, in a case where the behavior of the user 10 is "asking a question", a behavior of "answering" is stipulated as the behavior of the robot 100. In the reaction rule, in a case where the behavior of the user 10 is "sad", a behavior of "consoling" is stipulated as the behavior of the robot 100.

[0036] The robot 100 selects, on the basis of the reaction rule, a behavior of "laughing" stipulated by the reaction rule as the behavior of the robot 100 to be executed in a case where the behavior of the user 10 is recognized as "laughing". For example, the robot 100 performs an operation of "laughing" in a case where the behavior of "laughing" is selected, and outputs a voice indicating a laugh. Thus, the robot 100 is capable of laughing together with the user 10.

[0037] In addition, the robot 100 transmits user reaction information indicating that a positive reaction was obtained from the user 10 by the behavior to the server 300. The user reaction information includes, for example, the user behavior of "laughing", the behavior of the robot 100 of "laughing", the reaction of the user 10 being positive, and the attribute of the user 10.

[0038] The server 300 stores the user reaction information received from the robot 100. In addition, the server 300 receives and stores user reaction information from the robot 101 and the robot 102, respectively, in addition to the robot 100. Then, the server 300 analyzes the user reaction information from the robot 100, the robot 101, and the robot 102, and updates the reaction rule.

[0039] The robot 100 receives the updated reaction rule from the server 300 by inquiring the server 300 for the updated reaction rule. The robot 100 combines the updated reaction rule into the reaction rule stored in the robot 100. Thus, the robot 100 can combine the reaction rule obtained by the robot 101 or the robot 102 into the reaction rule of itself.

[0040] Figure 2 is a schematic diagram showing the functional structure of the robot 100. The robot 100 has a sensor section 200, a sensor module section 210, and a storage section 220. In addition, the robot 100 has a user state recognition section 230, a user behavior recognition section 231, a reply generation section 232, an emotion determination section 233, a behavior determination section 234, a behavior control section 235, a control object 236, and a communication processing section 237.

[0041] The control object 236 includes a display device, a speaker, and an LED of an eye section, and a motor that drives an arm, a hand, and a leg, and the like. The posture and the manner of the robot 100 are controlled by controlling the motor that drives the arm, the hand, and the leg, and the like. A part of the emotion of the robot 100 can be expressed by controlling these motors. In addition, by controlling the light emission state of the LED of the eye section of the robot 100, a part of the emotion of the robot 100 can also be expressed. In addition, the posture, the manner, and the expression of the robot 100 are one example of the attitude of the robot 100.

[0042] The sensor section 200 includes a microphone 201, a 3D depth sensor 202, a 2D camera 203, and a distance sensor 204. The microphone 201 continuously detects sound and outputs sound data. In addition, the microphone 201 can be provided at the head of the robot 100, and have a function of making binaural recording. The 3D depth sensor 202 continuously irradiates an infrared pattern, and detects the outline of an object by analyzing the infrared pattern from an infrared image continuously captured by an infrared camera. The 2D camera 203 is one example of an image sensor. The 2D camera 203 captures with visible light, and generates video information of the visible light. The distance sensor 204 detects the distance to an object, for example, by irradiating laser light or ultrasonic waves. In addition, the sensor section 200 can include a clock, a gyro sensor, a touch sensor, a sensor for motor feedback, and the like, in addition to these.

[0043] In addition, in Figure 2 Among the constituent elements of the robot 100 illustrated in FIG. 1, the constituent elements other than the control target 236 and the sensor section 200 are one example of the constituent elements of the behavior control system 2 possessed by the robot 100. The behavior control system 2 of the robot 100 takes the control target 236 as an object of control.

[0044] The storage section 220 includes a reaction rule 221. At least a part of the storage section 220 is realized by a storage medium such as a memory. The storage section 220 can also include a person DB that stores the face image of the user 10, the attribute information of the user 10, and the like. In addition, the storage section 220 can include a storage section that stores a program, a database, and the like. Figure 2 Among the constituent elements of the robot 100 illustrated in FIG. 1, the functions of the constituent elements other than the control target 236, the sensor section 200, and the storage section 220 can be realized by a CPU (Central Processing Unit) based on program execution. For example, by a basic software (OS: Operating System) and a program running on the OS, the functions of these constituent elements can be realized by execution of the CPU.

[0045] The sensor module section 210 includes a sound emotion recognition section 211, a speech understanding section 212, an expression recognition section 213, and a face recognition section 214. To the sensor module section 210, information detected by the sensor section 200 is input. The sensor module section 210 analyzes the information detected by the sensor section 200, and outputs the analysis result to the user state recognition section 230.

[0046] The voice emotion recognition section 211 of the sensor module section 210 analyzes the voice of the user 10 detected by the microphone 201, and recognizes the emotion of the user 10. For example, the voice emotion recognition section 211 extracts a feature amount such as a frequency component of the voice, and recognizes the emotion of the user 10 on the basis of the extracted feature amount. The utterance understanding section 212 analyzes the voice of the user 10 detected by the microphone 201, and outputs text information indicating the content of the utterance of the user 10.

[0047] The expression recognition section 213 recognizes the expression of the user 10 and the emotion of the user 10 on the basis of the image of the user 10 captured by the 2D camera 203. For example, the expression recognition section 213 recognizes the expression and the emotion of the user 10 on the basis of the shape, positional relationship, and the like of the eyes and the mouth.

[0048] The face recognition section 214 recognizes the face of the user 10. For example, the face recognition section 214 recognizes the user 10 by matching a face image stored in a person DB (not shown in the drawing) and a face image of the user 10 captured by the 2D camera 203.

[0049] The user state recognition section 230 recognizes the state of the user 10 on the basis of the information analyzed by the sensor module section 210. For example, the user state recognition section 230 mainly performs a process related to perception using the analysis result of the sensor module section 210. For example, the user state recognition section 230 generates perception information such as "Dad is alone.", "The probability that Dad is not smiling is 90%.", and the like. A process of understanding the meaning of the generated perception information is performed. For example, the user state recognition section 230 generates meaning information such as "Dad is alone and looks very lonely.", and the like.

[0050] The user behavior recognition section 231 recognizes the behavior of the user 10 on the basis of the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the user state recognition section 230. For example, the user behavior recognition section 231 inputs the information analyzed by the sensor module section 210 and the recognized state of the user 10 to a neural network learned in advance, obtains the respective probabilities of a plurality of behavior classifications (for example, "laugh", "anger", "question", and "sadness") set in advance, and recognizes the behavior classification with the highest probability as the behavior of the user 10.

[0051] The answer generation section 232 generates an answer to the question of the user 10 on the basis of a dialogue function that causes the user 10 and the robot 100 to dialogue in a case where the behavior of the user is recognized as "question" by the user behavior recognition section 231. The dialogue function can be interpreted as the above-described chat engine. The answer generation section 232 outputs information indicating the answer to the question to the emotion determination section 233.

[0052] The emotion determination section 233 determines the emotion of the user 10 with respect to the answer based on the emotion estimation function that estimates the emotion of the user 10, from the information indicating the answer to the question input from the answer generation section 232. The emotion estimation function can be interpreted as the emotion engine described above.

[0053] The emotion determination section 233 outputs information indicating that the emotion is the first emotion to the behavior determination section 234 in a case where the emotion of the user 10 indicates the first emotion. The first emotion can be interpreted as "happy". In addition, the emotion determination section 233 outputs information indicating that the emotion is the second emotion to the behavior determination section 234 in a case where the emotion of the user 10 indicates the second emotion different from the first emotion. The second emotion can be interpreted as "angry".

[0054] In addition, the first emotion and the second emotion are not limited to the above examples, and can be, for example, "joy", "comfort", "unhappiness", "relief", "unease", "sadness", "excitement", "worry", "relief", "satisfaction", "emptiness", and the like.

[0055] The behavior determination section 234 determines the behavior of the robot 100 based on the behavior of the user 10 recognized by the user behavior recognition section 231, the reaction rule 221, and the emotion determined by the emotion determination section 233.

[0056] Specifically, the behavior determination section 234 determines a first specific behavior in a case where the emotion indicates the first emotion of the user 10, and determines a second specific behavior different from the first specific behavior in a case where the emotion indicates the second emotion different from the first emotion of the user 10. In addition, the number of emotions and the number of specific behaviors are not limited to two, and can be three or more. The behavior determination section 234 can determine the specific behavior from among a plurality of emotions. The behavior determination section 234 outputs information indicating the determined specific behavior to the behavior control section 235.

[0057] More specifically, the behavior determination section 234 determines an operation in which the hand of the robot 100 is brought into contact with the hand of the user 10 by causing the hand of the robot 100 to be linked with the arm movement of the user 10 (handshaking) as the first specific behavior in a case where the first emotion is "happy" of the user 10. In addition, the behavior determination section 234 determines an operation in which the hand of the robot 100 is raised to the upper portion of the robot 100 without causing the hand of the robot 100 to be linked with the arm movement of the user 10 as the second specific behavior in a case where the second emotion is "angry" of the user 10.

[0058] The behavior control section 235 controls the control object 236 based on the behavior determined by the behavior determination section 234.

[0059] Specifically, the behavior control section 235 generates a drive command of a motor that drives the arm, the hand, and the like, to keep the palm side of the hand facing the user 10 while the fingers of the hand are extended in a state where the fingers of the hand are spread, in a case where the behavior determination section 234 determines the first specific behavior. In addition, the behavior control section 235 generates a drive command of a motor that drives the arm, the hand, and the like, to cause the arm to operate to raise the fist in a state where the fingers of the hand are closed to form a fist, in a case where the behavior determination section 234 determines the second specific behavior.

[0060] In addition, the behavior control section 235 causes a speaker included in the control object 236 to output a sound in a case where the behavior determination section 234 determines a behavior including utterance. At this time, the behavior control section 235 can determine a speed of utterance of the sound on the basis of the emotional value of the robot 100.

[0061] In addition, the behavior control section 235 causes the speaker to output a sound indicating pride (for example, "It is my specialty to answer such a question") in a case where the behavior determination section 234 determines a behavior of "feeling happy about being asked a question by the user". In addition, the behavior control section 235 expresses a posture and a manner of the robot 100 indicating pride (for example, an operation of slowly raising one hand) by driving a motor that drives the arm, the hand, the feet, and the like.

[0062] The behavior control section 235 can recognize a change in the emotion of the user 10 with respect to the behavior determined by the behavior determination section 234. For example, the behavior control section 235 can recognize the change in the emotion on the basis of the voice or the expression of the user 10. In addition, the behavior control section 235 can recognize the change in the emotion of the user 10 on the basis of a shock detected by a touch sensor included in the sensor section 200. The behavior control section 235 can recognize that the emotion of the user 10 is deteriorating in a case where a shock is detected by the touch sensor included in the sensor section 200, or can recognize that the emotion of the user 10 is improving in a case where it is determined from the detection result of the touch sensor included in the sensor section 200 that the reaction of the user 10 is laughing or is happy, and the like. Information indicating the reaction of the user 10 is output to the communication processing section 237.

[0063] The communication processing section 237 undertakes communication with the server 300 (refer to Figure 1 ). As described above, the communication processing section 237 transmits the user reaction information to the server 300. In addition, the communication processing section 237 receives an updated reaction rule from the server 300. If the communication processing section 237 receives the updated reaction rule from the server 300, the reaction rule 221 is updated.

[0064] The server 300 performs communication between the robot 100, the robot 101, and the robot 102 and the server 300, receives the user reaction information transmitted from the robot 100, and updates the reaction rule based on the reaction rule including the behavior for which a positive reaction is obtained.

[0065] Figure 3 is a diagram illustrating one example of an operation flow related to an operation of determining a behavior in the robot 100. The operation flow illustrated is repeatedly executed Figure 3 This time, it is assumed that the information analyzed by the sensor module section 210 is input. In addition, "S" in the operation flow indicates a step of execution.

[0066] First, in step S100, the user state recognition section 230 recognizes the state of the user 10 based on the information analyzed by the sensor module section 210.

[0067] In step S101, the user behavior recognition section 231 recognizes the behavior classification of the user 10 based on the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the user state recognition section 230.

[0068] In step S102, it is determined whether the behavior classification of the user 10 recognized in the above step S101 is "questioning". In the case where the behavior classification of the user 10 recognized in step S101 is "questioning", the processing proceeds to step S104. On the other hand, in the case where the behavior classification of the user 10 recognized in the above step S101 is not "questioning", the processing proceeds to step S103.

[0069] In step S104, the answer generation section 232 generates an answer to the question of the user 10 based on a conversation function for the user 10 to converse with the robot 100.

[0070] In step S105, the emotion determination section 233 determines the emotion of the user 10 to the answer based on the information from the answer generation section 232 and an emotion estimation function. In the case where the emotion of the user 10 indicates a first emotion, the processing proceeds to step S106. On the other hand, in the case where the emotion of the user 10 indicates a second emotion, the processing proceeds to step S107.

[0071] In step S106, the behavior determination section 234 determines a first specific behavior based on the information indicating the first emotion determined by the emotion determination section 233. The behavior determination section 234 determines a patting operation as the first specific behavior in the case where the first emotion is "pleasure" of the user 10.

[0072] In step S107, the behavior determination section 234 determines the second specific behavior on the basis of the information indicating the second emotion determined by the emotion determination section 233. The behavior determination section 234 determines the operation of raising and clenching a fist as the second specific behavior in a case where the second emotion is "angry" of the user 10.

[0073] In step S108, the behavior control section 235 controls the control target 236 on the basis of the behavior determined by the behavior determination section 234.

[0074] As described above, the behavior control system 2 of the present disclosure is configured to generate an answer to a question of the user 10, determine an emotion of the user 10 with respect to the answer, and determine a behavior of the robot 100 corresponding to the emotion. With this configuration, it is possible to cause the robot 100 to perform an appropriate behavior with respect to a question from the user 10, and thus it is possible to impart the robot 100 with the ability to express the emotion of the user 10 in a gesture or the like.

[0075] In addition, the robot 100 is one example of an electronic device provided with the behavior control system 2. The application target of the behavior control system 2 is not limited to the robot 100, and the behavior control system 2 can be applied to various electronic devices. Furthermore, the functions of the server 300 can be implemented by one or a plurality of computers. At least a part of the functions of the server 300 can be implemented by a virtual machine. In addition, at least a part of the functions of the server 300 can be implemented in the cloud.

[0076] Figure 4 is a schematic diagram showing one example of a hardware structure of the computer 1200 that functions as the robot 100 and the server 300. A program installed in the computer 1200 can cause the computer 1200 to function as one or a plurality of "sections" of the apparatuses related to the present embodiment, or cause the computer 1200 to execute operations or one or a plurality of "sections" associated with the apparatuses related to the present embodiment, and / or cause the computer 1200 to execute processes or stages of processes related to the present embodiment. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described in the present specification.

[0077] The computer 1200 in the present embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are connected to each other through a host controller 1210. The computer 1200 further includes input output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input output controller 1220. The DVD drive 1226 can be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 can be a hard disk drive, a solid state drive, or the like. The computer 1200 further includes input output units such as a ROM 1230 and a keyboard, which are connected to the input output controller 1220 via an input output chip 1240.

[0078] The CPU 1212 operates in accordance with programs stored in the ROM 1230 and the RAM 1214, and thereby controls the units. The graphics controller 1216 acquires image data generated by the CPU 1212 in a frame buffer or the like provided in the RAM 1214 or in itself, and causes the image data to be displayed on a display device 1218.

[0079] The communication interface 1222 communicates with other electronic devices through a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive 1226 reads programs or data from a DVD-ROM 1227 or the like, and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card, and / or writes programs and data to the IC card.

[0080] The ROM 1230 stores a boot program or the like executed by the computer 1200 at activation and / or a program dependent on hardware of the computer 1200. The input output chip 1240 can also connect various input output units to the input output controller 1220 through a USB port, a parallel port, a serial port, a keyboard port, a mouse port, or the like.

[0081] The programs are provided from a computer readable storage medium such as the DVD-ROM 1227 or an IC card. The programs are read from the computer readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230 as examples of the computer readable storage medium, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and generates cooperation between the programs and the above-described various types of hardware resources. An apparatus or a method can be constituted by implementing operations or processing of information based on use of the computer 1200.

[0082] For example, in a case where communication is performed between the computer 1200 and an external device, the CPU 1212 can execute a communication program loaded in the RAM 1214, and instruct the communication interface 1222 to perform a communication process based on a process described in the communication program. The communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as the RAM 1214, the storage 1224, the DVD-ROM 1227, or an IC card under the control of the CPU 1212, transmits the read transmission data to a network, or writes reception data received from the network to a reception buffer area provided on the recording medium, and the like.

[0083] In addition, the CPU 1212 can read all or a necessary part of a file or a database stored in the storage 1224, the DVD drive 1226 (DVD-ROM 1227), an external recording medium such as an IC card, in the RAM 1214, and perform various types of processes on data on the RAM 1214. Then, the CPU 1212 can write back the processed data to the external recording medium.

[0084] Various types of information such as programs, data, tables, and databases can be stored in a recording medium and receive information processing. The CPU 1212 can perform various types of processes on data read from the RAM 1214, the processes including various types of operations, information processing, condition judgment, condition branching, unconditional branching, search / replacement of information, and the like, which are described anywhere in the present disclosure, which are specified by an instruction sequence of a program, and write back the results to the RAM 1214. In addition, the CPU 1212 can search for information in a file, a database, or the like in the recording medium. For example, if a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 can search for an entry that coincides with a condition that specifies an attribute value of the first attribute from the plurality of entries, read an attribute value of the second attribute stored in the entry, and thereby acquire an attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0085] The program or the software module described in the above description can be stored in a computer-readable storage medium in the computer 1200 or a computer near the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, and thereby a program can be provided to the computer 1200 via a network.

[0086] In the flowcharts and block diagrams of this embodiment, boxes may represent stages of a process for performing an operation or "parts" of a device that performs the operation. Specific stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. Dedicated circuitry may include digital and / or analog hardware circuitry and may include integrated circuits (ICs) and / or discrete circuitry. Programmable circuitry may include reconfigurable hardware circuitry including logical products, logical sums, XOR, negated logical products, negated logical sums, and other logical operations, flip-flops, registers, and storage elements, such as field-programmable gate arrays (FPGAs) and programmable logic arrays (PLAs).

[0087] Computer-readable storage media can include any tangible device capable of storing instructions executable by a suitable device, resulting in a computer-readable storage medium having instructions stored therein comprising instructions executable to create elements for performing operations specified in a flowchart or block diagram. Examples of computer-readable storage media include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media include floppy disks, magnetic disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital multipurpose disc (DVD), Blu-ray disc, memory sticks, integrated circuit cards, etc.

[0088] Computer-readable instructions may include any combination of source code or object code described in one or more programming languages, including assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or object-oriented programming languages ​​such as Smalltalk, JAVA (registered trademark), C++, and conventional programming languages ​​such as the "C" programming language or similar programming languages.

[0089] Computer-readable instructions may be provided via a local area network (LAN), a wide area network (WAN), or the Internet to a processor or programmable circuit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so as to cause the processor or programmable circuit of the general-purpose computer, special-purpose computer, or other programmable data processing device to execute the computer-readable instructions, which generate elements for performing operations specified in a flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0090] [Second Implementation]

[0091] Next, a second embodiment of this disclosure will be described. Furthermore, the same symbols will be used to denote the same parts as in the first embodiment, and descriptions will be omitted.

[0092] Figure 5 The functional structure of the robot 100 according to this embodiment is shown in a simplified manner. The robot 100 includes a sensor unit 200, a sensor module unit 210, a retrieval unit 330, a display unit 332, an ordering unit 334, and an output device 260.

[0093] The output device 260 includes a display 262 for displaying text and images, and a speaker 264 for outputting sound.

[0094] In addition, Figure 5 The components of the robot 100 shown, excluding the sensor unit 200 and the output device 260, are an example of the components of the information display system possessed by the robot 100. Furthermore, in Figure 5 Of the components of the robot 100 shown, the functions of the components other than the sensor unit 200 and the output device 260 can be implemented by the CPU based on a program. For example, the functions of these components can be implemented by the CPU through the basic software (OS) and the program running on the OS.

[0095] The sensor module 210 includes a voice emotion recognition unit 211, a speech understanding unit 212, an expression recognition unit 213, and a face recognition unit 214. Information detected by the sensor unit 200 is input to the sensor module 210. The sensor module 210 analyzes the information detected by the sensor unit 200 and outputs the analysis results to the retrieval unit 330 and the ordering unit 334.

[0096] The search section 330 searches based on the character information indicating the speech content of the user 10 output from the speech understanding section 212 in the case where the character information is the content of the search information search. The search section 330 can be a dialog-type AI system, can search from the Internet using a search algorithm prepared in advance, or can search using a publicly known search engine. As examples of the speech content of the user 10 in this case, "please tell me a good shaved ice shop in Kyoto" and "please tell me a shop where the product A is cheap" and the like can be given. In the search result of the search section 330, an image corresponding to the character information is included in addition to the character information. For example, in the case where the character information indicates a shop that serves shaved ice, an image corresponding to the shop, an image of shaved ice, a ranking of recommended shaved ice shops, and the like are included in the search result.

[0097] The presentation section 332 presents the text and the image to the user 10 by outputting the search result of the search section 330 to the display 262. In addition, the presentation section 332 can also generate sound data corresponding to the character information included in the search result of the search section 330 and output the sound data to the speaker 264, thereby presenting the search result to the user 10 by sound output. Further, the presentation section 332 can also not directly display the search result, but display recommended shaved ice in accordance with a plurality of images, in combination with the preference of the user 10, or present a recommended shop from a ranking of shaved ice shops.

[0098] In the case where the speech content of the user 10 output from the speech understanding section 212 is content that instructs to order from a shop included in the search result presented by the presentation section 332 by analyzing the sound of the speech of the user 10 with respect to the search result presented by the presentation section 332, the ordering section 334 performs the following processing. That is, in this case, the ordering section 334 orders from the shop by generating sound data for ordering from the shop and outputting the sound data to the speaker 264. At this time, the ordering section 334 also performs processing accompanying the ordering, such as making a phone call to the shop, having a conversation with a clerk who answers the phone, and the like. As examples of the speech content of the user 10 in this case, "I would like to make a reservation at this shaved ice shop" and "I would like to order product A at shop B" and the like with respect to the search result presented by the robot can be given. That is, the ordering described here includes a reservation of a shop, a reservation of a product, an order of a product, and the like.

[0099] Figure 6 One example of an operation flow related to the operation of presenting information in the robot 100 is schematically shown. At this time, it is assumed that information input is analyzed by the sensor module section 210. In addition, "S" in the operation flow indicates a step of execution.

[0100] First, in step S200, the search section 330 searches based on the character information indicating the content of the utterance of the user 10 output from the utterance understanding section 212 in a case where the character information is the content of the search information search.

[0101] In step S202, the presentation section 332 presents the text and the image to the user 10 by outputting the search result of the search section 330 to the display 262.

[0102] In step S204, in a case where the content of the utterance of the user 10 with respect to the search result presented by the presentation section 332 is the content of the order to the shop included in the search result, the order section 334 generates sound data for ordering to the shop, outputs the sound data to the speaker 264, and thereby orders to the shop.

[0103] As described above, as the search result, the robot 100 presents not only the text but also the image to the user. In addition, in a case where the user wishes to order, the order is also made. Therefore, as compared with a case where only the text is presented to the user, the interactive AI system can effectively assist the user.

[0104] In addition, the robot 100 according to the second embodiment is one example of an electronic device provided with an information presentation system. The application object of the information presentation system is not limited to the robot 100, and the information presentation system can be applied to various electronic devices. Furthermore, the functions of the server 300 can be implemented by one or a plurality of computers. At least a part of the functions of the server 300 can be implemented by a virtual machine. In addition, at least a part of the functions of the server 300 can be implemented in the cloud.

[0105] The above describes the present disclosure using the embodiments, but the technical scope of the present disclosure is not limited to the range described in the above embodiments. It is obvious for those skilled in the art that various changes or improvements can be made to the above embodiments. It is clear from the description of the claims that such changes or improvements can also be included in the technical scope of the present disclosure.

[0106] Note that the order of execution of each process of the operations, sequences, steps, and stages of the apparatuses, systems, programs, and methods shown in the claims, the description, and the drawings can be implemented in any order as long as it is not specifically indicated as "before," "prior to," or the like or the output of the previous process is used for the next process. For the operation flow in the claims, the description, and the drawings, even if it is described using "first," "next," or the like for convenience, it does not mean that it must be implemented in that order.

[0107] In addition, regarding the above description, the following supplementary notes are also disclosed.

[0108] (Supplementary Note 1)

[0109] A behavior control system including: an answer generation section that generates an answer to a question of a user based on a dialogue function that dialogues with the user with a robot; a sentiment determination section that determines a sentiment of the user with respect to the answer based on a sentiment estimation function that estimates the sentiment of the user; and a behavior determination section that determines a behavior of the robot corresponding to the sentiment.

[0110] (Note 2)

[0111] The behavior control system according to Note 1, in which the behavior determination section determines a specific behavior from among a plurality of sentiments.

[0112] (Note 3)

[0113] The behavior control system according to Note 1, in which the behavior determination section determines a first specific behavior when the sentiment indicates a first sentiment of the user, and determines a second specific behavior different from the first specific behavior when the sentiment indicates a second sentiment of the user different from the first sentiment.

[0114] (Note 4)

[0115] The behavior control system according to Note 3, in which the behavior determination section determines, as the first specific behavior, an operation that brings a hand of the robot into contact with a hand of the user by linking the hand of the robot with an arm motion of the user when the first sentiment is that the user is happy, and determines, as the second specific behavior, an operation that does not link the hand of the robot with the arm motion of the user but causes the hand to be raised up to an upper portion of the robot when the second sentiment is that the user is angry.

[0116] (Note 5)

[0117] A program for causing a computer to function as the behavior control system according to any one of Notes 1 to 4.

[0118] The disclosure of Japanese Patent Application No. 2023-047332, filed March 23, 2023, and the disclosure of Japanese Patent Application No. 2023-061768, filed April 5, 2023, are incorporated herein by reference in their entirety.

Claims

1. A behavior control system comprising: a reply generation section that generates a reply to a question of a user based on a dialogue function that dialogues with the user and a robot; an emotion determination section that determines an emotion of the user with respect to the reply based on an emotion estimation function that estimates the emotion of the user; and a behavior determination section that determines a behavior of the robot corresponding to the emotion.

2. The behavior control system according to claim 1, wherein the behavior determination section determines a specific behavior from among a plurality of emotions.

3. The behavior control system according to claim 1, wherein the behavior determination section determines a first specific behavior when the emotion indicates a first emotion of the user, the behavior determination section determines a second specific behavior different from the first specific behavior when the emotion indicates a second emotion of the user different from the first emotion.

4. The behavior control system according to claim 3, wherein the behavior determination section determines, as the first specific behavior, an operation that brings a hand of the robot into contact with a hand of the user by linking the hand of the robot with an arm motion of the user when the first emotion is that the user is happy, the behavior determination section determines, as the second specific behavior, an operation that does not link the hand of the robot with the arm motion of the user but raises the hand toward an upper portion of the robot when the second emotion is that the user is angry.

5. A program for causing a computer to function as the behavior control system according to any one of claims 1 to 4.

6. An information presentation system comprising: a search section that searches based on character information indicating a content of a speech of a user; and a presentation section that presents a result of the search of the search section to the user by at least one of voice output, text display, and image display.

7. The information presentation system according to claim 6, wherein the information presentation system further includes an ordering section that, when a speech of the user with respect to the result of the search presented by the presentation section is a content of ordering to a store included in the result of the search presented by the presentation section, orders to the store by generating voice data for ordering to the store and outputting the voice data.

8. A program for causing a computer to function as the information presentation system according to claim 6 or 7.

9. A robot comprising: the information presentation system according to claim 6 or 7; and an output device that outputs at least one of voice, text, and image presented by the presentation section. ​ ​ ​

Images