Behavior control system, control system, and information processing system

Abstract

The behavior control system includes: an emotion determination unit that determines an emotion of a user or an emotion of a robot; and a behavior determination unit that generates behavior content of the robot for the behavior of the user and the emotion of the user or the emotion of the robot on the basis of a dialogue function that causes the user to dialogue with the robot, and determines the behavior of the robot corresponding to the behavior content. The behavior determination unit is provided with: an image acquisition unit capable of capturing an image of an athletic space in which a specific athletic can be implemented; and a competitor emotion analysis unit that analyzes the emotions of a plurality of competitors that are performing competitions in the competitive space captured by the image acquisition unit, and determines the behavior of the robot on the basis of the analysis results of the competitor emotion analysis unit.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a behavior control system, a control system, and an information processing system. BACKGROUND

[0002] In Patent Literature 1, a technology of determining a robot appropriate behavior for a user is disclosed. For the related art of Patent Literature 1, a user reaction is recognized when a robot performs a determined behavior, and in a case where a behavior of the robot cannot be determined for the recognized user reaction, the behavior of the robot is updated by receiving information related to a behavior appropriate for the recognized user state from a server.

[0003] In Patent Literature 2, a method is disclosed, which is a character chatbot control method executed by at least one processor, including: a step of receiving a user utterance; a step of adding the user utterance to a prompt word containing an instruction sentence associated with an explanation of a chatbot character; a step of encoding the prompt word; and a step of inputting the encoded prompt word into a language model and generating a chatbot utterance in response to the user utterance.

[0004] In Patent Literature 3, an emotion determination system that determines an emotion of a robot is described.

[0005] Prior Art Documents Patent Literature Patent Literature 1: Japanese Patent No. 6053847 Patent Literature 2: Japanese Patent Application Laid-Open No. 2022-180282 Patent Literature 3: Japanese Patent Application Laid-Open No. 2017-199319 However, in the related art, there is room for improvement in making a robot perform an appropriate behavior for a user behavior.

[0006] In addition, in the related art, with respect to a one-on-one meeting conducted by two participants, there is room for improvement in efficiently generating presentation content for the meeting.

[0007] In addition, in an earthquake rapid report, a studio of a television station is only given information such as intensity, magnitude, and focal depth. Therefore, an announcer can only broadcast to viewers: "Please be careful of a tsunami. Please do not approach a cliff or the like. Repeat" and the like predetermined sentences, and it is difficult for the viewers to take measures against the earthquake.

[0008] In addition, in the related art, sometimes appropriate information cannot be output in response to an event that has occurred. SUMMARY

[0009] According to a first aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that generates behavior content of the robot with respect to a behavior of the user and the emotion of the user or the emotion of the robot based on a dialogue function that causes the user to dialogue with the robot, determines a behavior of the robot corresponding to the behavior content, the behavior determination section including: an image acquisition section that can photograph a competition space in which a specific competition can be implemented; and a competitor emotion analysis section that analyzes emotions of a plurality of competitors who are implementing a competition in the competition space photographed by the image acquisition section, determines the behavior of the robot based on a result of the analysis by the competitor emotion analysis section.

[0010] According to a second aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that generates behavior content of the robot with respect to a behavior of the user and the emotion of the user or the emotion of the robot based on an article generation model having a dialogue function that causes the user to dialogue with the robot, determines a behavior of the robot corresponding to the behavior content, the behavior determination section including: an image acquisition section that can photograph a competition space in which a specific competition can be implemented; and a feature determination section that determines features of a plurality of competitors who are implementing a competition in the competition space photographed by the image acquisition section, determines the behavior of the robot based on a result of the determination by the feature determination section.

[0011] According to a third aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that generates behavior content of the robot with respect to a behavior of the user and the emotion of the user or the emotion of the robot based on an article generation model having a dialogue function that causes the user to dialogue with the robot, determines a behavior of the robot corresponding to the behavior content, the behavior determination section, when started at a prescribed time, acquires a summary of a previous day by adding a fixed sentence indicating a summary of the history of the previous day to a text representing history data of the previous day and inputting to the article generation model, and speaks the acquired summary content.

[0012] According to a fourth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that generates behavior content of the robot with respect to a behavior of the user and the emotion of the user or the emotion of the robot based on an article generation model having a dialogue function that causes the user to dialogue with the robot, determines a behavior of the robot corresponding to the behavior content, and the behavior determination section, when activated at a prescribed time, acquires a summary of a history of a previous day by adding a fixed sentence for indicating summarizing the history of the previous day to a text representing the history data of the previous day and inputting to the article generation model, acquires an image summarizing the history of the previous day by inputting the acquired summary of the history of the previous day to an image generation model, and displays the acquired image.

[0013] According to a fifth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that generates behavior content of the robot with respect to a behavior of the user and the emotion of the user or the emotion of the robot based on an article generation model having a dialogue function that causes the user to dialogue with the robot, determines a behavior of the robot corresponding to the behavior content, and the behavior determination section, when activated at a prescribed time, determines an emotion of the robot corresponding to a history of a previous day by adding a fixed sentence for asking the emotion that the robot should have to a text representing the history data of the previous day and inputting to the article generation model.

[0014] According to a sixth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that generates behavior content of the robot with respect to a behavior of the user and the emotion of the user or the emotion of the robot based on a dialogue function that causes the user to dialogue with the robot, determines a behavior of the robot corresponding to the behavior content, and the behavior determination section, at a timing when the user gets up, determines an emotion on the basis of a history of the previous day of the user by adding history data including the history of the behavior and the emotion of the previous day of the user to a fixed sentence asking the emotion of the user to the dialogue function.

[0015] According to a seventh aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that generates a behavior content of the robot with respect to a behavior of the user and the emotion of the user or the emotion of the robot based on a conversation function that causes the user to converse with the robot, determines a behavior of the robot corresponding to the behavior content, and the behavior determination section acquires a summary of history data including a history of a behavior and an emotion of the user on a previous day at a timing at which the user gets up, acquires music based on the summary, and plays the music.

[0016] According to a seventh aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that generates a behavior content of the robot with respect to a behavior of the user and the emotion of the user or the emotion of the robot based on a conversation function that causes the user to converse with the robot, determines a behavior of the robot corresponding to the behavior content, and the behavior determination section acquires a summary of history data including a history of a behavior and an emotion of the user on a previous day at a timing at which the user gets up, acquires music based on the summary, and plays the music.

[0017] According to a seventh aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that generates a behavior content of the robot with respect to a behavior of the user and the emotion of the user or the emotion of the robot based on a conversation function that causes the user to converse with the robot, determines a behavior of the robot corresponding to the behavior content, and the behavior determination section acquires a summary of history data including a history of a behavior and an emotion of the user on a previous day at a timing at which the user gets up, acquires music based on the summary, and plays the music.

[0018] According to a seventh aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that generates a behavior content of the robot with respect to a behavior of the user and the emotion of the user or the emotion of the robot based on a conversation function that causes the user to converse with the robot, determines a behavior of the robot corresponding to the behavior content, and the behavior determination section acquires a summary of history data including a history of a behavior and an emotion of the user on a previous day at a timing at which the user gets up, acquires music based on the summary, and plays the music.

[0019] According to a tenth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that generates a behavior content of the robot for a behavior of the user based on a conversation function that causes the user to converse with the robot, determines a behavior of the robot corresponding to the behavior content, and determines the behavior of the robot corresponding to a topic of a conversation being output and other topics output according to an emotion of at least one of the users who are conversing.

[0020] According to an eleventh aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user; an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that determines a behavior of the robot corresponding to the user state and the emotion of the user or the emotion of the robot based on an article generation model having a conversation function that causes the user to converse with the robot, and determines a behavior content of the robot to acquire a musical score of a lyric and a melody corresponding to an environment in which the robot is located based on the article generation model and play music based on the lyric and the melody using a sound synthesis engine.

[0021] According to a twelfth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user; an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that determines a behavior of the robot corresponding to the user state and the emotion of the user or the emotion of the robot based on an article generation model having a conversation function that causes the user to converse with the robot, and generates a life improvement application that proposes improvement of life based on a conversation of the user with the robot. Here, the robot includes a device that performs a physical operation, a device that outputs an image or a sound without performing a physical operation, and an agent that operates on software.

[0022] According to a thirteenth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user; an emotion determination section that determines an emotion of a user or an emotion of an electronic machine; and a behavior determination section that determines a behavior of the electronic machine corresponding to the user state and the emotion of the user or a behavior of the electronic machine corresponding to the user state and the emotion of the electronic machine based on an article generation model having a conversation function that causes the user to converse with the electronic machine, and performs diet management based on the user state.

[0023] According to a fourteenth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user; an emotion determination section that determines an emotion of the user or an emotion of an electronic machine; and a behavior determination section that determines a behavior of the electronic machine corresponding to the user state and the emotion of the user or a behavior of the electronic machine corresponding to the user state and the emotion of the electronic machine based on an article generation model that generates an article with a dialogue function that causes the user to dialogue with the electronic machine, the behavior determination section performing diet management based on the user state.

[0024] According to a fifteenth aspect of the present disclosure, there is provided a control system. The control system includes: a processing section that performs specific processing using an article generation model that generates an article from input data; and an output section that controls a behavior of an electronic machine to output a result of the specific processing. The processing section determines whether a condition of presentation content in a meeting performed by a user is satisfied as a predetermined trigger condition, and in a case where the trigger condition is satisfied, acquires and outputs, as a result of the specific processing, a response related to the presentation content in the meeting using an output of the article generation model at least when a mail record matter, a schedule record matter, and a meeting speech matter obtained from a user input for a specific period are used as the input data. The electronic machine can be a robot. Here, the robot includes a device that performs a physical operation, a device that does not perform a physical operation but outputs an image or a sound, and an agent that operates on software.

[0025] According to a sixteenth aspect of the present disclosure, there is provided an information processing system. The information processing system includes: an input section that receives a user input; a processing section that performs specific processing using a generation model that generates a result from input data; and an output section that controls a behavior of an electronic machine to output a result of the specific processing, the processing section acquiring the result of the specific processing using an output of the generation model when a text indicating presentation of earthquake-related information is used as the input data. The generation model can be a generation model based on an article generation result or a generation model based on an input of information such as an image and a sound to generate a result. The electronic machine can be a robot. Here, the robot includes a device that performs a physical operation, a device that does not perform a physical operation but outputs an image or a sound, and an agent that operates on software.

[0026] According to a seventeenth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user and a state of a robot; an emotion determination section that determines an emotion of the user or an emotion of the robot; and a behavior determination section that determines, at a prescribed timing, a behavior of the robot from among a plurality of behaviors of the robot including no behavior, using at least one of the user state, the state of the electronic machine, the emotion of the user, and the emotion of the electronic machine, and a behavior determination model.

[0027] According to a nineteenth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user and a state of an electronic machine; an emotion determination section that determines an emotion of the user or an emotion of the electronic machine; a behavior determination section that determines, at a prescribed timing, any one of a plurality of machine actions including a non-action using at least one of the user state, the state of the electronic machine, the emotion of the user, and the emotion of the electronic machine and a behavior determination model, the machine action including a proposed activity, the behavior determination section determining, in a case where the proposed activity is determined as the behavior of the electronic machine, a behavior of the user who makes the proposal based on event data including an emotion value determined by the emotion determination section and data including the behavior of the user.

[0028] According to a twentieth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user and a state of an electronic machine; an emotion determination section that determines an emotion of the user or an emotion of the electronic machine; a behavior determination section that determines, at a prescribed timing, any one of a plurality of machine actions including a non-action using at least one of the user state, the state of the electronic machine, the emotion of the user, and the emotion of the electronic machine and a behavior determination model, the machine action including facilitating communication with another person, the behavior determination section determining, in a case where the facilitating communication with another person is determined as the behavior of the electronic machine, at least one of a communication partner or a communication method based on event data. Here, the electronic machine can be a robot, the robot including a device that performs a physical operation, a device that outputs an image or a sound without performing a physical operation, and an agent that operates on software.

[0029] According to a twenty-first aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user and a state of an electronic machine; an emotion determination section that determines an emotion of the user or an emotion of the electronic machine; and a behavior determination section that determines, at a prescribed timing, any one of a plurality of machine actions including no action, as a behavior of the electronic machine, using at least one of the user state, the state of the electronic machine, the emotion of the user, and the emotion of the electronic machine, and a behavior determination model, the machine actions including making a suggestion related to a specific competition to the user participating in the specific competition, the behavior determination section including: an image acquisition section that is capable of photographing a competition space in which the specific competition that the user participates in can be implemented; and a competitor analysis section that analyzes emotions of a plurality of competitors who are implementing the specific competition in the competition space photographed by the image acquisition section, in a case where making a suggestion related to the specific competition to the user participating in the specific competition is determined as a behavior of the electronic machine, making the suggestion to the user based on a result of the analysis by the competitor analysis section. Here, the electronic machine includes a device that performs a physical operation like a robot, a device that outputs an image or a sound without performing a physical operation, and an agent that operates on software.

[0030] According to a twenty-second aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user and a state of an electronic machine; an emotion determination section that determines an emotion of the user or an emotion of the electronic machine; and a behavior determination section that determines, at a prescribed timing, any one of a plurality of machine actions including no action, as a behavior of the electronic machine, using at least one of the user state, the state of the electronic machine, the emotion of the user, and the emotion of the electronic machine, and a behavior determination model, the machine actions including making a suggestion related to a specific competition to the user participating in the specific competition, the behavior determination section including: an image acquisition section that is capable of photographing a competition space in which the specific competition that the user participates in can be implemented; and a feature determination section that determines features of a plurality of competitors who are implementing a competition in the competition space photographed by the image acquisition section, in a case where making a suggestion related to the specific competition to the user participating in the specific competition is determined as a behavior of the electronic machine, making the suggestion to the user based on a result of the determination by the feature determination section. Here, the electronic machine includes a device that performs a physical operation like a robot, a device that outputs an image or a sound without performing a physical operation, and an agent that operates on software.

[0031] According to a twenty-third aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user and a state of an electronic machine; an emotion determination section that determines an emotion of the user or an emotion of the electronic machine; a behavior determination section that determines, at a prescribed timing, any one of a plurality of machine actions including a non-action, as a behavior of the robot using at least one of the user state, the state of the electronic machine, the emotion of the user, and the emotion of the electronic machine, and a behavior determination model, the machine action including a first behavior content that sets a correction of the behavior of the user, the behavior determination section spontaneously or periodically detecting the behavior of the user, and in a case where the correction of the behavior of the user is determined as the behavior of the electronic machine based on the detected behavior of the user and specific information stored in advance, the first behavior content is executed. Here, the robot includes a device that performs a physical operation, a device that outputs an image or a sound without performing a physical operation, and an agent that operates on software.

[0032] According to a twenty-fourth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user and a state of an electronic machine; an emotion determination section that determines an emotion of the user or an emotion of the electronic machine; a behavior determination section that determines, at a prescribed timing, any one of a plurality of machine actions including a non-action, as a behavior of the robot using at least one of the user state, the state of the electronic machine, the emotion of the user, and the emotion of the electronic machine, and a behavior determination model, the machine action including a suggestion to the user regarding a home, the behavior determination section, in a case where the suggestion to the user regarding the home is determined as the behavior of the electronic machine, using an article generation model based on data related to a home machine stored in the history data, proposing a recommended dish related to a physical condition, a food material that should be supplemented, or the like. Here, the robot includes a device that performs a physical operation, a device that outputs an image or a sound without performing a physical operation, and an agent that operates on software.

[0033] According to a twenty-fifth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user and a state of an electronic machine; an emotion determination section that determines an emotion of the user or an emotion of the electronic machine; and a behavior determination section that determines, at a prescribed timing, any one of a plurality of machine actions including no action, as a behavior of the electronic machine, using at least one of the user state, the state of the electronic machine, the emotion of the user, and the emotion of the electronic machine, and a behavior determination model, the machine actions including a suggestion to the user regarding a work issue, the behavior determination section determining the suggestion to the user regarding the work issue as the behavior of the electronic machine.

[0034] According to a twenty-sixth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user and a state of an electronic machine; an emotion determination section that determines an emotion of the user or an emotion of the electronic machine; a behavior determination section that determines, at a prescribed timing, any one of a plurality of machine actions including no action, as a behavior of the electronic machine, using at least one of the user state, the state of the electronic machine, the emotion of the user, and the emotion of the electronic machine, and a behavior determination model, the machine actions including a suggestion to the user regarding a work issue, the behavior determination section determining the suggestion to the user regarding the work issue as the behavior of the electronic machine.

[0035] According to a twenty-seventh aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a state recognition section that recognizes a user state including a behavior of a user and a state of an electronic machine; an emotion determination section that determines an emotion of the user or an emotion of the electronic machine; a behavior determination section that determines, at a prescribed timing, any one of a plurality of machine actions including a non-action, as a behavior of the electronic machine, using at least one of the user state, the state of the electronic machine, the emotion of the user, and the emotion of the electronic machine, and a behavior determination model, the machine action including progress support of a meeting to the user in the meeting, the behavior determination section determining, in a case where the meeting becomes a predetermined state, output of the progress support of the meeting to the user in the meeting as the behavior of the electronic machine, and outputting the progress support of the meeting.

[0036] According to a twenty-eighth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a detection section that detects occurrence of a prescribed event; and an output control section that controls a robot provided with an article generation model to output information corresponding to the event detected by the detection section to a user.

[0037] According to a twenty-ninth aspect of the present disclosure, there is provided a behavior control system. The behavior control system includes: a collection section that collects condition information indicating a condition of a user; and an output control section that controls a robot provided with an article generation model to propose, to the user, an outfit corresponding to the condition information collected by the collection section.

[0038] According to one embodiment of the present disclosure, there is provided a control system. The control system can be provided with a diagnosis result acquisition section that acquires a diagnosis result of an image character diagnosis including at least one of a color diagnosis, a bone diagnosis, and a face style diagnosis of a user who has a conversation with an electronic machine. The control system can be provided with a user feature acquisition section that acquires a user feature including at least one of a voice size, a voice tone, and an expression of the user. The control system can be provided with a user desire acquisition section that acquires a user desire including at least one of a profession and a self-image desired by the user. The control system can be provided with a proposal content generation section that generates, based on the diagnosis result, the user feature, and the user desire, a proposal content for the user. The control system can be provided with a control section that causes the electronic machine to output the proposal content to the user. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a diagram that schematically illustrates an example of a system 5 according to a first embodiment.

[0040] Figure 2 FIG. 1 is a diagram schematically showing a functional configuration of a robot 100.

[0041] Figure 3 FIG. 3 is a diagram schematically showing an example of an operation flow of the robot 100.

[0042] Figure 4 FIG. 6 is a diagram schematically showing an example of a hardware configuration of a computer 1200.

[0043] Figure 5 FIG. 10 is a diagram showing an emotion map 400 that maps a plurality of emotions.

[0044] Figure 6 FIG. 14 is a diagram showing an emotion map 900 that maps a plurality of emotions.

[0045] Figure 7 (A) is an appearance view of a stuffed toy involved in other embodiments,(B) is an internal structure view of the stuffed toy. Figure 7

[0046] Figure 8 FIG. 18 is a rear elevation view of the stuffed toy involved in other embodiments.

[0047] Figure 9 FIG. 22 is a diagram schematically showing a functional configuration of a robot 100 involved in a second embodiment.

[0048] Figure 10 FIG. 25 is a diagram schematically showing an example of a collection processing operation flow performed by the robot 100 involved in the second embodiment.

[0049] Figure 11A FIG. 28 is a diagram schematically showing an example of a response processing operation flow performed by the robot 100 involved in the second embodiment.

[0050] Figure 11B FIG. 31 is a diagram schematically showing an example of an autonomous processing operation flow performed by the robot 100 involved in the second embodiment.

[0051] Figure 12 FIG. 35 is a diagram schematically showing a functional configuration of a stuffed toy 100N involved in a third embodiment.

[0052] Figure 13 FIG. 39 is a diagram schematically showing a functional configuration of an agent system 500 involved in a fourth embodiment.

[0053] Figure 14 FIG. 43 is a diagram showing an example of an operation of the agent system.

[0054] Figure 15is a diagram showing an example of the operation of the agent system.

[0055] Figure 16 is a functional block diagram of the agent system 700 configured using part or all of the functions of the behavior control system.

[0056] Figure 17 is a diagram showing an example of the usage mode of the agent system 700 based on the smart glasses 720.

[0057] Figure 18 is a diagram showing an example of the operation flow of the specific processing performed by the robot 100 according to the eleventh embodiment.

[0058] Figure 19 is a diagram showing the functional structure of the robot 100 according to the twenty-second embodiment.

[0059] Figure 20 is a diagram showing the data structure of the character data 223.

[0060] Figure 21 is a diagram showing an example of the operation flow related to the character setting.

[0061] Figure 22 is a diagram showing an example of the operation flow performed by the robot 100.

[0062] Figure 23 is a diagram showing the functional structure of the event detection unit 2900.

[0063] Figure 24 is a diagram showing an example of the operation flow of the event detection unit 2900.

[0064] Explanation of Reference Signs 5: system; 10, 11, 12: user; 20: communication network; 100, 100N, 101, 102: robot; 200: sensor section; 201: microphone; 202: depth sensor; 203: camera; 204: distance sensor; 210: sensor module section; 211: voice emotion recognition section; 212: speech understanding section; 213: expression recognition section; 214: face recognition section; 220: storage section; 221: reaction rule; 222A: behavior determination model; 2222: history data; 230: state recognition section; 230: state recognition section; 230: state recognition section; 232: emotion determination section; 234: behavior recognition section; 236: behavior determination section; 238: storage control section; 250: behavior control section; 252: control object; 270: association information collection section; 280: communication processing section; 300: server; 500: agent system; 1200: computer; 1210: main controller; 1212: CPU; 1214: RAM; 1216: graphics controller; 1218: display device; 1220: input / output controller; 1222: communication interface; 1224: storage device; 1226: DVD drive; 1227: DVD-ROM; 1230: ROM; 1240: input / output chip. DETAILED DESCRIPTION

[0065] Hereinafter, the present disclosure will be described by embodiments of the present disclosure, but the following embodiments do not limit the technical solutions involved in the claims. In addition, not all of the combinations of features described in the embodiments are necessary for the solution means of the present disclosure. In addition, the above summary of the present disclosure does not list all the necessary features of the present disclosure. In addition, sub-combinations of these feature groups can also be the present disclosure.

[0066] (First Embodiment) Figure 1 An example of a system 5 involved in the first embodiment is schematically shown. The system 5 has a robot 100, a robot 101, a robot 102, and a server 300. A user 10a, a user 10b, a user 10c, and a user 10d are users of the robot 100. A user 11a, a user 11b, and a user 11c are users of the robot 101. A user 12a and a user 12b are users of the robot 102. In addition, in the description of the first embodiment, the user 10a, the user 10b, the user 10c, and the user 10d are sometimes collectively referred to as the user 10. In addition, the user 11a, the user 11b, and the user 11c are sometimes collectively referred to as the user 11. In addition, the user 12a and the user 12b are sometimes collectively referred to as the user 12. The robot 101 and the robot 102 have substantially the same functions as the robot 100. Therefore, the functions of the robot 100 are mainly used to describe the system 5.

[0067] The robot 100 has a conversation with the user 10 or provides the user 10 with an image. At this time, the robot 100 cooperates with the server 300 or the like that can communicate via the communication network 20, and has a conversation with the user 10, provides the user 10 with an image, or the like. For example, the robot 100 not only learns an appropriate conversation by itself but also cooperates with the server 300 and learns to have a more appropriate conversation with the user 10. In addition, the robot 100 causes the server 300 to record image data or the like of the user 10 who is photographed, requests the image data or the like from the server 300 as needed, and provides the user 10 with the image data or the like.

[0068] In addition, the robot 100 has an emotion value that indicates a kind of emotion of itself. For example, the robot 100 has an emotion value that indicates an emotion intensity of each of "joy", "anger", "sorrow", "cheer", "happiness", "unhappiness", "comfort", "discomfort", "melancholy", "excitement", "worry", "steadiness", "feeling of fulfillment", "feeling of emptiness", and "normal". The robot 100, for example, emits a sound at a faster speed when having a conversation with the user 10 in a state in which the emotion value of excitement is large. In this way, the robot 100 can express its own emotion with behavior.

[0069] In addition, the robot 100 can also be configured to determine a behavior of the robot 100 corresponding to the emotion of the user 10 by matching an article generation model and an emotion engine using AI (Artificial Intelligence). Specifically, the robot 100 can be configured to recognize a behavior of the user 10, determine the emotion of the user 10 with respect to the behavior of the user, and determine a behavior of the robot 100 corresponding to the determined emotion.

[0070] More specifically, the robot 100, in a case where a behavior of the user 10 is recognized, automatically generates a behavior content that the robot 100 should take with respect to the behavior of the user 10 using a pre-set article generation model. The article generation model can be interpreted as an algorithm and an operation for automatically processing a dialogue by text. The article generation model is, for example, publicly known knowledge disclosed in Japanese Patent Application Publication No. 2018-081444 or ChatGPT (Internet search <URL: https: / / openai.com / blog / ChatGPT>), and thus a detailed description thereof is omitted. Such an article generation model is constituted by a large-scale language model (LLM).

[0071] In the above, in the first embodiment, by combining the large-scale language model and the emotion engine, it is possible to make the behavior of the robot 100 reflect the emotion of the user 10 or the robot 100 and various language information. That is, according to the first embodiment, by combining the article generation model and the emotion engine, a synergistic effect can be obtained.

[0072] In addition, the robot 100 has a function of recognizing the behavior of the user 10. The robot 100 recognizes the behavior of the user 10 by analyzing a face image of the user 10 acquired with a camera function, a voice of the user 10 acquired with a microphone function. The robot 100 determines the behavior of the robot 100 to be executed on the basis of the recognized behavior of the user 10 and the like.

[0073] The robot 100 stores a rule that specifies the behavior of the robot 100 to be executed on the basis of the emotion of the user 10, the emotion of the robot 100, and the behavior of the user 10, and performs various behaviors in accordance with the rule.

[0074] Specifically, the robot 100 takes a reaction rule for determining the behavior of the robot 100 on the basis of the emotion of the user 10, the emotion of the robot 100, and the behavior of the user 10 as an example of the behavior determination model. In the reaction rule, for example, in the case where the behavior of the user 10 is "laugh", the behavior of "laugh" is specified as the behavior of the robot 100. In addition, in the reaction rule, in the case where the behavior of the user 10 is "anger", the behavior of "apology" is specified as the behavior of the robot 100. In addition, in the reaction rule, in the case where the behavior of the user 10 is "question", the behavior of "answer" is specified as the behavior of the robot 100. In the reaction rule, in the case where the behavior of the user 10 is "sorrow", the behavior of "greeting" is specified as the behavior of the robot 100.

[0075] The robot 100 selects the behavior of "apology" specified in the reaction rule as the behavior of the robot 100 to be executed on the basis of the reaction rule in the case where the behavior of the user 10 is recognized as "anger". For example, the robot 100 performs an "apology" operation and outputs a voice indicating an "apology" language in the case where the behavior of "apology" is selected.

[0076] In addition, in the case where the state of the user 10 satisfies the condition of "looking very lonely alone", the emotion of the robot 100 is specified to be able to perform the content of the emotion change of "worry" and the behavior of "greeting" in the case where the emotion of the robot 100 is "normal" (i.e., "joy" = 0, "anger" = 0, "sorrow" = 0, and "cheerfulness" = 0).

[0077] The robot 100 increases the emotion value of "sorrow" of the robot 100 on the basis of the reaction rule in the case where the current emotion of the robot 100 is "normal" and the user 10 is recognized to be in a lonely state alone. In addition, the robot 100 selects the behavior of "greeting" specified in the reaction rule as the behavior to be executed on the user 10. For example, the robot 100 converts a phrase of "What's wrong?" indicating the worry into a voice of the worry and outputs the voice in the case where the behavior of "greeting" is selected.

[0078] In addition, the robot 100 transmits user reaction information indicating that a positive reaction was obtained from the user 10 by the behavior to the server 300. The user reaction information includes, for example, the user behavior of "angry", the behavior of the robot 100 of "apology", the reaction of the user 10 being positive, and the attribute of the user 10.

[0079] The server 300 stores the user reaction information received from the robot 100. In addition, the server 300 not only receives the user reaction information from the robot 100, but also receives and stores the user reaction information from the robot 101 and the robot 102, respectively. Then, the server 300 analyzes the user reaction information from the robot 100, the robot 101, and the robot 102, and updates the reaction rule.

[0080] The robot 100 receives the updated reaction rule from the server 300 by inquiring the server 300 for the updated reaction rule. The robot 100 incorporates the updated reaction rule into the reaction rule stored in the robot 100. Thereby, the robot 100 can incorporate the reaction rule obtained by the robot 101 or the robot 102, etc. into the reaction rule of itself.

[0081] Figure 2 The functional structure of the robot 100 is shown schematically. The robot 100 has a sensor section 200, a sensor module section 210, a storage section 220, a state recognition section 230, an emotion determination section 232, a behavior recognition section 234, a behavior determination section 236, a storage control section 238, a behavior control section 250, a control object 252, and a communication processing section 280.

[0082] The control object 252 includes a display device, a speaker, and an LED of an eye section, and a motor that drives an arm, a hand, a foot, and the like. The posture or the manner of the robot 100 is controlled by controlling the motors of the arm, the hand, the foot, and the like. A part of the emotion of the robot 100 can be expressed by controlling these motors. In addition, by controlling the light emission state of the LED of the eye section of the robot 100, the expression of the robot 100 can also be expressed. In addition, the posture, the manner, and the expression of the robot 100 are examples of the attitude of the robot 100.

[0083] The sensor section 200 includes a microphone 201, a 3D depth sensor 202, a 2D camera 203, and a distance sensor 204. The microphone 201 continuously detects sound and outputs sound data. In addition, the microphone 201 can be provided at the head of the robot 100, and have a function of making binaural recording. The 3D depth sensor 202 continuously irradiates an infrared pattern, and detects the outline of an object by analyzing the infrared pattern based on an infrared image continuously captured with an infrared camera. The 2D camera 203 is an example of an image sensor. The 2D camera 203 captures with visible light, and generates video information of the visible light. The distance sensor 204 detects a distance to an object, for example, by irradiating laser light or ultrasonic waves. In addition, the sensor section 200 can include a clock, a gyro sensor, a touch sensor, a sensor for motor feedback, and the like, in addition to these.

[0084] In addition, among the constituent elements of the robot 100 illustrated in Figure 2 Among the constituent elements of the robot 100 illustrated in

[0085] The storage section 220 includes a reaction rule 221 and history data 2222. The history data 2222 includes a history of an emotional value and a behavior of the user 10 in the past. The history of the emotional value and the behavior is recorded for each user 10, for example, by corresponding to the identification information of the user 10. At least a part of the storage section 220 is realized by a storage medium such as a memory. A person DB in which a face image of the user 10, attribute information of the user 10, and the like are stored can also be included. In addition, among the constituent elements of the robot 100 illustrated in Figure 2 Among the constituent elements of the robot 100 illustrated in

[0086] The sensor module section 210 includes a sound emotion recognition section 211, a speech understanding section 212, an expression recognition section 213, and a face recognition section 214. The sensor module section 210 is input with information detected by the sensor section 200. The sensor module section 210 analyzes the information detected by the sensor section 200, and outputs the analysis result to the state recognition section 230.

[0087] The voice emotion recognition section 211 of the sensor module section 210 analyzes the voice of the user 10 detected by the microphone 201, and recognizes the emotion of the user 10. For example, the voice emotion recognition section 211 extracts a feature amount such as a frequency component of the voice, and recognizes the emotion of the user 10 on the basis of the extracted feature amount. The speech understanding section 212 analyzes the voice of the user 10 detected by the microphone 201, and outputs text information indicating the content of the speech of the user 10.

[0088] The expression recognition section 213 recognizes the expression of the user 10 and the emotion of the user 10 on the basis of the image of the user 10 captured by the 2D camera 203. For example, the expression recognition section 213 recognizes the expression and the emotion of the user 10 on the basis of the shape, positional relationship, and the like of the eye portion and the mouth portion.

[0089] The face recognition section 214 recognizes the face of the user 10. The face recognition section 214 recognizes the user 10 by matching the face image stored in a person DB (omitted from the drawing) and the face image of the user 10 captured by the 2D camera 203.

[0090] The state recognition section 230 recognizes the state of the user 10 on the basis of the information analyzed by the sensor module section 210. For example, using the analysis result of the sensor module section 210, processing mainly related to perception is performed. For example, perception information such as "Dad is alone" and "The probability that Dad has no smile is 90%" is generated. Processing to understand the meaning of the generated perception information is performed. For example, meaning information such as "Dad is alone and looks very lonely" is generated.

[0091] The emotion determination section 232 determines the emotion value indicating the emotion of the user 10 on the basis of the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230. For example, the information analyzed by the sensor module section 210 and the recognized state of the user 10 are input to a neural network learned in advance, and the emotion value indicating the emotion of the user 10 is acquired.

[0092] Here, the emotion value indicating the emotion of the user 10 is a value indicating the positive or negative of the emotion of the user, and for example, if the emotion of the user is a bright emotion such as "joy", "cheer", "happiness", "comfort", "excitement", "satisfaction", and "feeling of fulfillment" that accompanies a pleasant feeling or peace, a positive value is indicated, and the more bright the emotion, the larger the value. If the emotion of the user is an unpleasing emotion such as "anger", "sorrow", "unhappiness", "unease", "melancholy", "worry", and "feeling of emptiness" that becomes an unpleasing emotion, a negative value is indicated, and the more unpleasing the emotion, the larger the absolute value of the negative value. In a case where the emotion of the user is none of the above ( "normal" ), a value of 0 is indicated.

[0093] Further, the emotion determination section 232 determines the emotion value of the robot 100 representing an emotion on the basis of the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230.

[0094] The emotion value of the robot 100 contains an emotion value for each of a plurality of emotion categories, for example, values (0 to 5) representing the intensity of each of "joy", "anger", "sorrow", and "pleasure".

[0095] Specifically, the emotion determination section 232 determines the emotion value of the robot 100 representing an emotion in accordance with a rule for updating the emotion value of the robot 100 prescribed in correspondence with the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230.

[0096] For example, the emotion determination section 232 increases the emotion value of "sorrow" of the robot 100 in a case where the state recognition section 230 recognizes that the user 10 looks very lonely. Further, the emotion determination section 232 increases the emotion value of "joy" of the robot 100 in a case where the state recognition section 230 recognizes that the user 10 is smiling.

[0097] Further, the emotion determination section 232 can further take into account the state of the robot 100 to determine the emotion value of the robot 100 representing an emotion. For example, the emotion value of "sorrow" of the robot 100 can be increased in a case where the remaining battery level of the robot 100 is low or the surrounding environment of the robot 100 is pitch dark, or the like. Further, the emotion value of "anger" can be increased in a case where the user 10 continues to talk to the robot 100 despite the low remaining battery level.

[0098] The behavior recognition section 234 recognizes the behavior of the user 10 on the basis of the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230. For example, the information analyzed by the sensor module section 210 and the recognized state of the user 10 are input to a neural network learned in advance, and the probability of each of a predetermined plurality of behavior categories (e.g., "laugh", "anger", "question", "sorrow") is obtained, and the behavior category with the highest probability is recognized as the behavior of the user 10.

[0099] As described above, in the first embodiment, the robot 100 acquires the utterance content of the user 10 on a specific user 10 basis, but the behavior control system of the robot 100 involved in the first embodiment takes into account the protection of the personal information and privacy of the user 10 in addition to acquiring the necessary consent from the user 10 in accordance with the law when acquiring and utilizing the utterance content, and the like.

[0100] The behavior determination section 236 determines a behavior corresponding to the behavior of the user 10 recognized by the behavior recognition section 234, on the basis of the current emotional value of the user 10 determined by the emotion determination section 232, the history data 2222 of the past emotional values determined by the emotion determination section 232 before the current emotional value of the user 10 is determined, and the emotional value of the robot 100. In the first embodiment, the behavior determination section 236 is described as using the most recent one of the emotional values contained in the history data 2222 as the past emotional value of the user 10, but the technology disclosed is not limited to this aspect. For example, the behavior determination section 236 can use a plurality of recent emotional values as the past emotional value of the user 10, or can use an emotional value from a unit period ago, such as a day ago, as the past emotional value of the user 10. In addition, the behavior determination section 236 can determine a behavior corresponding to the behavior of the user 10, further taking into account not only the current emotional value of the robot 100 but also the history of the past emotional values of the robot 100. The behavior determined by the behavior determination section 236 includes a gesture performed by the robot 100 or a content of a speech of the robot 100.

[0101] The behavior determination section 236 determines a behavior of the robot 100 as a behavior corresponding to the behavior of the user 10, on the basis of a combination of the past emotional value and the current emotional value of the user 10, the emotional value of the robot 100, and the reaction rule 221. For example, in a case where the past emotional value of the user 10 is a positive value and the current emotional value is a negative value, the behavior determination section 236 determines a behavior for changing the emotional value of the user 10 to be positive as a behavior corresponding to the behavior of the user 10.

[0102] In the reaction rule 221, a behavior of the robot 100 corresponding to a combination of the past emotional value and the current emotional value of the user 10, the emotional value of the robot 100, and the behavior of the user 10 is specified. For example, in a case where the past emotional value of the user 10 is a positive value, the current emotional value is a negative value, and the behavior of the user 10 is pitiful, a combination of a gesture and a content of a speech at the time of an inquiry to encourage the user 10 in combination with the gesture is specified as a behavior of the robot 100.

[0103] For example, in the reaction rule 221, the behavior of the robot 100 is specified for all combinations of the pattern of the emotional value of the robot 100 (6 values of the 4th power of the values of "0" to "5" of "joy", "anger", "sorrow", and "happiness", that is, a 1296 pattern), the pattern of the combination of the past emotional value and the current emotional value of the user 10, and the behavior pattern of the user 10. That is, for each pattern of the emotional value of the robot 100, the combination of the past emotional value and the current emotional value of the user 10 is, like negative and negative, negative and positive, positive and negative, positive and positive, negative and normal, and normal and normal, and for each of a plurality of combinations, the behavior of the robot 100 corresponding to the behavior pattern of the user 10 is specified. In addition, the behavior determination section 236 can also shift to an operation mode in which the behavior of the robot 100 is determined using the history data 2222, for example, in a case where the user 10 makes a speech in which the user 10 intends to continue the dialogue of the past topic "want to say the topic that was said before". In addition, in the reaction rule 221, at least one of the posture and the speech content can be specified as the behavior of the robot 100 one by one at the maximum for each pattern of the emotional value pattern (1296 pattern) of the robot 100. Alternatively, in the reaction rule 221, at least one of the posture and the speech content can be specified as the behavior of the robot 100 for each group of the pattern of the emotional value of the robot 100.

[0104] The intensity of each posture included in the behavior of the robot 100 specified in the reaction rule 221 is specified in advance. The intensity of each speech content included in the behavior of the robot 100 specified in the reaction rule 221 is specified in advance.

[0105] The storage control section 238 determines whether or not to store the data including the behavior of the user 10 in the history data 2222 on the basis of the intensity of the behavior specified in advance for the behavior determined by the behavior determination section 236 and the emotional value of the robot 100 determined by the emotion determination section 232.

[0106] Specifically, in a case where the integrated value of the intensity, that is, the sum of the intensity specified in advance for the posture included in the behavior determined by the behavior determination section 236 and the intensity specified in advance for the speech content included in the behavior determined by the behavior determination section 236, is equal to or greater than a threshold value, the data including the behavior of the user 10 is determined to be stored in the history data 2222.

[0107] The storage control section 238 stores, in the history data 2222, the behavior determined by the behavior determination section 236, information (for example, all of the surrounding information such as the data of the sound, the image, the odor, and the like in the field) analyzed by the sensor module section 210 from the current timing to a certain period before, and the state (for example, the expression, the emotion, and the like of the user 10) of the user 10 identified by the state identification section 230 in the case where it is determined to store the data including the behavior of the user 10 in the history data 2222.

[0108] The behavior control section 250 controls the control object 252 in accordance with the behavior determined by the behavior determination section 236. For example, in the case where the behavior determination section 236 determines the behavior including the utterance, the behavior control section 250 outputs the sound from the speaker included in the control object 252. At this time, the behavior control section 250 can also determine the speed of the utterance of the sound on the basis of the emotion value of the robot 100. For example, the greater the emotion value of the robot 100, the faster the speed of the utterance determined by the behavior control section 250. In this way, the behavior control section 250 determines the execution manner of the behavior determined by the behavior determination section 236 in accordance with the emotion value determined by the emotion determination section 232.

[0109] The behavior control section 250 can identify the change in the emotion of the user 10 with respect to the execution of the behavior determined by the behavior determination section 236. For example, the change in the emotion can be identified on the basis of the sound or the expression of the user 10. In addition to this, the change in the emotion of the user 10 can also be identified on the basis of the detection of the impact by the touch sensor included in the sensor section 200. It can also be identified that the emotion of the user 10 is deteriorated in the case where the impact is detected by the touch sensor included in the sensor section 200, or that the emotion of the user 10 is improved in the case where it is judged from the detection result of the touch sensor included in the sensor section 200 that the reaction of the user 10 is laughter or happiness, and the like. The information indicating the reaction of the user 10 is output to the communication processing section 280.

[0110] In addition, after the behavior control section 250 executes the behavior determined by the behavior determination section 236 in the execution manner determined in accordance with the emotion of the robot 100, the emotion determination section 232 further changes the emotion value of the robot 100 on the basis of the reaction of the user to the execution of the behavior. Specifically, the emotion determination section 232 increases the emotion value of "joy" of the robot 100 in the case where the reaction of the user to the execution of the behavior determined by the behavior determination section 236 in the execution manner determined by the behavior control section 250 is not bad, and the emotion determination section 232 increases the emotion value of "sorrow" of the robot 100 in the case where the reaction of the user to the execution of the behavior determined by the behavior determination section 236 in the execution manner determined by the behavior control section 250 is bad.

[0111] Further, the behavior control section 250 embodies the emotion of the robot 100 based on the determined emotion value of the robot 100. For example, the behavior control section 250 controls the control target 252 so that the robot 100 performs a happy behavior in a case where the emotion value of "joy" of the robot 100 is increased. In addition, the behavior control section 250 controls the control target 252 so that the posture of the robot 100 becomes a dejected posture in a case where the emotion value of "sorrow" of the robot 100 is increased.

[0112] The communication processing section 280 undertakes communication with the server 300. As described above, the communication processing section 280 transmits the user reaction information to the server 300. In addition, the communication processing section 280 receives the updated reaction rule from the server 300. If the communication processing section 280 receives the updated reaction rule from the server 300, the reaction rule 221 is updated.

[0113] The server 300 undertakes communication between the robot 100, the robot 101, and the robot 102 and the server 300, receives the user reaction information transmitted from the robot 100, and updates the reaction rule based on the reaction rule including the behavior for which a positive reaction is obtained.

[0114] Figure 3 An example of an operation flow related to the operation of determining the behavior in the robot 100 is schematically shown. The operation flow shown is repeatedly executed Figure 3 The operation flow shown is repeatedly executed

[0115] First, in step S100, the state recognition section 230 recognizes the state of the user 10 based on the information analyzed by the sensor module section 210.

[0116] In step S102, the emotion determination section 232 determines the emotion value indicating the emotion of the user 10 based on the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230.

[0117] In step S103, the emotion determination section 232 determines the emotion value indicating the emotion of the robot 100 based on the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230. The emotion determination section 232 adds the determined emotion value of the user 10 to the history data 2222.

[0118] In step S104, the behavior recognition section 234 recognizes the behavior classification of the user 10 based on the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230.

[0119] In step S106, the behavior determination section 236 determines the behavior of the robot 100 based on a combination of the current emotional value of the user 10 determined in step S102 and the past emotional values contained in the history data 2222, the emotional value of the robot 100, the behavior of the user 10 recognized by the behavior recognition section 234, and the reaction rule 221.

[0120] In step S108, the behavior control section 250 controls the control object 252 based on the behavior determined by the behavior determination section 236.

[0121] In step S110, the storage control section 238 calculates a comprehensive value of the intensity based on the intensity of the behavior prescribed in advance for the behavior determined by the behavior determination section 236 and the emotional value of the robot 100 determined by the emotion determination section 232.

[0122] In step S112, the storage control section 238 determines whether the comprehensive value of the intensity is equal to or greater than a threshold value. In the case where the comprehensive value of the intensity is less than the threshold value, the data including the behavior of the user 10 is not stored in the history data 2222, and the process ends. On the other hand, in the case where the comprehensive value of the intensity is equal to or greater than the threshold value, the process proceeds to step S114.

[0123] In step S114, the behavior determined by the behavior determination section 236, the information analyzed by the sensor module section 210 from the current timing to a certain period before, and the state of the user 10 recognized by the state recognition section 230 are stored in the history data 2222.

[0124] As described above, according to the robot 100, the emotional value indicating the emotion of the robot 100 is determined based on the state of the user, and it is determined whether to store the data including the behavior of the user 10 in the history data 2222 based on the emotional value of the robot 100. Thereby, it is possible to suppress the capacity of the history data 2222 storing the data including the behavior of the user 10. Also, for example, in the case where the robot 100 determines that the state of the user 10 is the same as the state 10 years before, by reading the history data 2222 10 years before, the robot 100 is able to exhibit the state of the user 10 at that time 10 years before (for example, the expression of the user 10, the emotion, and the like), further the data of the sound, the image, the odor, and the like at that time, and the like to the user 10.

[0125] Further, according to the robot 100, it is possible to cause the robot 100 to perform a suitable behavior with respect to the behavior of the user 10. In the past, a behavior including an expression or an appearance of a robot was determined by classifying a behavior of a user. In contrast, the robot 100 determines a current emotion value of the user 10 and performs a behavior with respect to the user 10 on the basis of a past emotion value and the current emotion value. Thus, for example, in a case where the user 10 who was good yesterday is in a low mood today, the robot 100 can make a statement such as "You were good yesterday, but what's wrong today?". Further, the robot 100 can make a statement in combination with a gesture. Further, for example, in a case where the user 10 who was in a low mood yesterday is good today, the robot 100 can make a statement such as "You were in a low mood yesterday, but are you good today?". Further, for example, in a case where the user 10 who was good yesterday is better than yesterday, the robot 100 can make a statement such as "You are better than yesterday. Is there something better than yesterday?". Further, for example, the robot 100 can make a statement such as "You have been in a stable mood recently, and you feel good" with respect to the user 10 whose emotion value is 0 or more and whose variation in emotion value continues within a certain range.

[0126] Further, for example, the robot 100 asks the user 10 "Did you finish the homework you said yesterday?", and in a case where an answer of "Yes, I did" is obtained from the user 10, it is possible to make an affirmative statement such as "You are amazing!" and to make an affirmative gesture such as clapping or a gesture. Further, for example, the robot 100 can make an affirmative statement such as "You did it!" and make the above-described affirmative gesture when the user 10 says "The presentation you said the other day went well". In this way, by causing the robot 100 to perform a behavior on the basis of a history of a state of the user 10, it is possible to expect that the user 10 will develop an affinity for the robot 100.

[0127] In the above-described embodiment, a case where the robot 100 recognizes the user 10 using a facial image of the user 10 is described, but the disclosed technology is not limited to this aspect. For example, the robot 100 can recognize the user 10 using a voice uttered by the user 10, a mail address of the user 10, an ID of an SNS of the user 10, or an ID card of a built-in wireless IC tag held by the user 10.

[0128] Further, the robot 100 is an example of an electronic machine provided with a behavior control system. The application target of the behavior control system is not limited to the robot 100, and the behavior control system can be applied to various electronic machines. Further, the functions of the server 300 can be implemented by one or more computers. At least a part of the functions of the server 300 can be implemented by a virtual machine. Further, at least a part of the functions of the server 300 can be implemented by a cloud.

[0129] Figure 4An example of a hardware structure of the computer 1200 functioning as the smartphone 50, the robot 100, the server 300, and the agent system 500 is schematically shown. A program installed in the computer 1200 can cause the computer 1200 to function as one or more "parts" of the apparatuses related to the first embodiment, or to perform operations or one or more "parts" associated with the apparatuses related to the first embodiment, and / or to perform processes or stages of processes related to the first embodiment. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described in this specification.

[0130] The computer 1200 in the first embodiment includes the CPU 1212, the RAM 1214, and the graphics controller 1216, which are connected to each other through the host controller 1210. The computer 1200 further includes input output units such as the communication interface 1222, the storage device 1224, the DVD drive 1226, and the IC card drive, which are connected to the host controller 1210 via the input output controller 1220. The DVD drive 1226 can be a DVD-ROM 1227 drive, a DVD-RAM drive, or the like. The storage device 1224 can be a hard disk drive, a solid state drive, or the like. The computer 1200 further includes old input output units such as the ROM 1230 and the keyboard, which are connected to the input output controller 1220 via the input output chip 1240.

[0131] The CPU 1212 operates in accordance with programs stored in the ROM 1230 and the RAM 1214, and thereby controls the units. The graphics controller 1216 acquires image data generated by the CPU 1212 in a frame buffer or the like provided in the RAM 1214 or in itself, and causes the image data to be displayed on the display device 1218.

[0132] The communication interface 1222 communicates with other electronic devices through a network. The storage device 1224 holds programs and data used by the CPU 1212 in the computer 1200. The DVD drive 1226 reads programs or data from the DVD-ROM 1227 or the like, and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card, and / or writes programs and data to the IC card.

[0133] The ROM 1230 holds therein, among other things, a boot program or the like executed by the computer 1200 at activation and / or a program dependent on the hardware of the computer 1200. The input output chip 1240 can also connect various input output units to the input output controller 1220 through a USB port, a parallel port, a serial port, a keyboard port, a mouse port, or the like.

[0134] The programs are provided by a computer-readable storage medium such as the DVD-ROM 1227 or the IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230 as examples of the computer-readable storage medium, and executed by the CPU 1212. The information processing described in these programs is read by the computer 1200, resulting in cooperation between the programs and the various types of hardware resources described above. An apparatus or a method can be constituted by implementing the operation or processing of information based on the use of the computer 1200.

[0135] For example, in the case where communication is performed between the computer 1200 and an external device, the CPU 1212 can execute a communication program loaded in the RAM 1214, and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. The communication interface 1222 reads transmission data saved in a transmission buffer area provided in the RAM 1214, the storage device 1224, a recording medium such as the DVD-ROM 1227 or the IC card, or the like under the control of the CPU 1212, transmits the read transmission data to a network, or writes reception data received from the network to a reception buffer area provided on the recording medium, or the like.

[0136] In addition, the CPU 1212 can read all or a necessary part of a file or a database saved in the storage device 1224, the DVD drive 1226 (DVD-ROM 1227), an external recording medium such as an IC card, or the like in the RAM 1214, and perform various types of processing on the data on the RAM 1214. Subsequently, the CPU 1212 can write the processed data back to the external recording medium.

[0137] Various types of information such as programs, data, tables, and databases can be saved in the recording medium and receive information processing. The CPU 1212 can perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgments, conditional branching, unconditional branching, search / replacement of information, and the like specified by a sequence of instructions of a program described anywhere in the present disclosure, and write the results back to the RAM 1214. In addition, the CPU 1212 can search for information in a file, a database, or the like in the recording medium. For example, if a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are saved in the recording medium, the CPU 1212 can search for an entry that coincides with a condition that specifies an attribute value of the first attribute from the plurality of entries, read the attribute value of the second attribute saved in the entry, and thereby acquire the attribute value of the second attribute associated with the first attribute that satisfies the predetermined condition.

[0138] The programs or software modules described above can be stored in a computer-readable storage medium in or near the computer 1200. In addition, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as the computer-readable storage medium, so that the programs are provided to the computer 1200 via the network.

[0139] The blocks in the flowcharts and block diagrams in the first embodiment can represent stages of processes that perform operations or "parts" of apparatuses that have roles of performing operations. The specific stages and "parts" can be implemented by dedicated circuits, programmable circuits provided with computer-readable instructions stored on a computer-readable storage medium, and / or processors provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuits can include digital and / or analog hardware circuits, and can include integrated circuits (ICs) and / or discrete circuits. The programmable circuits can include reconfigurable hardware circuits including logical product, logical sum, exclusive OR, negated logical product, negated logical sum, and other logical operations, flip-flops, registers, and storage elements such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs).

[0140] The computer-readable storage medium can include any tangible device that is capable of storing instructions for execution by an appropriate device, and having a computer-readable storage medium with instructions stored therein result in a product comprising instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of the computer-readable storage medium can include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and so forth. More specific examples of the computer-readable storage medium can include a floppy disk, a magnetic hard disk, a semiconductor memory, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an electrically erasable programmable read-only memory (EEPROM), a static random access memory (SRAM), a compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a Blu-ray Disc, a memory stick, an integrated circuit card, and so forth.

[0141] The computer-readable instructions can include any one of source code or object code that is described in any combination of one or more programming languages including assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or a target-oriented programming language such as Smalltalk, JAVA (registered trademark), C++, and the like, and a conventional program programming language such as the "C" programming language or the like.

[0142] The computer readable instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate means for implementing the functions of the flowchart block or the block diagram block or the elements thereof. As an example, the processor can include one or more of a central processing unit (CPU), a processor core, a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, or any combination thereof.

[0143] The above describes the present disclosure using embodiments, but the technical scope of the present disclosure is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various changes or improvements can be made to the above embodiments. It is obvious that such changes or improvements can be included in the technical scope of the present disclosure based on the recitations of the claims.

[0144] Note that the order of execution of the processes of the operations, sequences, steps, and stages of the apparatuses, systems, programs, and methods shown in the claims, the specification, and the drawings can be changed as long as it is not explicitly stated that the preceding process is "before," "prior to," or the like, or the output of the preceding process is used in the subsequent process. For the flow of operations in the claims, the specification, and the drawings, even if described using "first," "next," and the like for convenience, it does not mean that it must be implemented in that order.

[0145] (Other Embodiment 1) The robot 100 of the present embodiment has an image acquisition section that images a play space in which a specific play can be performed. The image acquisition section can be implemented using a part of the sensor section 200 described above, for example. Here, the specific play can be a sport performed by a team composed of a plurality of people, such as volleyball, soccer, rugby, and the like. In addition, the play space can include a space corresponding to each play, such as a volleyball court or a soccer field, and the like. In addition, the play space can also include a surrounding area of the above-described court and the like.

[0146] The robot 100 can be disposed so as to be able to look around the play space by the image acquisition section. Alternatively, the image acquisition section of the robot 100 can be disposed at a position separate from the robot 100 but capable of looking around the play space.

[0147] Furthermore, the robot 100 of this embodiment also includes a competitor emotion determination unit capable of determining the emotions of multiple competitors within an image acquired by the image acquisition unit. This competitor emotion determination unit can determine the emotions of multiple competitors using the same method as the emotion determination unit 232. Specifically, for example, the emotion of each competitor can be determined by inputting information from the analysis of the image acquired by the image acquisition unit by the sensor module unit 210 into a pre-learned neural network to determine the emotion values ​​representing the emotions of the multiple competitors.

[0148] Furthermore, the robot 100 of this embodiment also includes a feature determination unit capable of determining the characteristics of multiple competitors within an image acquired by the image acquisition unit described above. This feature determination unit analyzes past competition data using the same method as the emotion value determination method in the emotion determination unit 232, collects and analyzes information related to each competitor from SNS or similar sources, or combines one or more of these methods to determine the characteristics of multiple competitors.

[0149] A competitor's characteristics refer to their habits, actions, number of mistakes, unskilled actions, reaction speed, and other competitive abilities or information related to their current or recent state.

[0150] If, based on the emotional values ​​of a competitor engaged in a specific sport, such as volleyball, it can be determined that the competitor's emotions are unstable (e.g., by detecting the emotional map 400 described later (see reference)). Figure 5 The robot 100 detects emotions in the direction of "bitterness" or "fear" in the emotion map 400 (e.g., detecting emotions in the direction of "anger" in the emotion map 400 described later). By reflecting this determination in the team's strategy, it is possible to make the game more favorable. Specifically, the probability of making mistakes is said to be higher for emotionally unstable or agitated competitors than for emotionally stable competitors. Therefore, for example, in the case of volleyball, if the emotionally unstable or agitated competitor has more opportunities to touch the ball, it can be said that the probability of making a mistake is higher. Therefore, in this embodiment, it is proposed to make the game more favorable by communicating the emotion values ​​of each competitor determined by the robot 100 to the user 10, such as the coach of a team in the game.

[0151] The player analyzed by the player emotion analysis section can be a player belonging to a specific team among the plurality of players in the play space. More specifically, the specific team can be a team different from the team to which the user 10 of the robot 100 belongs, that is, the opposing team. By scanning the emotions of the players of the opposing team, determining the player whose emotion is most unstable or most agitated, focusing on the position of the player and playing the game (for example, if the play content is volleyball, concentrating the spike ball on the player whose emotion is unstable or agitated), the game will be favorably played. In view of the above, the player whose characteristics are determined by the characteristic determination section can be a player belonging to a specific team among the plurality of players in the play space. More specifically, the specific team can be a team different from the team to which the user belongs, in other words, the opposing team. By scanning the characteristics of each player of the opposing team, determining the player who has a specific habit and the player who makes a lot of mistakes, and providing the user with information about the characteristics of the player, effective strategy making can be assisted.

[0152] In addition, more specifically, a player who makes a lot of mistakes or a player who has a specific habit can become a weak point of the team. Therefore, in the present embodiment, by transmitting the characteristics of each player determined by the robot 100 to the user, for example, the coach of one team in the game, an element for favorably advancing the game in the play is provided.

[0153] In this way, if the coach or the like uses the robot 100 in a play game in which teams are in a stalemate, it is expected that the game will be favorably played. More specifically, by determining the player who is most unstable in spirit in the play, implementing a strategy of focusing on the player, it is possible to come closer to victory. If such a robot 100 is used in a play game in which teams are in a stalemate, it is expected that the game will be favorably played. More specifically, by determining the player who makes a lot of mistakes in the game or the like, adopting a strategy of focusing on the position of the player, it is possible to come closer to victory.

[0154] The emotion determination section 232 can determine the emotion of the user in accordance with a specific mapping. More specifically, the emotion determination section 232 can determine the emotion of the user in accordance with an emotion map (refer to FIG. 6) as a specific mapping. Figure 5

[0155] Figure 5 ​is a diagram showing the emotion map 400 that maps a plurality of emotions. In the emotion map 400, emotions are radially arranged in concentric circles from the center. The closer to the center of the concentric circles, the more the emotions of the original state are arranged. On the outside of the concentric circles, emotions that represent states or behaviors generated from the mind are arranged. Emotions are concepts that include moods or psychological states. On the left side of the concentric circles, emotions generated by reactions caused in the brain are generally arranged. On the right side of the concentric circles, emotions induced by situation judgment are generally arranged. Above and below the concentric circles, emotions generated by reactions caused in the brain and induced by situation judgment are generally arranged. In addition, emotions of "pleasure" are arranged on the upper side of the concentric circles, and emotions of "unpleasure" are arranged on the lower side of the concentric circles. In this way, in the emotion map 400, a plurality of emotions are mapped based on the structure of generating emotions, and emotions that are likely to be generated at the same time are mapped in the vicinity.

[0156] (1) For example, in a case where the emotion engine as the emotion determination unit 232 of the robot 100 detects emotions at around 100 milliseconds, the determination of the reaction operation (for example, the echo response) of the robot 100 can be set to a timing at least as frequently as the detection frequency (100 milliseconds) of the emotion engine, or can be set to a timing earlier than that. The detection frequency of the emotion engine can be interpreted as a sampling rate.

[0157] By detecting emotions at around 100 milliseconds and immediately linking to perform the reaction operation (for example, the echo response), it is possible to achieve an echo response that is not unnatural, but a natural and appropriate conversation. The robot 100 performs the reaction operation (echo response, etc.) in accordance with the mandelbrot directivity of the emotion map 400 and the degree (intensity) thereof. In addition, the detection frequency (sampling rate) of the emotion engine is not limited to 100 milliseconds, and can be changed in accordance with the scenario (in the case of performing a motion, etc.), the age of the user, and the like.

[0158] (2) In contrast to the emotion map 400, the directivity of the emotions and the intensity of the degree thereof are set in advance, and the operation of the echo response and the intensity of the echo response can be set. For example, in a case where the robot 100 feels a sense of stability, security, and the like, the robot 100 nods and continues to listen to the speech. In a case where the robot 100 feels unease, hesitation, strangeness, and the like, the robot 100 can tilt the head or can stop shaking the head.

[0159] These emotions are distributed in the direction of 3 o'clock of the emotion map 400, and generally go back and forth in the vicinity of security and unease. In the right half of the emotion map 400, situation recognition is more dominant than interoception, and thus a calm impression is made.

[0160] (3) When Robot 100 feels happy after being praised, a filler word such as "hmm" can be added before the dialogue. When Robot 100 feels pain after being harshly criticized, a filler word such as "ah!" can be added before the dialogue. In addition, physical reactions such as Robot 100 curling up while saying "ah!" can also be included. These emotions are distributed around the 9 o'clock position on the emotion map 400.

[0161] (4) In the left half of the emotion map 400, internal sensation (response) is more dominant than situation recognition. Therefore, it gives the impression of involuntary reaction.

[0162] When Robot 100 senses internal feelings of recognition (response) and also develops liking in situational awareness, it can look at the other party, nod deeply, and even make an "uh-huh" sound. In this way, Robot 100 can generate just the right amount of liking for the other party, that is, behaviors such as tolerance or forbearance. This emotion is distributed around the 12 o'clock position on the emotion map 400.

[0163] Conversely, when Robot 100 experiences an internal feeling (reaction) of displeasure and the same applies to situation recognition, Robot 100 can shake its head when feeling disgust, and if it feels aversion, it can turn the LEDs in its eyes red and stare at the other person. This emotion is distributed around the 6 o'clock position on the emotion map 400.

[0164] (5) Because the inner side of the emotional map 400 represents the inner world and the outer side of the emotional map 400 represents behavior, the further out of the emotional map 400, the more obvious the emotion becomes (manifested as behavior).

[0165] (6) When feeling at ease around the 3 o'clock position on the emotional map 400 and listening to people speak, the robot 100 nods slightly and says "uh-huh". However, when it reaches the love position around the 12 o'clock position, it can nod strongly like a deep bow.

[0166] The emotion determination unit 232 inputs the information analyzed by the sensor module unit 210 and the identified state of the user 10 into a pre-learned neural network to obtain the emotion values ​​representing each emotion shown in the emotion map 400, and determines the emotion of the user 10. This neural network is a pre-learned neural network based on a combination of the information analyzed by the sensor module unit 210, the identified state of the user 10, and the emotion values ​​representing each emotion shown in the emotion map 400—that is, a combination of multiple learning data. Furthermore, as... Figure 6 As shown in the sentiment map 900, the neural network learns by recognizing that sentiment values ​​in neighboring configurations are similar to each other. Figure 6In this case, as an example, a plurality of emotions such as "at ease", "at peace", and "reliable" are shown as emotions having similar values.

[0167] In addition, the emotion determination section 232 can determine the emotion of the robot 100 in accordance with a specific mapping. Specifically, the emotion determination section 232 inputs the information analyzed by the sensor module section 210, the state of the user 10 and the state of the robot 100 recognized by the state recognition section 230 into a neural network learned in advance, acquires the emotion values representing each emotion shown in the emotion map 400, and determines the emotion of the robot 100. This neural network is a neural network learned in advance based on a combination of the information analyzed by the sensor module section 210, the state of the user 10 and the state of the robot 100 recognized, and the emotion values representing each emotion shown in the emotion map 400, i.e., a plurality of learning data. For example, in a case where it is recognized that the robot 100 is being stroked by the user 10 in accordance with the output of the touch sensor (omitted from the drawing), the neural network is learned based on learning data representing an emotion value "3" that becomes "joy", or in a case where it is recognized that the robot 100 is being hit by the user 10 in accordance with the output of the acceleration sensor (omitted from the drawing), the neural network is learned based on learning data representing an emotion value "3" that becomes "anger". In addition, as shown in the emotion map 900, this neural network is learned such that emotions arranged in proximity to each other have similar values. Figure 6

[0168] The behavior determination section 236 generates the behavior content of the robot by inputting, into an article generation model having a dialogue function, a fixed sentence for asking about the behavior content of the robot corresponding to the behavior of the user, added in a text representing the behavior of the user, the emotion of the user, and the emotion of the robot.

[0169] For example, the behavior determination section 236 acquires a text representing the state of the robot 100 using the emotion table shown in Table 1, from among the emotions of the robot 100 determined by the emotion determination section 232. Here, in the emotion table, each emotion value is given an index number for each category of emotion, and for each index number, a text representing the state of the robot 100 is saved.

[0170] In a case where the emotion of the robot 100 determined by the emotion determination section 232 corresponds to the index number "2", a text of "a very happy state" is obtained. In addition, in a case where the emotion of the robot 100 corresponds to a plurality of index numbers, a plurality of texts representing the state of the robot 100 are obtained.

[0171] ​In addition, for the emotion of the user 10, an emotion table shown in Table 2 is prepared. Here, in a case where the behavior of the user is "Is there an irritable player?", the emotion of the robot 100 is index number "2", and the emotion of the user 10 is index number "3", "The robot is in a very happy state. The user is in a generally happy state. The user has been addressed "Is there an irritable player?". How does the robot reply as a robot?" is input to the article generation model, and the behavior content of the robot is acquired. The behavior determination section 236 determines the behavior of the robot based on the behavior content.

[0172] Table 1 Table 2 Thus, since the robot 100 can change the behavior of the robot according to the index number corresponding to the emotion of the robot, the user has an impression that the robot has a mind, and the user is encouraged to take the behavior of addressing the robot and the like.

[0173] Further, the behavior determination section 236 can determine the behavior of the robot using the analysis result of the above-mentioned player emotion analysis section in a case where a question related to the player is received as described above. Specifically, as a reply to the inquiry from the above-mentioned user, it is possible to utter the analysis result such as "The player number 2 of the opposing team is quite irritable".

[0174] In addition, the behavior determination section 236 can generate the behavior content of the robot by adding not only the text indicating the behavior of the user, the emotion of the user, and the emotion of the robot, but also the text indicating the content of the history data 2222, and then adding a fixed sentence for asking the behavior content of the robot corresponding to the behavior of the user, and inputting the article generation model having a dialogue function. Thus, since the robot 100 can change the behavior of the robot according to the history data indicating the emotion or the behavior of the user, the user has an impression that the robot has a personality, and the user is encouraged to take the behavior of addressing the robot and the like. Further, the emotion or the behavior of the robot can be further included in the history data.

[0175] Furthermore, the emotion determination unit 232 can also determine the emotion of the robot 100 based on the behavioral content of the robot 100 generated by the article generation model. Specifically, the emotion determination unit 232 inputs the behavioral content of the robot 100 generated by the article generation model into a pre-learned neural network, obtains the emotion values ​​representing each emotion shown in the emotion map 400, integrates the obtained emotion values ​​representing each emotion with the current emotion values ​​representing each emotion of the robot 100, and updates the emotion of the robot 100. For example, the obtained emotion values ​​representing each emotion and the current emotion values ​​representing each emotion of the robot 100 are averaged and integrated respectively. This neural network is a neural network pre-learned based on a combination of text representing the behavioral content of the robot 100 generated by the article generation model and emotion values ​​representing each emotion shown in the emotion map 400, i.e., multiple learning data.

[0176] For example, when the robot 100's speech content "That's great. How lucky!" is obtained, as the behavioral content of the robot 100 generated by the article generation model, the sentiment value of the emotion "joy" is increased when the text representing the speech content is input into the neural network, so that the sentiment value of the robot 100 is updated in a way that increases the sentiment value of the emotion "joy".

[0177] Furthermore, the robot 100 can be mounted on a plush toy or used in a control device that connects wirelessly or wiredly to a control object (speaker or camera) mounted on the plush toy. In this case, the specific configuration is as follows. For example, the robot 100 can be applied to cohabiting individuals (specifically, Figure 7 and Figure 8 The plush toy 100N shown above, while interacting with the user 10 in daily life, advances dialogue based on information relevant to the user 10's daily life, or provides information that matches the user 10's interests and hobbies. In this embodiment (other embodiments), an example of applying the control part of the robot 100 described above to a smartphone 50 will be described.

[0178] For the plush toy 100N, which is equipped with an input / output device that functions as an input / output device for the robot 100, a smartphone 50 that functions as a control unit for the robot 100 can be detached and attached. Inside the plush toy 100N, the input / output device is connected to the stored smartphone 50.

[0179] like Figure 7 As shown in (A), the plush toy 100N in this embodiment (the embodiment mounted on a plush toy) is in the shape of a bear with its exterior covered by soft fabric, as... Figure 7(B), in a space portion 52 formed in the inner side thereof, as input and output devices, a microphone 201 of a sensor portion 200 is arranged in a portion corresponding to an ear portion 54 (refer to Figure 2 ), a 2D camera 203 of the sensor portion 200 is arranged in a portion corresponding to an eye portion 56 (refer to Figure 2 ), and a speaker 60 which constitutes a part of the control target 252 (refer to Figure 2 ) is arranged in a portion corresponding to a mouth portion 58. In addition, the microphone 201 and the speaker 60 are not necessarily separate bodies, but can be a single unit. In the case of a unit, it can be arranged at a position where it can naturally hear a speech, such as a nose position of the stuffed toy 100N. In addition, the description has been made taking the case where the stuffed toy 100N is in the shape of an animal as an example, but is not limited thereto. The stuffed toy 100N can be in the shape of a specific character.

[0180] The smartphone 50 has the functions as the sensor module portion 210, the functions as the storage portion 220, the functions as the state recognition portion 230, the functions as the emotion determination portion 232, the functions as the behavior recognition portion 234, the functions as the behavior determination portion 236, the functions as the storage control portion 238, the functions as the behavior control portion 250, and the functions as the communication processing portion 280 as shown in Figure 2

[0181] As shown in Figure 8 , a zipper 62 is attached to a part (for example, a back portion) of the stuffed toy 100N, and by opening the zipper 62, a structure is formed in which the outside is communicated with the space portion 52.

[0182] Here, the smartphone 50 is housed in the space portion 52 from the outside, and is connected to each input and output device via a USB hub 64 (refer to Figure 7 (B), and thus can have the same functions as the robot 100 shown in Figure 1 .

[0183] In addition, a non-contact power receiving pad 66 is connected to the USB hub 64. A power receiving coil 66A is attached to the power receiving pad 66. The power receiving pad 66 is an example of a wireless power receiving portion which receives wireless power supply.

[0184] The power receiving pad 66 is arranged near both leg base portions 68 of the stuffed toy 100N, and is positioned closest to a placement pedestal 70 when the stuffed toy 100N is placed on the placement pedestal 70. The placement pedestal 70 is an example of an external wireless power supply portion.

[0185] The stuffed toy 100N placed on the placement pedestal 70 can be enjoyed as an ornament in a natural state.

[0186] ​In addition, the root portion is formed to be thinner than the thickness of the surface layer of the plush toy 100N in other portions, and is held in a state closer to the placement base 70.

[0187] The placement base 70 is provided with a charging pad 72. The charging pad 72 is assembled with a power supply coil 72A that transmits a signal to search for the power receiving coil 66A of the power receiving pad 66, and if the power receiving coil 66A is found, a magnetic field is generated by the current flowing through the power supply coil 72A, and the power receiving coil 66A reacts to the magnetic field to start electromagnetic induction. Thus, the current flows through the power receiving coil 66A, and stores power in the battery (omitted from the drawing) of the smartphone 50 via the USB hub 64.

[0188] That is, by placing the plush toy 100N as a prop on the placement base 70, the smartphone 50 is automatically charged, and therefore it is not necessary to take out the smartphone 50 from the space portion 52 of the plush toy 100N in order to charge.

[0189] In addition, in the present embodiment (embodiment mounted on a plush toy), the smartphone 50 is housed in the space portion 52 of the plush toy 100N, and connected by wire (USB connection), but is not limited thereto. For example, a control device with wireless function (for example, "Bluetooth (registered trademark)") can be housed in the space portion 52 of the plush toy 100N, and the control device can be connected to the USB hub 64. In this case, the smartphone 50 is not put into the space portion 52, and the smartphone 50 communicates wirelessly with the control device, and the external smartphone 50 is connected to each input and output device via the control device, and thus it is possible to have the same function as the robot 100 shown in FIG. 1. Figure 1 In addition, the control device housed in the space portion 52 of the plush toy 100N can be connected to the external smartphone 50 by wire.

[0190] In addition, in the present embodiment (embodiment mounted on a plush toy), a plush toy 100N of a bear is exemplified, but it can be a plush toy of another animal, or a doll, or a specific character shape. In addition, it can be changed. Furthermore, the material of the skin is not limited to cloth, and can be another material such as soft plastic, but a soft material is preferable.

[0191] Furthermore, a monitor can be installed on the skin of the plush toy 100N, and a control object 252 that provides information to the user 10 by vision can be added. For example, the eye portion 56 can be a monitor, and joy and sorrow can be expressed by an image reflected in the eye, or a window through which a monitor of the built-in smartphone 50 is set in the abdomen. Furthermore, the eye portion 56 can be a projector, and joy and sorrow can be expressed by an image projected on a wall surface.

[0192] According to another embodiment, an existing smartphone 50 is placed inside a plush toy 100N, thereby extending the camera 203, microphone 201, speaker 60, etc., to appropriate positions via a USB connection.

[0193] Furthermore, in order to perform wireless charging, the device is configured to connect the smartphone 50 and the power receiving board 66 via USB, with the power receiving board 66 positioned as close to the outside as possible when viewed from inside the plush toy 100N.

[0194] If you want to use the wireless charging of the smartphone 50, you must place the smartphone 50 as far out as possible when looking from the inside of the plush toy 100N. When you touch the plush toy 100N from the outside, it will become uneven.

[0195] Therefore, the smartphone 50 is positioned as centrally as possible within the plush toy 100N, while the wireless charging function (power receiving board 66) is positioned as far out as possible when viewed from the inside of the plush toy 100N. The camera 203, microphone 201, speaker 60, and smartphone 50 receive wireless power via the power receiving board 66.

[0196] The following notes are also disclosed regarding the above implementation methods.

[0197] (Note 1) A behavior control system, comprising: The emotion determination department determines the user's or the robot's emotions; and The behavior determination unit, based on the dialogue function that enables the user to converse with the robot, generates robot behavior content based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavior content. The behavior determination unit has the following features: The image acquisition unit is capable of capturing images of a competition space where a specific competition can be conducted; and The competitor emotion analysis unit analyzes the emotions of multiple competitors engaged in competition within the competition space, as captured by the image acquisition unit. The robot's behavior is determined based on the analysis results of the competitor's emotion analysis unit.

[0198] (Note 2) According to the behavior control system described in Appendix 1, the competitor emotion analysis unit analyzes the emotions of competitors belonging to a specific team among the plurality of competitors.

[0199] (Note 3) According to the behavior control system described in Appendix 1 or Appendix 2, the robot is mounted on a plush toy or connected wirelessly or wiredly to a controlled object machine mounted on the plush toy.

[0200] (Other Embodiment 2) The robot 100 of the present embodiment has an image acquisition section that images a competition space in which a specific competition can be performed. The image acquisition section can be implemented using a part of the sensor section 200 described above, for example. Here, the specific competition can be a sport performed by a team composed of a plurality of people, such as volleyball, soccer, rugby, or the like. In addition, the competition space can include a space corresponding to each competition, such as a volleyball court or a soccer field, or the like. In addition, the competition space can also include a surrounding area of the above-described court or the like.

[0201] The robot 100 can take into account its set position so as to be able to look around the competition space by the image acquisition section. Alternatively, the image acquisition section of the robot 100 can also be provided at a position separate from the robot 100 but capable of looking around the competition space.

[0202] In addition, the robot 100 of the present embodiment also has a characteristic determination section that can determine characteristics of a plurality of competitors within an image acquired by the above-described image acquisition section. The characteristic determination section can determine the characteristics of the plurality of competitors by analyzing past game data by the same method as the determination method of the emotion value in the emotion determination section 232, by collecting and analyzing information related to each competitor from SNS or the like, or by combining one or more of these methods.

[0203] The characteristics of the competitors refer to habits, actions, number of mistakes, actions not good at, reaction speed, or the like of the competitors, which are abilities related to the competition or information related to the present or recent state of the competitors.

[0204] If the characteristics of the competitors who are performing a specific competition, such as volleyball, can be determined, it is possible to make the game progress favorably by reflecting the determination result in the strategy of the team. Specifically, a competitor who makes many mistakes or a competitor who has a specific habit can become a weak point of the team. Therefore, in the present embodiment, by transmitting the characteristics of each competitor determined by the robot 100 to a user, such as a coach of one team in the game, an element for advancing the game favorably in the competition is provided.

[0205] In view of the above, the competitors whose characteristics are determined by the characteristic determination section can be competitors belonging to a specific team among the plurality of competitors within the competition space. More specifically, the specific team can be a team different from the team to which the user belongs, in other words, an opposing team. By scanning the characteristics of each competitor of the opposing team, determining a competitor who has a specific habit and a competitor who makes many mistakes, and providing the user with information about the characteristics of the competitor, effective strategy making can be assisted.

[0206] If such a robot 100 is used in a game in which teams are in a state of being in a deadlock with each other, it is expected that the game will be played with an advantage. Specifically, by determining a player who makes more mistakes in the game and the like, a strategy of concentrating on attacking the player's position can be taken, and a victory can be approached.

[0207] The emotion determination section 232 can determine the emotion of the user in accordance with a specific mapping. Specifically, the emotion determination section 232 can determine the emotion of the user in accordance with an emotion map (refer to FIG. 6) as a specific mapping. Figure 5

[0208] The behavior determination section 236 generates the behavior content of the robot by inputting, to an article generation model having a dialogue function, a fixed sentence for asking about the behavior content of the robot corresponding to the behavior of the user, which is added to a text representing the behavior of the user, the emotion of the user, and the emotion of the robot.

[0209] For example, the behavior determination section 236 acquires a text representing the state of the robot 100 using the emotion table shown in Table 1, based on the emotion of the robot 100 determined by the emotion determination section 232. Here, in the emotion table, each emotion value is given an index number for each kind of emotion, and for each index number, a text representing the state of the robot 100 is saved.

[0210] In a case where the emotion of the robot 100 determined by the emotion determination section 232 corresponds to the index number "2", a text of "a very happy state" is obtained. In addition, in a case where the emotion of the robot 100 corresponds to a plurality of index numbers, a plurality of texts representing the state of the robot 100 are obtained.

[0211] In addition, for the emotion of the user 10, an emotion table shown in Table 2 is prepared. Here, in a case where the behavior of the user is "please tell me the weakness of the opposing team", the emotion of the robot 100 is the index number "2", and the emotion of the user 10 is the index number "3", "the robot is in a very happy state. The user is in a generally happy state. The user was interrupted by "please tell me the weakness of the opposing team". How should the robot respond?" is input to the article generation model, and the behavior content of the robot is acquired. The behavior determination section 236 determines the behavior of the robot in accordance with the behavior content.

[0212] ​In this way, the behavior determination unit 236 determines the behavior content of the robot 100 in correspondence with the state related to the robot 100's emotion, which is predetermined according to each type of emotion and each intensity of that emotion, and the behavior of the user 10. In this manner, the robot 100's speech during dialogue with the user 10 can branch according to the state related to the robot 100's emotion. That is, since the robot 100 can change its behavior according to the index number corresponding to the robot's emotion, the user gets the impression that the robot has a mind, which encourages them to take actions such as striking up a conversation with the robot.

[0213] Furthermore, when the behavior determination unit 236 receives a question related to the competitor as described above, it can also use the determination result of the feature determination unit to determine the robot's behavior. Specifically, as an answer to the question from the user, it can, for example, state a determination such as "The opposing team's number 2 player made the most mistakes today."

[0214] (Note 1) A behavior control system, comprising: The emotion determination department determines the user's or the robot's emotions; and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, generates behavioral content for the robot based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavioral content. The behavior determination unit includes: The image acquisition unit is capable of capturing images of a competition space where a specific competition can be conducted; and The feature determination unit determines the features of multiple competitors engaged in competition within the competition space captured by the image acquisition unit. The robot's behavior is determined based on the determination result of the feature determination unit.

[0215] (Note 2) According to the behavior control system described in Appendix 1, the feature determination unit analyzes the features of competitors belonging to a specific team among the plurality of competitors.

[0216] (Note 3) According to the behavior control system described in Appendix 1 or Appendix 2, the robot is mounted on a plush toy or connected wirelessly or wiredly to a controlled object machine mounted on the plush toy.

[0217] (Other implementation method 3) The robot 100 of the present embodiment has a time counting function and is configured to be activated at a prescribed time. The behavior determination section 236 generates a behavior content of the robot with respect to a behavior of the user and an emotion of the user or an emotion of the robot on the basis of an article generation model having a dialogue function for the user to dialogue with the robot, determines a behavior of the robot corresponding to the behavior content, and this time, when the robot is activated at the prescribed time, adds a fixed sentence for indicating a summary of the history of the previous day to a text representing the history data of the previous day and inputs the text to the article generation model, thereby acquiring a summary of the history of the previous day, and speaks the acquired summary content.

[0218] Specifically, the behavior determination section 236, when the robot is activated in the morning (for example, from 5 to 9), adds a fixed sentence such as "summarize this content" to a text representing the history data of the previous day and inputs the text to the article generation model, thereby acquiring a summary of the history of the previous day, and speaks the acquired summary content.

[0219] The emotion determination section 232 can determine the emotion of the user in accordance with a specific mapping. Specifically, the emotion determination section 232 can determine the emotion of the user in accordance with an emotion map as a specific mapping (refer to FIG. 6). Figure 5

[0220] The behavior determination section 236 generates a behavior content of the robot by adding a fixed sentence for asking the behavior content of the robot corresponding to the behavior of the user to a text representing the behavior of the user, the emotion of the user, and the emotion of the robot and inputs the text to an article generation model having a dialogue function, thereby generating the behavior content of the robot.

[0221] For example, the behavior determination section 236 acquires a text representing the state of the robot 100 on the basis of the emotion of the robot 100 determined by the emotion determination section 232 using the emotion table shown in Table 1. Here, in the emotion table, each emotion value is given an index number with respect to each kind of emotion, and for each index number, a text representing the state of the robot 100 is saved.

[0222] In a case where the emotion of the robot 100 determined by the emotion determination section 232 corresponds to the index number "2", a text of "a very happy state" is obtained. In addition, in a case where the emotion of the robot 100 corresponds to a plurality of index numbers, a plurality of texts representing the state of the robot 100 are obtained.

[0223] In addition, with respect to the emotion of the user 10, an emotion table shown in Table 2 is prepared.

[0224] ​Here, given that the user's behavior is to strike up a conversation with "XXX", the robot 100's emotion is index number "2", and the user 10's emotion is index number "3", the article generation model is input with the question: "The robot is in a very happy state. The user is in a normally happy state. The user struck up a conversation with 'XXX'. As the robot, how should you respond?" to obtain the robot's behavior content. The behavior determination unit 236 determines the robot's behavior based on this behavior content.

[0225] In this way, the behavior determination unit 236 determines the behavior content of the robot 100 in correspondence with the state related to the robot 100's emotion, which is predetermined according to each type of emotion and each intensity of that emotion, and the behavior of the user 10. In this manner, the robot 100's speech during dialogue with the user 10 can branch according to the state related to the robot 100's emotion. That is, since the robot 100 can change its behavior according to the index number corresponding to the robot's emotion, the user gets the impression that the robot has a mind, which encourages them to take actions such as striking up a conversation with the robot.

[0226] (Note 1) A behavior control system, comprising: The emotion determination department determines the user's or the robot's emotions; and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, generates behavioral content for the robot based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavioral content. When the behavior determination unit is activated at a specified time, it obtains a summary of the previous day's history by adding a fixed sentence to the text representing the previous day's historical data and inputting it into the article generation model. The summary content obtained is then presented.

[0227] (Note 2) According to the behavior control system described in Appendix 1, the robot is mounted on a plush toy or connected wirelessly or wiredly to a controlled object machine mounted on the plush toy.

[0228] (Other implementation method 4) The robot 100 of this embodiment has a timing function and is configured to start at a predetermined time. Furthermore, the robot 100 of this embodiment has an image generation model configured to generate an image based on an input sentence. The behavior determination unit 236, based on an article generation model having a dialogue function that allows the user to converse with the robot, generates behavioral content for the robot based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavioral content. At this time, when the behavior determination unit 236 starts at the predetermined time, it adds a fixed sentence to the text representing the previous day's historical data to indicate a summary of the previous day's history, and inputs it into the article generation model to obtain a summary of the previous day's history. By inputting the obtained summary of the previous day's history into the image generation model, it obtains an image summarizing the previous day's history and displays the obtained image.

[0229] Specifically, when the behavior determination unit 236 is activated in the morning (e.g., from 5 a.m. to 9 a.m.), it adds a fixed sentence such as "summarize the content" to the text representing the historical data of the previous day and inputs it into the article generation model to obtain a historical summary of the previous day. By inputting the obtained historical summary of the previous day into the image generation model, it obtains an image that summarizes the history of the previous day and displays the obtained image.

[0230] In this way, the behavior determination unit 236 determines the behavior content of the robot 100 in correspondence with the state related to the robot 100's emotion, which is predetermined according to each type of emotion and each intensity of that emotion, and the behavior of the user 10. In this manner, the robot 100's speech during dialogue with the user 10 can branch according to the state related to the robot 100's emotion. That is, since the robot 100 can change its behavior according to the index number corresponding to the robot's emotion, the user gets the impression that the robot has a mind, which encourages them to take actions such as striking up a conversation with the robot.

[0231] (Note 1) A behavior control system, comprising: The emotion determination department determines the user's or the robot's emotions; and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, generates behavioral content for the robot based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavioral content. The behavior determination section, when activated at a prescribed time, acquires a summary of the previous day's history by adding a fixed sentence indicating a summary of the previous day's history to a text representing the previous day's history data and inputting the text to the article generation model, acquires an image summarizing the previous day's history by inputting the acquired summary of the previous day's history to an image generation model, and displays the acquired image.

[0232] (Addendum 2) The behavior control system according to Addendum 1, wherein the robot is mounted on a stuffed toy, or is connected wirelessly or by wire to a control target robot mounted on a stuffed toy.

[0233] (Other Embodiment 5) The robot 100 of the present embodiment has a timekeeping function and is configured to be activated at a prescribed time.

[0234] The behavior determination section 236 generates a behavior content of the robot with respect to a behavior of the user and an emotion of the user or the robot on the basis of an article generation model having a dialogue function with which the user dialogues with the robot, and determines a behavior of the robot corresponding to the behavior content. At this time, the behavior determination section 236, when activated at a prescribed time, adds a fixed sentence for asking an emotion that the user should have to a text representing history data of the previous day, and inputs the text to the article generation model, thereby acquiring an emotion of the robot corresponding to the previous day's history.

[0235] Specifically, the behavior determination section 236, when activated in the morning (for example, from 5 to 9 o'clock), adds a fixed sentence such as "What kind of emotion should the robot have this morning?" to a text representing history data of the previous day, and inputs the text to the article generation model, thereby determining an emotion of the robot on the basis of the previous day's history.

[0236] The emotion determination section 232 can determine an emotion of the user in accordance with a specific mapping. Specifically, the emotion determination section 232 can determine an emotion of the user in accordance with an emotion map (refer to FIG. 6) as a specific mapping. Figure 5

[0237] ​Thus, the behavior determination section 236 determines the behavior content of the robot 100 in correspondence with the state related to the emotion of the robot 100 and the behavior of the user 10 that are prescribed in advance for each kind of emotion of the robot 100 and for each intensity of the emotion. In this way, the utterance content of the robot 100 when conversing with the user 10 can be branched according to the state related to the emotion of the robot 100. That is, since the robot 100 can change the behavior of the robot according to the index number corresponding to the emotion of the robot, the user has an impression that the robot has a mind, and this promotes the user to take the behavior of striking up a conversation with the robot and the like.

[0238] (Note 1) A behavior control system includes: an emotion determination section that determines an emotion of a user or an emotion of a robot; and a behavior determination section that generates a behavior content of a robot for a behavior of a user and an emotion of the user or an emotion of the robot based on an article generation model having a conversation function with which the user converses with the robot, and determines a behavior of the robot corresponding to the behavior content, the behavior determination section determines the emotion of the robot corresponding to a history of the day before when the behavior determination section is activated at a prescribed time by adding a fixed sentence for inquiring about an emotion that the robot should have to a text representing the history of the day before and inputting to the article generation model.

[0239] (Note 2) The behavior control system according to Note 1, in which the robot is mounted on a stuffed toy, or is connected to a control target robot mounted on a stuffed toy in a wireless or wired manner.

[0240] (Other Embodiment 6) The behavior determination section 236 can, for example, at the time when the user gets up, add a fixed sentence inquiring about the emotion of the user, such as "What kind of emotion does the user have?", to the history data including the history of the behavior and the emotion of the user of the day before, input the history data with the added text to the conversation function, and determine the emotion of the user by the conversation function on the basis of the history of the day before of the user.

[0241] In this scenario, the behavior determination unit 236 determines the robot 100's behavior based on the determined user emotions. For example, if the user's behavior and emotional history from the previous day was happy, then the user's emotion is cheerful. Therefore, the behavior determination unit 236 determines the robot 100's behavior based on dialogue functions or response rules to generate behaviors and language that reflect happiness. Conversely, if the user's behavior and emotional history from the previous day was sad, then the user's emotion is somber. Therefore, the behavior determination unit 236 determines the robot 100's behavior based on dialogue functions or response rules to produce behaviors and language that can encourage the user.

[0242] In addition, the robot 100 can be mounted on plush toys or used in control devices that connect wirelessly or wiredly to the controlled object machine (speaker or camera) mounted on the plush toy.

[0243] (Note 1) A behavior control system, comprising: The emotion determination department determines the user's or the robot's emotions; and The behavior determination unit, based on the dialogue function that enables the user to converse with the robot, generates robot behavior content based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavior content. The behavior determination unit determines the emotion based on the user's previous day's history by adding historical data, including the user's behavior and emotions from the previous day, to a fixed sentence that asks the user about their emotions based on the historical data and inputting it into the dialogue function when the user wakes up.

[0244] (Note 2) According to the behavior control system described in Appendix 1, the robot is mounted on a plush toy or connected wirelessly or wiredly to a controlled object machine mounted on the plush toy.

[0245] (Other implementation method 7) The behavior determination unit 236 can also, at the moment the user wakes up, add a fixed sentence, "Please summarize this content," to the text containing the user's historical data of the previous day's behavior and emotions, and input it into the article generation model. The article generation model then obtains a summary of the user's historical data from the previous day, i.e., a summary of the user's behavior and emotions from the previous day. Then, the behavior determination unit 236 can also input the obtained summary into the music generation engine, obtain music summarizing the user's previous day's behavior and emotions from the music generation engine, and play the obtained music. For example, if the summary of the user's previous behavior is "Yesterday was my girlfriend's birthday, we ate at a fancy restaurant, and I had a great time," then the music generation engine adds a melody to the summary to generate music most suitable for the user's previous behavior and emotions. The behavior determination unit 236 obtains the music generated by the music generation engine and plays it. Thus, the user can review their previous day's behavior and emotions while waking up using music.

[0246] (Note 1) A behavior control system, comprising: The emotion determination department determines the user's or the robot's emotions; and The behavior determination unit, based on the dialogue function that enables the user to converse with the robot, generates robot behavior content based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavior content. The behavior determination unit acquires a summary of historical data, including the user's behavior and emotions from the previous day, at the time the user wakes up, acquires music based on the summary, and plays the music.

[0247] (Other implementation methods 8) The behavior system of the robot 100 in other embodiments is characterized by including: an emotion determination unit that determines the emotion of a user or the emotion of a robot; and a behavior determination unit that, based on a dialogue function that enables the user to converse with the robot, generates behavioral content for the robot based on the user's behavior and the user's emotion or the robot's emotion, determines the robot's behavior corresponding to the behavioral content, and when the behavior determination unit engages in a battle game with the user that assigns a win or loss or advantage or disadvantage, determines a user level that represents the user's strength in the battle game, and sets a robot level that represents the robot's strength based on the determined user level.

[0248] When User 10 and Robot 100 play a card game or similar competitive game, Robot 100 stores User 100's strength index (user level). Compared to User 10's strength index (user level), a slightly weaker computer strength index (robot level) is set. In the card game against User 10, User 10 will win, but not completely.

[0249] That is, the robot 100 can judge the situation of the game that the user 10 can enjoy and advance the game.

[0250] The situation in which the user 10 is bound to win but not to win completely can be realized by setting the robot level lower than the user level. Thereby, the game in which the user is advantageous can be maintained. In addition, it is preferable to give a clear advantage or disadvantage even in a one game of win a few times and lose once.

[0251] In addition, the robot level can also be adjusted based on the emotion of the user 10 who is executing the game such as poker.

[0252] The user 10 has different emotions at the time of the game, such as the emotion of wanting to win no matter what, wanting to enjoy the game regardless of win or lose, and wanting to lose completely. Therefore, by adjusting the robot level based on the emotion of the user 10 at the time of the game, the game can be advanced in accordance with the emotion of the user 10 at the time of the game.

[0253] In addition, by storing the skill of the user each time the game such as poker is executed, so-called big data of the user 10 for the game is constructed. For example, by learning using the stored big data, it is possible to predict the degree of progress of the user 10 in the future and the like in accordance with the progress of the skill of the user 10, and it is possible to set the robot level in accordance with the prediction.

[0254] The user level and the robot level are preferably expressed (informed to the user 10) by each numerical value (for example, 0 to 100 or the like), but can also be classified in segments by setting ranges in the numerical value, such as "weakest (0 to 20)", "weak (21 to 40)", "normal (41 to 60)", "strong (61 to 80)", and "strongest (81 to 100)". In addition, in the case of expressing the robot level visually, it is possible to display the numerical value of the robot level on the display portion of the robot 100, or to express the robot level by changing the shape or the hue (color tone feeling), the lightness (brightness), the chroma (vividness) of a part (for example, eyes, face, hands, feet, and the like) of the robot 100.

[0255] The emotion determination portion 232 can determine the emotion of the user in accordance with a specific mapping. Specifically, the emotion determination portion 232 can determine the emotion of the user in accordance with an emotion map (refer to FIG. 6) as a specific mapping. Figure 5

[0256] The behavior determination portion 236 generates the behavior content of the robot by inputting, to an article generation model having a dialogue function, a fixed sentence for asking the behavior content of the robot corresponding to the behavior of the user by adding the fixed sentence to the text representing the behavior of the user, the emotion of the user, and the emotion of the robot.

[0257] ​For example, the behavior determination section 236 acquires a text indicating the state of the robot 100 using the emotion table shown in Table 1, among the emotions of the robot 100 determined by the emotion determination section 232. Here, in the emotion table, each emotion value is given an index number for each kind of emotion, and for each index number, a text indicating the state of the robot 100 is saved.

[0258] In a case where the emotion of the robot 100 determined by the emotion determination section 232 corresponds to the index number "2", a text of "a very happy state" is obtained. In addition, in a case where the emotion of the robot 100 corresponds to a plurality of index numbers, a plurality of texts indicating the state of the robot 100 are obtained.

[0259] In addition, for the emotion of the user 10, an emotion table shown in Table 2 is prepared. Here, the robot 100 is in the customer conversation mode, and the behavior of the user 10 does not say to the other person family, friends, and lovers, and the like, but makes the robot 100 listen to the content of who is said. At this time, in a case where the emotion of the robot 100 is the index number "2" and the emotion of the user 10 is the index number "3", "the robot is in a very happy state. The user is in a generally happy state. The user was said "Tomorrow is my wife's birthday. What about a gift?". As a robot, how to reply?" is input to the article generation model, and the behavior content of the robot is acquired. The behavior determination section 236 determines the behavior of the robot in accordance with the behavior content.

[0260] In this way, since the robot 100 can change the behavior of the robot in accordance with the index number corresponding to the emotion of the robot, the user has an impression that the robot has a mind, and the behavior of approaching the robot and the like is promoted.

[0261] In addition, the behavior determination section 236 can generate the behavior content of the robot by adding not only a text indicating the behavior of the user, the emotion of the user, and the emotion of the robot, but also a text indicating the content of the history data 2222, and then adding a fixed sentence for asking the behavior content of the robot corresponding to the behavior of the user, and inputting an article generation model having a conversation function. Thus, since the robot 100 can change the behavior of the robot in accordance with the history data indicating the emotion or the behavior of the user, the user has an impression that the robot has individuality, and the behavior of approaching the robot and the like is promoted. Furthermore, the emotion or the behavior of the robot can be further included in the history data.

[0262] (Postscript 1) A behavior control system including: an emotion determination section that determines an emotion of a user or an emotion of a robot; and The behavior determination section determines a behavior of the robot corresponding to a behavior content of the robot generated based on an article generation model having a dialogue function with which the user dialogues with the robot, for a behavior of the user and an emotion of the user or an emotion of the robot, The behavior determination section determines a user level indicating a strength of the user in the battle game when a battle game in which a win or a loss is given between the user and the robot is being played, and sets a robot level indicating a strength of the robot in accordance with the determined user level.

[0263] (Note 2) The behavior control system according to Note 1, in which the robot is mounted on a stuffed toy, or connected wirelessly or by wire with a control target robot mounted on a stuffed toy.

[0264] (Note 3) The behavior control system according to Note 1, in which the robot level is set lower than the user level.

[0265] (Note 4) The behavior control system according to Note 1, in which the robot level is adjusted based on an emotion of the user during execution of the battle game.

[0266] (Note 5) The behavior control system according to Note 1, in which a skill of the user is stored each time the battle game is executed, and the user level is determined based on the stored skill of the user over time.

[0267] (Other Embodiment 9) The robot 100 according to an embodiment of the present disclosure includes an emotion determination section that determines an emotion of a user or an emotion of the robot 100, and a behavior determination section that determines a behavior of the robot 100 corresponding to a behavior content of the robot 100 generated based on a dialogue function with which the user dialogues with the robot 100, for a behavior of the user and the emotion of the user or the emotion of the robot 100. The behavior determination section 236 selects at least one of two or more things to answer the user from the two or more things based on at least historical information related to the user, in a case where a question is received from the user about which of the two or more things should be selected.

[0268] The things include a thing that the user is interested in, a thing that the user can be interested in, a thing that the user likes, and the like.

[0269] The question is "Which of the clothes in A and B is more suitable?", "If it were you, which of the clothes in A and B would you choose?", "Where is a good hotel for a party?", "Which route to the destination will make the family happy?", and the like.

[0270] The history information about the user can include the user's personality, interests, preferences, past actions, and the like. The history information about the user can include the user's name, age, sex, occupation, income, assets, educational background, place of residence, family composition, demand items, disease history, family composition, and the like. These history information can be stored in a specific database.

[0271] The behavior determination section 236 identifies the user who is the source of the question in a case where a question about which of two or more things should be selected is received from the user, determines history information corresponding to the identified user from the database. The behavior determination section 236 determines the user's characteristics, features, and the like in accordance with the history information about the specified user. The behavior determination section 236 determines the user's characteristics, features, and the like, for example, in accordance with the user's age, annual income, personality, assets, preferences, and the like. The behavior determination section 236 can select at least one of the two or more things using the features and the like of the specified user.

[0272] (Case 1) For a question about which of a product A' similar to a product A that the user has purchased many times in the past and a product A" should be purchased by the user, the behavior determination section 236 can select the product A' that is inexpensive but has higher functionality than the product A in view of the user's age, assets (an example of the determined features and the like), and cause the robot 100 to play a voice recommending the product A'. In this case, the behavior determination section 236 can also recommend the product A" that is higher in price than the inexpensive product A' but has high brand power at the same time. The behavior determination section 236 can also attach a voice indicating the reason to the product A' that is recommended first and the product A" that is recommended next, respectively. The behavior determination section 236 can also recommend the product A' first in view of the user's preference change tendency (an example of the determined features and the like). The behavior determination section 236 can also transmit an image of the product A' and the product A", an image showing the specifications of these products, and the like to a terminal used by the user. Thus, since the detailed information of the recommended products is displayed on the terminal, the user can easily understand the product contents without investigating these products.

[0273] (Case 2) In response to a question about a route (road) that the user has passed while driving a vehicle, the behavior determination section 236 can select the route A that is flat and has the shortest travel distance because the vehicle has a small fuel consumption amount, and cause the robot 100 to play a voice that recommends the route A, in view of the kind of vehicle that the user has (an example of the determined characteristics and the like). In this case, the behavior determination section 236 can also recommend the route B that has a larger fuel consumption amount of the car but enables the user to enjoy driving at the same time. The behavior determination section 236 can also attach a voice that indicates the reason to the route A that is recommended first and the route B that is recommended next, respectively. The behavior determination section 236 can also recommend the route B first in view of the tendency of the user's preference to change (an example of the determined characteristics and the like). The behavior determination section 236 can also transmit the route maps of the route A and the route B to a terminal used by the user. The behavior determination section 236 can also transmit moving images that record the actual travel scenery of the route A and the route B to the terminal in addition to the route maps of the route A and the route B. Thus, since detailed information of the recommended routes is displayed on the terminal, the user can imagine the atmosphere and the feeling at the time of travel even if the user does not know the routes.

[0274] The behavior determination section 236 can select at least one from two or more things to answer in accordance with the history information about the user and the information about society issued from a plurality of information sources. The information about society can include at least one of news, an economic situation, a social situation, a political situation, a financial situation, an international situation, sports news, entertainment news, a birth obituary news, a cultural situation, and a trend.

[0275] For example, the behavior determination section 236 can select a commodity that is likely to increase in price in the future in accordance with the information about the economic situation, the social situation, the financial situation, and the like, and propose the commodity to the user. In addition, the behavior determination section 236 can select a commodity that is likely to be needed in the future in accordance with the information about the news, the trend, and the like, and propose the commodity to the user. In addition, the behavior determination section 236 can select a sightseeing spot that is likely to increase in popularity in the future in accordance with the information about the cultural situation, the entertainment news, and the like, and propose the sightseeing spot to the user.

[0276] The behavior control system according to the embodiment of the present disclosure can select and present any one of the options in a case where the user is forced to make a selection, on the basis of all the history data of the user's character, preference, past behavior, and the like, a trend, a world situation, and the like, and thus can recommend a thing that is suitable for the user for the user who has difficulty in selecting a thing.

[0277] The emotion determination section 232 can determine the user's emotion in accordance with a specific mapping. Specifically, the emotion determination section 232 can determine the user's emotion in accordance with an emotion map (see FIG. 2) as a specific mapping. Figure 5

[0278] ​In addition, a feeling table shown in Table 2 is prepared for the user 10. Here, in a case where the user's behavior is the chat-up "Which clothes should I choose?", the robot 100's feeling is the index number "2", and the user 10's feeling is the index number "3", the robot is in a very happy state. The user is in a generally happy state. The user is chatted up "Which clothes should I choose?". How should the robot reply as a robot?" is input to the article generation model, and the robot's behavior content is acquired. The behavior determination section 236 determines the robot's behavior based on the behavior content.

[0279] (NOTE 1) A behavior control system includes: a feeling determination section that determines a user's feeling or a robot's feeling; and a behavior determination section that generates a robot's behavior content for a user's behavior and the user's feeling or the robot's feeling based on a conversation function that causes the user to converse with the robot, determines the robot's behavior corresponding to the behavior content, the behavior determination section selects at least one from two or more things and answers the user based on at least historical information related to the user in a case where a question about which of two or more things should be selected is received from the user.

[0280] (NOTE 2) The behavior control system according to Note 1, wherein the behavior determination section selects at least one from two or more things to answer based on historical information related to the user and information related to society issued from a plurality of information sources.

[0281] (NOTE 3) The behavior control system according to Note 1, wherein the robot is mounted on a stuffed toy, or is connected to a control target robot mounted on a stuffed toy in a wireless or wired manner.

[0282] (OTHER EMBODIMENT 10) The robot 100 according to an embodiment of the present disclosure includes a feeling determination section that determines a user's feeling or a robot's feeling, and a behavior determination section 236 that generates a robot's behavior content for a user's behavior and the user's feeling or the robot 100's feeling based on a conversation function that causes the user to converse with the robot 100, determines the robot 100's behavior corresponding to the behavior content. The behavior determination section stores a kind of behavior performed by the user within the home as specific information associated with a timing of performing the behavior, determines a timing of performing as a timing at which the user should perform the behavior based on the specific information, and notifies the user.

[0283] The behavior performed by the user in the home can include housework, nail clipping, watering of a plant, going-out preparation, animal walking, and the like. The housework can include toilet cleaning, meal preparation, bathroom cleaning, clothes collection, floor cleaning, child care, shopping, garbage disposal, room airing, and the like.

[0284] The behavior determination section 236 can store these behaviors as specific information corresponding to the timing of performing the behavior. Specifically, the user information of the user (person) included in the specific home, information indicating the kind of behavior performed by the user in the home, and the past timing of performing each of these behaviors are established in correspondence and stored. The past timing can be the number of times of behavior performance at least once or more.

[0285] (1) In a case where the husband of the home performs nail clipping, the behavior determination section 236 records the past nail clipping action by monitoring the behavior of the husband, and records the timing of performing nail clipping (the time of starting nail clipping, the time of ending nail clipping, and the like). The behavior determination section 236 estimates the interval of nail clipping by the husband (for example, the number of days of 10 days, 20 days, and the like) from the timing of performing nail clipping by each person who performed nail clipping, by recording the past nail clipping action a plurality of times. In this way, the behavior determination section 236 can estimate the timing of performing nail clipping next time by recording the timing of performing nail clipping, and notify the user of the timing of performance when the estimated number of days elapses from the time of performing nail clipping last time. Specifically, the behavior determination section 236 can enable the user to grasp the timing of performing nail clipping by causing the robot 100 to play a sound such as "Isn't it about time to clip your nails?", "Your nails might be long", and the like.

[0286] (2) In a case where the wife of the home waters a tree, the behavior determination section 236 records the past watering action by monitoring the behavior of the wife, and records the timing of performing watering (the time of starting watering, the time of ending watering, and the like). The behavior determination section 236 estimates the interval of watering by the wife (for example, the number of days of 10 days, 20 days, and the like) from the timing of performing watering by each person who performed watering, by recording the past watering action a plurality of times. In this way, the behavior determination section 236 can estimate the timing of performing watering next time by recording the timing of performing watering, and notify the user of the timing of performance when the estimated number of days elapses from the time of performing watering last time. Specifically, the behavior determination section 236 can enable the user to grasp the timing of performing watering by causing the robot 100 to play a sound such as "Isn't it about time to water the tree?", "The water for the tree might be low", and the like.

[0287] (3) In a case where a child in the family has performed toilet cleaning, the behavior determination section 236 records past toilet cleaning actions by monitoring the child's behavior, and records the timing of performing toilet cleaning (the time of starting toilet cleaning, the time of finishing toilet cleaning, etc.). The behavior determination section 236 estimates the interval of toilet cleaning by the child (for example, the number of days of 7 days, 14 days, etc.) from the timing of performing toilet cleaning by each person who has performed toilet cleaning by repeatedly recording past toilet cleaning actions. In this way, the behavior determination section 236 can estimate the timing of performing toilet cleaning next time when the estimated number of days has passed from the timing of performing toilet cleaning last time, by recording the timing of performing toilet cleaning, and notify the user of the timing. Specifically, the behavior determination section 236 can enable the user to grasp the timing of performing toilet cleaning by causing the robot 100 to play a sound such as "Isn't it about time to clean the toilet?" or "The time of cleaning the toilet can be close", etc.

[0288] (4) In a case where a child in the family has performed preparation for going out, the behavior determination section 236 records past preparation actions by monitoring the child's behavior, and records the timing of performing preparation (the time of starting preparation, the time of finishing preparation, etc.). The behavior determination section 236 estimates the timing of performing preparation by the child (for example, if it is a weekday, the time of going out for school, and if it is a holiday, the time of going out for study) from the timing of performing preparation by each person who has performed preparation by repeatedly recording past preparation actions. In this way, the behavior determination section 236 can estimate the timing of performing preparation next time by recording the timing of performing preparation, and notify the user of the estimated timing. Specifically, the behavior determination section 236 can enable the user to grasp the timing of performing preparation by causing the robot 100 to play a sound such as "Isn't it about time to go to the cram school?" or "Isn't it a morning exercise day today?", etc.

[0289] The behavior determination section 236 can also notify the timing of performing next time a plurality of times at a certain interval. Specifically, the behavior determination section 236 can notify the user of the timing of performing once or a plurality of times in a case where the user does not perform the behavior after the timing of performing has been notified to the user. That is, the behavior determination section 236 can also notify the timing of performing again. Thereby, even in a case where the user is notified of the timing of performing but forgets and does not perform the behavior, the determined behavior can be performed. In addition, since the user cannot immediately perform the determined behavior, even in a case where the timing of performing is temporarily reserved, the determined behavior can be performed without forgetting.

[0290] The behavior determination unit 236 can also notify the user in advance of the timing of the action, based on a certain period of time elapsed since the last action. For example, if the next watering is scheduled for a specific day 20 days after the last watering, the behavior determination unit 236 can issue a notification urging the next watering a few days before that specific day. Specifically, the behavior determination unit 236 can enable the user to understand the timing of watering by having the robot 100 play announcements such as "The time to water the trees is approaching" or "It's about time to water the trees."

[0291] By configuring the robot 100 in this way, the robot 100 installed in the home can store all the behaviors of the user's family members, and provide appropriate timing for all behaviors such as when is the best time to trim nails, when it's almost time to water the plants, when it's almost time to clean the toilet, and when it's almost time to start preparing.

[0292] (Note 1) A behavior control system, comprising: The emotion determination department determines the user's or the robot's emotions; and The behavior determination unit, based on the dialogue function that enables the user to converse with the robot, generates robot behavior content based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavior content. The behavior determination unit stores the types of behaviors performed by the user within the home as specific information associated with the timing of performing the behavior. Based on the specific information, it determines the timing of the behavior when the user should perform it and notifies the user.

[0293] (Note 2) According to the behavior control system described in Appendix 1, after the behavior determination unit notifies the user of the execution opportunity, it notifies the user of the execution opportunity again if the user does not perform the behavior.

[0294] (Note 3) According to the behavior control system described in Appendix 1, the robot is mounted on a plush toy or connected wirelessly or wiredly to a controlled object machine mounted on the plush toy.

[0295] (Other implementation method 11) The robot 100 of the present embodiment includes an emotion determination section that determines the emotion of the user or the emotion of the robot, and a behavior determination section that generates a behavior content of the robot with respect to the behavior of the user and the emotion of the user or the emotion of the robot on the basis of a conversation function that causes the user to converse with the robot, and determines a behavior of the robot corresponding to the behavior content. The behavior determination section is configured to receive utterances of a plurality of users who are conversing, to output a topic of the conversation and to output another topic in accordance with the emotion of at least one of the users who are conversing as the behavior of the robot.

[0296] Specifically, the robot 100 is disposed at a party or a gathering, a blind date venue, or the like. Also, the robot 100 of the present embodiment acquires utterances of a plurality of users who are conversing through a microphone function.

[0297] The behavior determination section 236 outputs a topic related to the acquired utterance content from the speaker in accordance with the utterance content. For example, when a sports conversation is in progress, a topic that expands the topic by conveying the latest game results or the like is output.

[0298] In addition, in a case where the emotion of one of the users who are conversing is in a predetermined state, such as an emotion value of "unhappiness", "emptiness", or the like is negative, a different topic is output from the topic of the conversation so far, such as "By the way, are you interested in...?", or the like. Here, the different topic is a topic that is different from the stored topic so far, and includes a topic of a recent event, a topic that the user is likely to like, which is judged on the basis of attribute information of the user (for example, information that can be acquired through a camera function, such as age, gender, or the like, information that is input at the time of reservation of the venue, such as occupation, residential area, family composition, or the like), or the like.

[0299] Further, the robot 100 can also accept a question of "What is a good topic?" from the user in addition to the acquisition of the conversation, and provide a topic.

[0300] By being configured in this way, the robot 100 assists the conversation in a party or a gathering, a blind date, or the like, and can prevent the user from feeling embarrassed due to the interruption of the conversation, or feeling unhappy by continuing the conversation of a topic that the user does not want to talk about.

[0301] The emotion determination section 232 can determine the emotion of the user in accordance with a specific mapping. Specifically, the emotion determination section 232 can determine the emotion of the user in accordance with an emotion map (refer to FIG. 6) as a specific mapping. Figure 5

[0302] ​In this way, the behavior determination unit 236 determines the behavior content of the robot 100 in correspondence with the state related to the robot 100's emotion, which is predetermined according to each type of emotion and each intensity of that emotion, and the behavior of the user 10. In this manner, the robot 100's speech during dialogue with the user 10 can branch according to the state related to the robot 100's emotion. That is, since the robot 100 can change its behavior according to the index number corresponding to the robot's emotion, the user gets the impression that the robot has a mind, which encourages them to take actions such as striking up a conversation with the robot.

[0303] (Note 1) A behavior control system, comprising: The emotion determination department determines the user's or the robot's emotions; and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, generates behavioral content for the robot based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavioral content. The behavior determination unit receives the statements of multiple users who are having a conversation, outputs the topic of the conversation, and outputs other topics based on the emotions of at least one of the users who are having the conversation to determine the robot's behavior.

[0304] (Note 2) According to the behavior control system described in Appendix 1, the topic is determined based on the user's attribute information.

[0305] (Note 3) According to the behavior control system described in Appendix 1 or 2, the robot is mounted on a plush toy or connected wirelessly or wiredly to a controlled object machine mounted on the plush toy.

[0306] (Second Implementation) Figure 9 The functional structure of robot 100 is shown in general. Robot 100 includes a sensor unit 200, a sensor module unit 210, a storage unit 220, a control unit 228, and a controlled object 252. The control unit 228 includes a state recognition unit 230, an emotion determination unit 232, a behavior recognition unit 234, a behavior determination unit 236, a storage control unit 238, a behavior control unit 250, an associated information collection unit 270, and a communication processing unit 280.

[0307] The control targets 252 include display devices, speakers, and LEDs of the eyes, and motors that drive arms, hands, feet, and the like. The posture or the behavior of the robot 100 is controlled by controlling the motors of the arms, the hands, the feet, and the like. Part of the emotions of the robot 100 can be expressed by controlling these motors. In addition, by controlling the light emission state of the LEDs of the eyes of the robot 100, the expression of the robot 100 can also be expressed. In addition, the posture, the behavior, and the expression of the robot 100 are examples of the attitude of the robot 100.

[0308] The sensor section 200 includes a microphone 201, a 3D depth sensor 202, a 2D camera 203, a distance sensor 204, a touch sensor 205, and an acceleration sensor 206. The microphone 201 continuously detects sound and outputs sound data. In addition, the microphone 201 can be provided at the head of the robot 100, and has a function of performing binaural recording. The 3D depth sensor 202 continuously irradiates an infrared pattern, and detects the outline of an object by analyzing the infrared pattern based on an infrared image continuously captured with an infrared camera. The 2D camera 203 is an example of an image sensor. The 2D camera 203 captures with visible light, and generates video information of the visible light. The distance sensor 204 detects the distance to an object, for example, by irradiating laser light or ultrasonic waves. In addition, the sensor section 200 can include a clock, a gyro sensor, a touch sensor, a motor feedback sensor, and the like, in addition to these.

[0309] In addition, among the constituent elements of the robot 100 illustrated in Figure 9 Among the constituent elements of the robot 100 illustrated in

[0310] The storage section 220 includes a behavior determination model 221A, history data 2222, collection data 2230, and behavior schedule data 224. The history data 2222 includes past emotional values of the user 10, past emotional values of the robot 100, and a behavior history, and specifically, includes a plurality of event data including the emotional value of the user 10, the emotional value of the robot 100, and the behavior of the user 10. The data including the behavior of the user 10 includes a camera image that represents the behavior of the user 10. The history of the emotional value and the behavior is recorded for each user 10, for example, by corresponding to the identification information of the user 10. At least a part of the storage section 220 is realized by a storage medium such as a memory. A person DB that stores the face image of the user 10, attribute information of the user 10, and the like can also be included. In addition, the storage section 220 can be realized by a server that is connected to the robot 100 via a network. Figure 9Among the constituent elements of the robot 100 shown, the functions of the constituent elements other than the control target 252, the sensor section 200, and the storage section 220 can be implemented by the CPU based on the operation of the program. For example, by the program operating on the basic software (OS) and the OS, the functions of these constituent elements can be implemented as the operation of the CPU.

[0311] The sensor module section 210 includes a voice emotion recognition section 211, a speech understanding section 212, an expression recognition section 213, and a face recognition section 214. The sensor module section 210 is input with information detected by the sensor section 200. The sensor module section 210 analyzes the information detected by the sensor section 200, and outputs the analysis result to the state recognition section 230.

[0312] The voice emotion recognition section 211 of the sensor module section 210 analyzes the voice of the user 10 detected by the microphone 201, and recognizes the emotion of the user 10. For example, the voice emotion recognition section 211 extracts a feature amount such as a frequency component of the voice, and recognizes the emotion of the user 10 based on the extracted feature amount. The speech understanding section 212 analyzes the voice of the user 10 detected by the microphone 201, and outputs text information indicating the speech content of the user 10.

[0313] The expression recognition section 213 recognizes the expression of the user 10 and the emotion of the user 10 from the image of the user 10 taken by the 2D camera 203. For example, the expression recognition section 213 recognizes the expression and the emotion of the user 10 based on the shape, the positional relationship, and the like of the eye portion and the mouth portion.

[0314] The face recognition section 214 recognizes the face of the user 10. The face recognition section 214 recognizes the user 10 by matching the face image stored in the person DB (omitted from the drawing) and the face image of the user 10 taken by the 2D camera 203.

[0315] The state recognition section 230 recognizes the state of the user 10 based on the information analyzed by the sensor module section 210. For example, using the analysis result of the sensor module section 210, mainly processing related to the perception is performed. For example, perception information such as "Dad is alone", "The probability that Dad has no smile is 90%" is generated. Processing to understand the meaning of the generated perception information is performed. For example, meaning information such as "Dad is alone and looks very lonely" is generated.

[0316] The state recognition section 230 recognizes the state of the robot 100 based on the information detected by the sensor section 200. For example, the state recognition section 230 recognizes the remaining battery level of the robot 100, the brightness of the surrounding environment of the robot 100, and the like as the state of the robot 100.

[0317] The emotion determination section 232 determines an emotion value indicating an emotion of the user 10 on the basis of the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230. For example, the information analyzed by the sensor module section 210 and the state of the user 10 recognized are input to a neural network learned in advance, and an emotion value indicating an emotion of the user 10 is acquired.

[0318] Here, the emotion value indicating an emotion of the user 10 is a value indicating the positive or negative of the emotion of the user, and for example, if the emotion of the user is a bright emotion such as "joy", "cheer", "happiness", "comfort", "excitement", "satisfaction", and "fulfillment" that accompanies a pleasant feeling or peace, a positive value is indicated, and the brighter the emotion, the larger the value. If the emotion of the user is an emotion such as "anger", "sorrow", "unhappiness", "unease", "melancholy", "worry", and "emptiness" that becomes a negative emotion, a negative value is indicated, and the more negative the emotion, the larger the absolute value of the negative value. In the case where the emotion of the user is none of the above ( "normal" ), a value of 0 is indicated.

[0319] In addition, the emotion determination section 232 determines an emotion value indicating an emotion of the robot 100 on the basis of the information analyzed by the sensor module section 210, the information detected by the sensor section 200, and the state of the user 10 recognized by the state recognition section 230.

[0320] The emotion value of the robot 100 contains an emotion value for each of a plurality of emotion categories, and for example, is a value indicating the intensity of each of "joy", "anger", "sorrow", and "cheer" (0 to 5).

[0321] Specifically, the emotion determination section 232 determines an emotion value indicating an emotion of the robot 100 in accordance with a rule for updating the emotion value of the robot 100 prescribed in correspondence with the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230.

[0322] For example, the emotion determination section 232 increases the emotion value of "sorrow" of the robot 100 in the case where it is recognized by the state recognition section 230 that the user 10 looks very lonely. In addition, the emotion determination section 232 increases the emotion value of "joy" of the robot 100 in the case where it is recognized by the state recognition section 230 that the user 10 is smiling.

[0323] In addition, the emotion determination section 232 can further take into account the state of the robot 100 to determine an emotion value indicating an emotion of the robot 100. For example, the emotion value of "sorrow" of the robot 100 can be increased in the case where the remaining battery level of the robot 100 is low or the surrounding environment of the robot 100 is pitch dark, or the like. Furthermore, the emotion value of "anger" can be increased in the case where the user 10 continues to talk to the robot 100 despite the low remaining battery level.

[0324] The behavior recognition unit 234 recognizes the behavior of the user 10 on the basis of the information analyzed by the sensor module unit 210 and the state of the user 10 recognized by the state recognition unit 230. For example, the information analyzed by the sensor module unit 210 and the recognized state of the user 10 are input to a neural network learned in advance, a probability of each of a predetermined plurality of behavior classifications (for example, "laugh," "anger," "question," and "sorrow") is obtained, and the behavior classification with the highest probability is recognized as the behavior of the user 10.

[0325] As described above, in the present embodiment, the robot 100 acquires the utterance content of the user 10 on the basis of the determination of the user 10, but in the acquisition and use of the utterance content and the like, the behavior control system of the robot 100 according to the present embodiment takes into account the protection of the personal information and privacy of the user 10 in addition to the acquisition of the necessary consent from the user 10 in accordance with the law.

[0326] Next, the processing of the behavior determination unit 236 at the time of response processing in which the robot 100 responds to the behavior of the user 10 will be described.

[0327] The behavior determination unit 236 determines the behavior corresponding to the behavior of the user 10 recognized by the behavior recognition unit 234 on the basis of the current emotion value of the user 10 determined by the emotion determination unit 232, the history data 2222 of the past emotion value determined by the emotion determination unit 232 before the current emotion value of the user 10 is determined, and the emotion value of the robot 100. In the present embodiment, the behavior determination unit 236 is described with respect to a case in which the most recent one of the emotion values included in the history data 2222 is used as the past emotion value of the user 10, but the disclosed technology is not limited to this aspect. For example, the behavior determination unit 236 can use a plurality of recent emotion values as the past emotion value of the user 10, or can use an emotion value of a unit period ago, such as one day ago, as the past emotion value of the user 10. In addition, the behavior determination unit 236 can determine the behavior corresponding to the behavior of the user 10 further considering not only the current emotion value of the robot 100 but also the history of the past emotion value of the robot 100. The behavior determined by the behavior determination unit 236 includes the posture of the robot 100 or the utterance content of the robot 100.

[0328] The behavior determination unit 236 according to the present embodiment determines the behavior of the robot 100 as the behavior corresponding to the behavior of the user 10 on the basis of the combination of the past emotion value and the current emotion value of the user 10, the emotion value of the robot 100, the behavior of the user 10, and the behavior determination model 221A. For example, in a case in which the past emotion value of the user 10 is a positive value and the current emotion value is a negative value, the behavior determination unit 236 determines a behavior for changing the emotion value of the user 10 to be positive as the behavior corresponding to the behavior of the user 10.

[0329] In the reaction rule as the behavior determination model 221A, a behavior of the robot 100 corresponding to a combination of the past emotional value and the current emotional value of the user 10, the emotional value of the robot 100, and the behavior of the user 10 is prescribed. For example, in a case where the past emotional value of the user 10 is a positive value, and the current emotional value is a negative value, and the behavior of the user 10 is pitiful, as the behavior of the robot 100, a combination of the posture and the utterance content at the time of the inquiry to encourage the user 10 in combination with the posture is prescribed.

[0330] For example, in the reaction rule as the behavior determination model 221A, for all combinations of the emotional value pattern of the robot 100 (6 values of the 4th power of the values "0" to "5" of "joy", "anger", "sorrow", and "cheerfulness", that is, 1296 patterns), the combination of the past emotional value and the current emotional value of the user 10, and the behavior pattern of the user 10, a behavior of the robot 100 is prescribed. That is, for each pattern of the emotional value of the robot 100, for each of a plurality of combinations of the combination of the past emotional value and the current emotional value of the user 10 such as a negative value and a negative value, a negative value and a positive value, a positive value and a negative value, a positive value and a positive value, a negative value and normal, and normal and normal, a behavior of the robot 100 corresponding to the behavior pattern of the user 10 is prescribed. In addition, the behavior determination section 236 can also shift to an operation mode of determining the behavior of the robot 100 using the history data 2222, for example, in a case where the user 10 has made a speech of the dialogue of the intention to continue the past topic "want to say the topic said before".

[0331] In addition, in the reaction rule as the behavior determination model 221A, at least one of the posture and the utterance content can be prescribed as the behavior of the robot 100 one by one at the maximum for each pattern of the emotional value pattern of the robot 100 (1296 patterns). Alternatively, in the reaction rule as the behavior determination model 221A, at least one of the posture and the utterance content can be prescribed as the behavior of the robot 100 for each group of the pattern group of the emotional value of the robot 100.

[0332] The intensity of each posture included in the behavior of the robot 100 prescribed in the reaction rule as the behavior determination model 221A is prescribed in advance. The intensity of each utterance content included in the behavior of the robot 100 prescribed in the reaction rule as the behavior determination model 221A is prescribed in advance.

[0333] The storage control section 238 determines whether or not to store data including the behavior of the user 10 in the history data 2222, on the basis of the intensity of the behavior prescribed in advance for the behavior determined by the behavior determination section 236 and the emotional value of the robot 100 determined by the emotion determination section 232.

[0334] Specifically, in a case where the integrated value of the sum of the emotion values in each of the plurality of emotion classifications of the robot 100, the intensity prescribed in advance for the gesture included in the behavior determined by the behavior determination section 236, and the intensity prescribed in advance for the utterance content included in the behavior determined by the behavior determination section 236, that is, the intensity, is equal to or greater than the threshold value, it is determined to store the data including the behavior of the user 10 in the history data 2222.

[0335] The storage control section 238 stores, in the history data 2222, the behavior determined by the behavior determination section 236, the information analyzed by the sensor module section 210 from the current timing to a certain period before (for example, all of the surrounding information such as the data of the sound, the image, the odor, and the like in the field) and the state of the user 10 (for example, the expression, the emotion, and the like of the user 10) recognized by the state recognition section 230 in a case where it is determined to store the data including the behavior of the user 10 in the history data 2222.

[0336] The behavior control section 250 controls the control object 252 in accordance with the behavior determined by the behavior determination section 236. For example, in a case where the behavior determination section 236 determines the behavior including the utterance, the behavior control section 250 outputs the sound from the speaker included in the control object 252. At this time, the behavior control section 250 can also determine the speed of the utterance of the sound on the basis of the emotion value of the robot 100. For example, the greater the emotion value of the robot 100, the faster the speed of the utterance determined by the behavior control section 250. In this way, the behavior control section 250 determines the execution manner of the behavior determined by the behavior determination section 236 in accordance with the emotion value determined by the emotion determination section 232.

[0337] The behavior control section 250 can recognize the change in the emotion of the user 10 with respect to the execution of the behavior determined by the behavior determination section 236. For example, the change in the emotion can be recognized on the basis of the sound or the expression of the user 10. In addition to this, the change in the emotion of the user 10 can also be recognized on the basis of the detection of the impact by the touch sensor 205 included in the sensor section 200. In a case where the impact is detected by the touch sensor 205 included in the sensor section 200, it can be recognized that the emotion of the user 10 is deteriorated, or in a case where it is judged from the detection result of the touch sensor 205 included in the sensor section 200 that the reaction of the user 10 is laughing or is happy, it can be recognized that the emotion of the user 10 is improved. The information indicating the reaction of the user 10 is output to the communication processing section 280.

[0338] Further, after the behavior control section 250 executes the behavior determined by the behavior determination section 236 in the execution mode determined in accordance with the emotion of the robot 100, the emotion determination section 232 further changes the emotion value of the robot 100 in accordance with the reaction of the user to the execution of the behavior. Specifically, the emotion determination section 232 increases the emotion value of "joy" of the robot 100 in a case where the reaction of the user to the execution of the behavior determined by the behavior determination section 236 in the execution mode determined by the behavior control section 250 is not bad. Further, the emotion determination section 232 increases the emotion value of "sorrow" of the robot 100 in a case where the reaction of the user to the execution of the behavior determined by the behavior determination section 236 in the execution mode determined by the behavior control section 250 is bad.

[0339] Further, the behavior control section 250 embodies the emotion of the robot 100 on the basis of the determined emotion value of the robot 100. For example, the behavior control section 250 controls the control target 252 so that the robot 100 performs a happy behavior in a case where the emotion value of "joy" of the robot 100 is increased. Further, the behavior control section 250 controls the control target 252 so that the posture of the robot 100 becomes a dejected posture in a case where the emotion value of "sorrow" of the robot 100 is increased.

[0340] The communication processing section 280 undertakes communication with the server 300. As described above, the communication processing section 280 transmits the user reaction information to the server 300. Further, the communication processing section 280 receives the updated reaction rule from the server 300. If the communication processing section 280 receives the updated reaction rule from the server 300, the reaction rule as the behavior determination model 221A is updated.

[0341] The server 300 undertakes communication between the robot 100, the robot 101, and the robot 102 and the server 300, receives the user reaction information transmitted from the robot 100, and updates the reaction rule on the basis of the reaction rule including the behavior for which a positive reaction is obtained.

[0342] The association information collection section 270 collects information related to the preference information from external data (a website of a news site, a video site, or the like) at a prescribed timing on the basis of the preference information obtained for the user 10.

[0343] Specifically, the association information collection section 270 acquires preference information indicating a matter of interest of the user 10, based on the utterance content of the user 10 or the setting operation by the user 10. For example, the association information collection section 270 collects news associated with the preference information from external data using, for example, a ChatGPT plugin (ChatGPT Plugin) (Internet search <URL: https: / / openai.com / blog / ChatGPT-plugins>) every certain period. For example, in a case where it is acquired as the preference information that the user 10 is a fan of a certain professional baseball team, the association information collection section 270 collects news associated with the game result of the certain professional baseball team from external data using, for example, the ChatGPT plugin every day at a prescribed time.

[0344] The emotion determination section 232 determines the emotion of the robot 100 based on information associated with the preference information collected by the association information collection section 270.

[0345] Specifically, the emotion determination section 232 inputs text indicating information associated with the preference information collected by the association information collection section 270 to a pre-learned neural network for determining an emotion, acquires an emotion value indicating each emotion, and determines the emotion of the robot 100. For example, in a case where the collected news associated with the game result of the certain professional baseball team indicates that the certain professional baseball team won, the emotion value of "joy" of the robot 100 is determined to be large.

[0346] The storage control section 238 saves information associated with the preference information collected by the association information collection section 270 in the collection data 2230 in a case where the emotion value of the robot 100 is equal to or greater than a threshold value.

[0347] Next, the processing of the behavior determination section 236 at the time of autonomous processing of the robot 100 will be described.

[0348] The behavior determination section 236 determines any one of a plurality of robot behaviors including no behavior as a behavior of the robot 100, using at least one of the state of the user 10, the emotion of the user 10, the emotion of the robot 100, and the state of the robot 100 and a behavior determination model 221A at a prescribed timing. Here, a case where an article generation model having a dialogue function is used as the behavior determination model 221A will be described as an example.

[0349] Specifically, the behavior determination section 236 inputs text indicating at least one of the state of the user 10, the emotion of the user 10, the emotion of the robot 100, and the state of the robot 100 and text of a question robot behavior into the article generation model, and determines the behavior of the robot 100 based on the output of the article generation model.

[0350] For example, the plurality of robot behaviors include the following (1) to (10).

[0351] (1) The robot does nothing.

[0352] (2) The robot dreams.

[0353] (3) The robot talks to the user.

[0354] (4) The robot makes a painting diary.

[0355] (5) The robot proposes an activity.

[0356] (6) The robot proposes a counterpart that the user should meet.

[0357] (7) The robot introduces news that the user is interested in.

[0358] (8) The robot edits a photo or a video.

[0359] (9) The robot studies with the user.

[0360] (10) The robot awakens a memory.

[0361] The behavior determination section 236 determines the behavior of the robot 100 every certain time by inputting, to the article generation model, a text indicating the state of the user 10 and the state of the robot 100 recognized by the state recognition section 230, the current emotional value of the user 10 and the current emotional value of the robot 100 determined by the emotion determination section 232, and a text asking for any one of a plurality of robot behaviors including no behavior. Here, in the case where there is no user 10 around the robot 100, the state of the user 10 and the current emotional value of the user 10 can not be included in the text input to the article generation model, or only a text indicating that the user 10 is not present can be included.

[0362] As an example, "The robot is in a very happy state. The user is in a generally happy state. The user is sleeping. Which of the following (1) to (10) is good as a behavior of the robot? (1) The robot does nothing.

[0363] (2) The robot dreams.

[0364] (3) The robot talks to the user.

[0365] ... such text is input to the article generation model. Based on the output of the article generation model, "which of (1) do nothing or (2) the robot dreams" can be said to be the most appropriate, as the behavior of the robot 100, " (1) do nothing" or "(2) the robot dreams" is determined.

[0366] As another example, "the robot is a little lonely state. The user is not present. The surroundings of the robot are dark. Which of the following (1) to (10) is good as the behavior of the robot? (1) The robot does nothing.

[0367] (2) The robot dreams.

[0368] (3) The robot talks to the user.

[0369] ... such text is input to the article generation model. Based on the output of the article generation model, "which of (2) the robot dreams or (4) the robot makes a painting diary" can be said to be the most appropriate, as the behavior of the robot 100, "(2) the robot dreams" or "(4) the robot makes a painting diary" is determined.

[0370] The behavior determination section 236, in the case where "(2) the robot dreams", that is, making an original event as the robot behavior is determined, uses the article generation model to make an original event that combines a plurality of event data in the history data 2222. At this time, the storage control section 238 stores the original event made in the history data 2222.

[0371] The behavior determination section 236, in the case where "(3) the robot talks to the user", that is, the robot 100 speaks as the robot behavior is determined, uses the article generation model to determine the content of the robot's speech corresponding to the user's state, the user's emotion, or the robot's emotion. At this time, the behavior control section 250 outputs a sound representing the determined content of the robot's speech from the speaker included in the control object 252. In addition, in the case where the user 10 is not present in the surroundings of the robot 100, the behavior control section 250 does not output a sound representing the determined content of the robot's speech, but saves the determined content of the robot's speech in the behavior scheduled data 224.

[0372] The behavior determination section 236 determines the utterance content of the robot corresponding to the information saved in the collection data 2230 using the article generation model in a case where it determines "(7) the robot introduces the news that the user is interested in" as the robot behavior. At this time, the behavior control section 250 outputs a sound indicating the determined utterance content of the robot from the speaker included in the control target 252. In addition, in a case where the user 10 is not in the periphery of the robot 100, the behavior control section 250 does not output the sound indicating the determined utterance content of the robot, but saves the determined utterance content of the robot in the behavior scheduled data 224 in advance.

[0373] The behavior determination section 236 generates an image indicating the event data using the image generation model and generates an explanatory sentence indicating the event data using the article generation model for the event data selected from the history data 2222 in a case where it determines "(4) the robot makes a drawing diary", that is, the robot 100 makes an event image as the robot behavior, and outputs a combination of the image indicating the event data and the explanatory sentence indicating the event data as the event image. In addition, in a case where the user 10 is not in the periphery of the robot 100, the behavior control section 250 does not output the event image, but saves the event image in the behavior scheduled data 224 in advance.

[0374] The behavior determination section 236 selects the event data from the history data 2222 based on the emotional value and edits the image data of the selected event data and outputs it in a case where it determines "(8) the robot edits a photo or a video", that is, edits an image as the robot behavior. In addition, in a case where the user 10 is not in the periphery of the robot 100, the behavior control section 250 does not output the edited image data, but saves the edited image data in the behavior scheduled data 224 in advance.

[0375] The behavior determination section 236 determines the proposed user behavior using the article generation model based on the event data stored in the history data 2222 in a case where it determines "(5) the robot proposes an activity", that is, proposes the behavior of the user 10 as the robot behavior. At this time, the behavior control section 250 outputs a sound proposing the user behavior from the speaker included in the control target 252. In addition, in a case where the user 10 is not in the periphery of the robot 100, the behavior control section 250 does not output the sound proposing the user behavior, but saves the proposed user behavior in the behavior scheduled data 224 in advance.

[0376] The behavior determination section 236, in the case where it is determined that "(6) the robot proposes a counterpart that the user should meet", that is, a counterpart that is proposed to be in contact with the user 10 as a robot behavior, determines a counterpart that is proposed to be in contact with the user based on the event data stored in the history data 2222 using the article generation model. At this time, the behavior control section 250 outputs a sound indicating the counterpart that is proposed to be in contact with the user from the speaker included in the control object 252. In addition, in the case where the user 10 is not in the periphery of the robot 100, the behavior control section 250 does not output a sound indicating the counterpart that is proposed to be in contact with the user, but saves the counterpart that is proposed to be in contact with the user in the behavior scheduled data 224 in advance.

[0377] The behavior determination section 236, in the case where it is determined that "(9) the robot learns together with the user", that is, the robot 100 determines to make a speech for learning as a robot behavior, determines a speech content of the robot for making a suggestion related to learning that corresponds to the user state, the emotion of the user, or the emotion of the robot, which promotes learning or proposes a learning question, using the article generation model. At this time, the behavior control section 250 outputs a sound indicating the determined speech content of the robot from the speaker included in the control object 252. In addition, in the case where the user 10 is not in the periphery of the robot 100, the behavior control section 250 does not output a sound indicating the determined speech content of the robot, but saves the determined speech content of the robot in the behavior scheduled data 224 in advance.

[0378] The behavior determination section 236, in the case where it is determined that "(10) the robot wakes up memory", that is, recalls event data as a robot behavior, selects event data from the history data 222. At this time, the emotion determination section 232 determines the emotion of the robot 100 based on the selected event data. Further, the behavior determination section 236, based on the selected event data, uses the article generation model to create an emotion change event indicating a speech content or a behavior of the robot 100 for causing a change in the emotion value of the user. At this time, the storage control section 238 saves the emotion change event in the behavior scheduled data 224 in advance.

[0379] For example, in a case where video watched by the user is stored as event data in the history data 2222 in association with a panda, in a case where the event data is selected, the article generation model is input with "What should be said next time when meeting the user on a topic about a panda? Give three" and the output of the article generation model is "(1) go to a zoo, (2) draw a picture of a panda, (3) go to buy a panda stuffed toy", the robot 100 inputs "What does the user like best among (1), (2), and (3)?" to the article generation model, and in a case where the output of the article generation model is "(1) go to a zoo", the robot 100 makes the robot 100 speak "(1) go to a zoo" as an emotional change event at the next meeting with the user, and stores it in the behavior schedule data 224.

[0380] In addition, for example, event data with a large emotional value of the robot 100 is selected as an impressive memory of the robot 100. Thereby, an emotional change event can be made based on event data selected as an impressive memory.

[0381] The behavior determination section 236 determines the behavior of the robot 100 based on the state of the user 10 recognized by the state recognition section 230, from the state where the user 10 does not perform a behavior to the robot 100 to a case where the user 10 performs a behavior to the robot 100, reads data stored in the behavior schedule data 224, and determines the behavior of the robot 100.

[0382] For example, in a case where the user 10 is not in the vicinity of the robot 100, if the user 10 is detected, the behavior determination section 236 reads data stored in the behavior schedule data 224, and determines the behavior of the robot 100. In addition, in a case where the user 10 is sleeping, if the user 10 is detected to wake up, the behavior determination section 236 reads data stored in the behavior schedule data 224, and determines the behavior of the robot 100.

[0383] The specific processing section 290 performs, for example, specific processing in which one of the users participates as a participant in a meeting implemented on a regular basis, acquires and outputs a response related to a presentation content in the meeting. Then, the behavior of the robot 100 is controlled to output the result of the specific processing.

[0384] One example of this meeting is so-called one-on-one meeting. The one-on-one meeting is specific to two people, for example, a superior and a subordinate in an organization, and is performed in a dialogue form including confirmation of a business progress or a schedule within a cycle, various reports, liaison, negotiations, and the like, at a specific period (for example, at a frequency of once a month or so). At this time, as the user 10 of the robot 100, the subordinate is applicable. Of course, there is no hindrance to the case where the superior is the user 10 of the robot 100.

[0385] In the specific processing relating to the meeting, as a predetermined trigger condition, a condition of presentation content presented by the subordinate in the meeting is set. The specific processing section 290 acquires and outputs, as a result of the specific processing, a response relating to the presentation content in the meeting using, as input to the article generation model at the time of generation of the article, the output of the article generation model when the condition is satisfied by the user input.

[0386] The specific processing section 290 includes an input section 292, a processing section 294, and an output section 296.

[0387] The input section 292 receives a user input. Specifically, the input section 292 acquires a character input and a voice input of the user 10.

[0388] In the disclosed technology, it is assumed that the user 10 uses an email in the business. The input section 292 acquires and texturizes all contents exchanged by the user 10 through the email during a month period as a certain cycle period. Further, in a case where the user 10 uses the email together with a social network service and exchanges information through the social network service, these exchanges are included. Hereinafter, the email and the social network service are collectively referred to as "email or the like". In addition, in the mail record matters relating to the disclosed technology, matters recorded by the user 10 in the email or the like are included.

[0389] In the disclosed technology, it is assumed that the user 10 uses a schedule table of so-called groupware, schedule management software, or the like in the business. The input section 292 acquires and texturizes all schedules input by the user 10 in these schedule tables during a month period as a certain cycle period. In the groupware or the schedule management software, in addition to the schedule relating to the business, there are cases where various memos or application procedures or the like are input. In the input section 292, these memos, application procedures, or the like are acquired and texturized. In the schedule table record matters relating to the disclosed technology, in addition to the schedule, these memos or application procedures or the like are included.

[0390] In the disclosed technology, it is assumed that the user 10 attends various meetings in the business. The input section 292 acquires and texturizes all matters of speech by the user 10 at the meetings attended during a month period as a certain cycle. As the meeting, there is a meeting in which the participants actually gather at a meeting place and proceed (sometimes referred to as "face-to-face meeting", "actual meeting", "offline meeting", or the like). In addition, as the meeting, there is a meeting using an information terminal on a network (sometimes referred to as "remote meeting", "network meeting", "online meeting", or the like). Also, there are cases where the "face-to-face meeting" and the "remote meeting" are used together. Further, the broad remote meeting can include a "telephone conference" or a "video conference" using a telephone line or the like. In any form of meeting, the content of the speech by the user 10 is acquired from, for example, recording data, video data, and a meeting record of the meeting.

[0391] The processing section 294 performs specific processing using the article generation model. Specifically, as described above, the processing section 294 determines whether a predetermined trigger condition is satisfied. More specifically, input of input data from the user 10 that becomes a candidate for presentation content in a one-on-one meeting is taken as a trigger condition.

[0392] Then, the processing section 294 inputs a text (cue word) indicating an instruction for obtaining data used for specific processing to the article generation model, and acquires a processing result based on an output of the article generation model. More specifically, for example, a cue word of "Please summarize the business implemented by the user 10 in the past month, and cite three points that become a presentation point in the next one-on-one meeting" is input to the article generation model, and a recommended presentation point in a one-on-one meeting is acquired from an output of the article generation model. As the article generation model of the presentation point, for example, there are "behavior time is accurate", "goal achievement rate is high", "business content is accurate", "reaction to e-mail and the like is fast", "summarizes the meeting", "takes the lead in the project", and the like. In addition, the processing section 294 can perform specific processing using the state of the user 10 and the article generation model. In addition, the processing section 294 can perform specific processing using the emotion of the user 10 and the article generation model.

[0393] The output section 296 controls the behavior of the robot 100 to output a result of specific processing. Specifically, a summary and a presentation point acquired by the processing section 294 are displayed on a display device possessed by the robot 100, or the robot 100 speaks the summary and the presentation point, or a message indicating the summary and the presentation point is transmitted to the user of a message application of a portable terminal.

[0394] In addition, a part of the robot 100 (for example, the sensor module section 210, the storage section 220, the control section 228) can be provided outside the robot 100 (for example, a server), and the robot 100 communicates with the outside to function as each section of the above-described robot 100.

[0395] In addition, a part of the robot 100 (for example, the sensor module section 210, the storage section 220, the control section 228) can be provided outside the robot 100 (for example, a server), and the robot 100 communicates with the outside to function as each section of the above-described robot 100.

[0396] Figure 10 An example of an operation flow related to a collection process of collecting information associated with preference information of the user 10 is schematically shown. Figure 10The illustrated operation flow is repeatedly executed at a certain period. It is assumed that the preference information indicating the matter of interest of the user 10 is acquired based on the utterance content of the user 10 or the setting operation performed by the user 10. Further, "S" in the operation flow indicates a step executed.

[0397] First, in step S90, the association information collection section 270 acquires the preference information indicating the matter of interest of the user 10.

[0398] In step S92, the association information collection section 270 collects information associated with the preference information from the outside.

[0399] In step S94, the emotion determination section 232 determines the emotion value of the robot 100 based on the information associated with the preference information collected by the association information collection section 270.

[0400] In step S96, the storage control section 238 determines whether or not the emotion value of the robot 100 determined in step S94 is equal to or greater than a threshold value. In the case where the emotion value of the robot 100 is less than the threshold value, the information associated with the collected preference information is not stored in the collected data 2230, and the processing is ended. On the other hand, in the case where the emotion value of the robot 100 is equal to or greater than the threshold value, the processing is shifted to step S98.

[0401] In step S98, the storage control section 238 stores the information associated with the collected preference information in the collected data 2230, and the processing is ended.

[0402] Figure 11A An example of an operation flow related to the operation of determining the behavior in the robot 100 at the time of response processing in which the robot 100 responds to the behavior of the user 10 is schematically shown. The illustrated operation flow is repeatedly executed Figure 11A The illustrated operation flow. At this time, it is assumed that the information analyzed by the sensor module section 210 is input.

[0403] First, in step S100, the state recognition section 230 recognizes the state of the user 10 and the state of the robot 100 based on the information analyzed by the sensor module section 210.

[0404] In step S102, the emotion determination section 232 determines the emotion value indicating the emotion of the user 10 based on the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230.

[0405] In step S103, the emotion determination section 232 determines an emotion value of the robot 100 representing an emotion based on the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230. The emotion determination section 232 adds the determined emotion value of the user 10 and the emotion value of the robot 100 to the history data 2222.

[0406] In step S104, the behavior recognition section 234 recognizes a behavior classification of the user 10 based on the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230.

[0407] In step S106, the behavior determination section 236 determines a behavior of the robot 100 based on a combination of the current emotion value of the user 10 determined in step S102 and the past emotion values included in the history data 2222, the emotion value of the robot 100, the behavior of the user 10 recognized in the above step S104, and the behavior determination model 221A.

[0408] In step S108, the behavior control section 250 controls the control object 252 based on the behavior determined by the behavior determination section 236.

[0409] In step S110, the storage control section 238 calculates a comprehensive value of the intensity based on the intensity of the behavior predetermined for the behavior determined by the behavior determination section 236 and the emotion value of the robot 100 determined by the emotion determination section 232.

[0410] In step S112, the storage control section 238 determines whether the comprehensive value of the intensity is equal to or greater than a threshold value. In a case where the comprehensive value of the intensity is less than the threshold value, the event data including the behavior of the user 10 is not stored in the history data 2222, and the process ends. On the other hand, in a case where the comprehensive value of the intensity is equal to or greater than the threshold value, the process proceeds to step S114.

[0411] In step S114, event data including the behavior determined by the behavior determination section 236 and the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230 from the current timing to a certain period before are stored in the history data 2222.

[0412] Figure 11B An example of an operation flow related to the operation of determining a behavior in the robot 100 is shown schematically when the robot 100 performs an autonomous process of autonomous behavior. Figure 11B The shown operation flow is repeatedly automatically performed, for example, every certain time. At this time, it is assumed that the information analyzed by the sensor module section 210 is input. In addition, the same step number is shown for the same process as the above Figure 11A ​

[0413] First, in step S100, the state recognition section 230 recognizes the state of the user 10 and the state of the robot 100 on the basis of information analyzed by the sensor module section 210.

[0414] In step S102, the emotion determination section 232 determines an emotional value indicating an emotion of the user 10 on the basis of information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230.

[0415] In step S103, the emotion determination section 232 determines an emotional value indicating an emotion of the robot 100 on the basis of information analyzed by the sensor module section 210 and the state of the robot 100 recognized by the state recognition section 230. The emotion determination section 232 adds the determined emotional value of the user 10 and the emotional value of the robot 100 to the history data 2222.

[0416] In step S104, the behavior recognition section 234 recognizes a behavior category of the user 10 on the basis of information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230.

[0417] In step S200, the behavior determination section 236 determines a behavior of the robot 100 as any one of a plurality of robot behaviors including no behavior on the basis of the state of the user 10 recognized in the above step S100, the emotion of the user 10 determined in step S102, the emotion of the robot 100, the state of the robot 100 recognized in the above step S100, the behavior of the user 10 recognized in the above step S104, and the behavior determination model 221A.

[0418] In step S201, the behavior determination section 236 determines whether or not no behavior is determined in the above step S200. As the behavior of the robot 100, in the case where no behavior is determined, the processing is ended. On the other hand, as the behavior of the robot 100, in the case where no behavior is not determined, the processing is shifted to step S202.

[0419] In step S202, the behavior determination section 236 performs processing corresponding to the kind of robot behavior determined in the above step S200. At this time, depending on the kind of robot behavior, the behavior control section 250, the emotion determination section 232, or the storage control section 238 performs the processing.

[0420] In step S110, the storage control section 238 calculates a comprehensive value of intensity on the basis of the intensity of the behavior predetermined for the behavior determined by the behavior determination section 236 and the emotional value of the robot 100 determined by the emotion determination section 232.

[0421] In step S112, the storage control section 238 determines whether the integrated value of the intensity is equal to or greater than a threshold value. In a case where the integrated value of the intensity is less than the threshold value, the event data including the behavior of the user 10 is not stored in the history data 2222, and the process ends. On the other hand, in a case where the integrated value of the intensity is equal to or greater than the threshold value, the process proceeds to step S114.

[0422] In step S114, the storage control section 238 stores, in the history data 2222, event data including the behavior determined by the behavior determination section 236 and information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230 from the current timing to a certain period before.

[0423] As described above, according to the robot 100, the emotional value indicating the emotion of the robot 100 is determined on the basis of the state of the user, and it is determined whether to store the data including the behavior of the user 10 in the history data 2222 on the basis of the emotional value of the robot 100. Thereby, it is possible to suppress the capacity of the history data 2222 storing the data including the behavior of the user 10. Also, for example, in a case where the robot 100 determines that the state of the user 10 is the same as that of 10 years ago, by reading the history data 2222 of 10 years ago, the robot 100 is able to present the user 10 with all the surrounding information such as the state of the user 10 (for example, the expression of the user 10, the emotion, and the like) at that time 10 years ago, and further the data of the voice, the image, the odor, and the like at that time.

[0424] In addition, according to the robot 100, it is possible to cause the robot 100 to perform an appropriate behavior with respect to the behavior of the user 10. In the past, the behavior of the user was classified, and a behavior including the expression or the appearance of the robot was determined. In contrast to this, the robot 100 determines the current emotional value of the user 10, and performs a behavior with respect to the user 10 on the basis of the past emotional value and the current emotional value. Therefore, for example, in a case where the user 10 who was good yesterday is in a low mood today, the robot 100 is able to make a statement such as "You were good yesterday, what's wrong today?" In addition, the robot 100 can also make a statement in conjunction with a posture. Also, for example, in a case where the user 10 who was in a low mood yesterday is good today, the robot 100 is able to make a statement such as "You were in a low mood yesterday, are you good today?" Also, for example, in a case where the user 10 who was good yesterday is better than yesterday, the robot 100 is able to make a statement such as "You are better than yesterday today. Is there something better than yesterday?" In addition, for example, the robot 100 is able to make a statement such as "You have been in a stable mood recently, and you feel good" with respect to the user 10 whose emotional value is equal to or greater than 0 and whose variation in the emotional value is within a certain range.

[0425] Further, for example, the robot 100 asks the user 10 "Did you finish the homework you talked about yesterday?", and in a case where the user 10 gives a reply of "Yes, I finished it", the robot 100 can make a positive utterance such as "You are amazing!" and can make a positive gesture such as clapping or a pose. Further, for example, the robot 100 can make a positive utterance such as "You worked hard!" and make the above positive gesture when the user 10 says "The presentation you talked about the other day went well". In this way, by the robot 100 making a behavior based on the state history of the user 10, it is expected that the user 10 will develop an affinity for the robot 100.

[0426] Further, for example, in a case where the user 10 watches a video related to a panda, and the emotional value of the "joy" of the user 10 is above the threshold value, the scene in which the panda appears in the video can also be stored as event data in the history data 2222.

[0427] By using the data accumulated in the history data 2222 or the collection data 2230, the robot 100 can always learn what kind of conversation with the user maximizes the emotional value of the user's happiness.

[0428] Further, in a state where the robot 100 is not conversing with the user 10, a behavior can be autonomously started based on the emotion of the robot 100.

[0429] Further, in the autonomous processing, the robot 100 automatically generates a question, inputs it to the article generation model, and by repeatedly obtaining the output of the article generation model as an answer to the question, can create an emotional change event for increasing a good emotion and save it in the behavior schedule data 224. In this way, the robot 100 can perform self-learning.

[0430] Further, in a state where the robot 100 is not subjected to a trigger from the outside, when a question is automatically generated, the question can be automatically generated based on the impressive event data determined from the past emotional value history of the robot.

[0431] Further, the association information collection section 270 can perform self-learning by automatically performing keyword search corresponding to the user's preference information, and repeatedly performing a search execution stage of the search result.

[0432] Here, the search execution stage can also automatically perform keyword search based on the impressive event data determined from the past emotional value history of the robot in a state where it is not subjected to a trigger from the outside.

[0433] Further, the emotion determination section 232 can determine the emotion of the user in accordance with a specific map. Specifically, the emotion determination section 232 can determine the emotion of the user in accordance with an emotion map (see FIG. 6) as a specific map. Figure 5 ).

[0434] The behavior determination section 236 inputs the text indicating the behavior of the user, the emotion of the user, and the emotion of the robot, by adding a fixed sentence for asking the robot behavior content corresponding to the behavior of the user, to the article generation model having a dialogue function, thereby generating the robot behavior content.

[0435] For example, the behavior determination section 236 acquires the text indicating the state of the robot 100, using the emotion table shown in the aforementioned Table 1, from among the emotion of the robot 100 determined by the emotion determination section 232. Here, in the emotion table, each emotion value is given an index number for each kind of emotion, and for each index number, a text indicating the state of the robot 100 is saved.

[0436] In a case where the emotion of the robot 100 determined by the emotion determination section 232 corresponds to the index number "2", the text of "a very happy state" is obtained. Also, in a case where the emotion of the robot 100 corresponds to a plurality of index numbers, a plurality of texts indicating the state of the robot 100 are obtained.

[0437] Also, for the emotion of the user 10, the emotion table shown in the aforementioned Table 4 is prepared.

[0438] Here, in a case where the behavior of the user is "play together", the emotion of the robot 100 is the index number "2", and the emotion of the user 10 is the index number "3", the text of "the robot is in a very happy state. The user is in a generally happy state. The user has approached the robot with "play together". How should the robot reply?" is input to the article generation model, and the behavior content of the robot is acquired. The behavior determination section 236 determines the behavior of the robot in accordance with the behavior content.

[0439] In this way, the behavior determination section 236 determines the behavior content of the robot 100 in correspondence with the state related to the emotion of the robot 100 and the behavior of the user 10, in accordance with each kind of the emotion of the robot 100 and each intensity of the emotion. In this manner, the utterance content of the robot 100 when having a dialogue with the user 10 can be branched in accordance with the state related to the emotion of the robot 100. That is, since the robot 100 can change the behavior of the robot in accordance with the index number corresponding to the emotion of the robot, the user has an impression that the robot has a mind, and is encouraged to take an approach and the like to the robot.

[0440] In addition, the behavior determination section 236 can generate the behavior content of the robot by adding, to the text indicating the behavior of the user, the emotion of the user, and the emotion of the robot, a text indicating the content of the history data 2222, and then adding a fixed sentence for asking about the behavior content of the robot corresponding to the behavior of the user, and inputting the article generation model having a dialogue function. Thereby, since the robot 100 can change the behavior of the robot according to the history data indicating the emotion or the behavior of the user, the user has an impression that the robot has personality, and is encouraged to take an action such as making a conversation with the robot. In addition, the emotion or the behavior of the robot can be further included in the history data.

[0441] In addition, the emotion determination section 232 can also determine the emotion of the robot 100 according to the behavior content of the robot 100 generated by the article generation model. Specifically, the emotion determination section 232 inputs the behavior content of the robot 100 generated by the article generation model to a neural network learned in advance, acquires the emotion values indicating each emotion shown in the emotion map 400, integrates the acquired emotion values indicating each emotion with the emotion values indicating each emotion of the current robot 100, and updates the emotion of the robot 100. For example, the acquired emotion values indicating each emotion and the emotion values indicating each emotion of the current robot 100 are respectively averaged and integrated. The neural network is a neural network learned in advance based on a combination of the text indicating the behavior content of the robot 100 generated by the article generation model and the emotion values indicating each emotion shown in the emotion map 400, that is, a plurality of learning data.

[0442] For example, as the behavior content of the robot 100 generated by the article generation model, in a case where the utterance content of the robot 100 is "That's great. What a lucky day!", when a text indicating the utterance content is input to the neural network, a high value is obtained as the emotion value of the emotion "joy", and the emotion of the robot 100 is updated in such a way that the emotion value of the emotion "joy" becomes high.

[0443] In the robot 100, the article generation model such as ChatGPT is linked to the emotion determination section 232, and a method of growing with various parameters continuously during a period in which the user does not speak, with self, is executed.

[0444] ChatGPT is a large-scale language model using a deep learning method. ChatGPT can also refer to external data, such as in ChatGPT plugins, where it is known that various external data such as weather information, hotel reservation information, and the like are referred to while giving an answer as correctly as possible through a conversation. For example, in ChatGPT, when a purpose is given in natural language, source code can be automatically generated in various programming languages. For example, in ChatGPT, when problematic source code is given, it can also be debugged to find problems and automatically generate improved source code. If they are combined, a self-reliant agent appears that repeatedly performs code generation and debugging before the source code has no problems, given a purpose in natural language. As such a self-reliant agent, AutoGPT, babyAGI, JARVIS, and E2B are known.

[0445] In the robot 100 related to the present embodiment, as described in Patent Literature 2 (Patent No. 6199927), a technology can be used in which the robot remembers event data of strong emotions for a long time and quickly forgets event data of which the robot hardly emerges emotions, and event data that should be learned is retained in an impressive memory database.

[0446] In addition, the robot 100 can record image data and the like of the user 10 acquired through a camera function in the history data 2222. The robot 100 can acquire image data and the like from the history data 2222 as needed and provide it to the user 10. The stronger the emotion of the robot 100, the more information-rich image data can be generated and recorded in the history data 2222. For example, the robot 100, in the case of recording high compression form information such as bone data, can switch to recording low compression form information such as HD video, depending on whether the excitement emotion value exceeds a threshold value. According to the robot 100, for example, it is possible to retain high-precision image data at a time when the emotion of the robot 100 is high as a record.

[0447] When the robot 100 does not talk to the user 10, the robot 100 can automatically load event data from the history data 2222 that stores impressive event data, and continue to update the emotion of the robot 100 by the emotion determination unit 232. The robot 100 can create an emotion change event for making the emotion of the user 10 better based on impressive event data when the robot 100 does not talk to the user 10 and the emotion of the robot 100 becomes an emotion that promotes learning. Thereby, it is possible to implement autonomous learning (recall event data) at an appropriate timing corresponding to the emotional state of the robot 100, and it is possible to implement autonomous learning that appropriately reflects the emotional state of the robot 100.

[0448] As the emotion that promotes learning, in the negative state, the emotion that is near "Remorse" or "Regret" of the emotion map of Dr. Light, and in the positive state, the emotion that is near "Desire" of the emotion map.

[0449] The robot 100 can handle "Remorse" and "Regret" of the emotion map as the emotion that promotes learning in the negative state. The robot 100 can handle, in addition to "Remorse" and "Regret" of the emotion map, emotions adjacent to "Remorse" and "Regret" as the emotion that promotes learning in the negative state. For example, the robot 100 handles, in addition to "Remorse" and "Regret", at least one of "Begrudge", "Stubborn", "Self-destroy", "Self-restrain", "Regret", and "Despair" as the emotion that promotes learning. Thereby, for example, the robot 100 can perform autonomous learning when having a negative feeling of "Never want to experience this feeling again", "Don't want to be scolded again".

[0450] The robot 100 can handle "Desire" of the emotion map as the emotion that promotes learning in the positive state. The robot 100 can handle, in addition to "Desire", emotions adjacent to "Desire" as the emotion that promotes learning in the positive state. For example, the robot 100 handles, in addition to "Desire", at least one of "Joy", "Ecstasy", "Craving", "Expectation", and "Shame" as the emotion that promotes learning. Thereby, for example, the robot 100 can perform autonomous learning when having a positive feeling of "Want more", "Want to know more".

[0451] The robot 100 can not perform autonomous learning when the robot 100 has an emotion other than the above-described emotion that promotes learning. Thereby, for example, autonomous learning can not be performed when extremely angry or blindly feeling love.

[0452] The emotion change event refers to, for example, behavior after an impressive event. The behavior after the impressive event refers to an emotion label on the outermost side of the emotion map, for example, "Love" followed by "Tolerance" or "Compassion".

[0453] In autonomous learning performed when the robot 100 does not talk to the user 10, for people and oneself appearing in the impressive memory, respective emotions, situations, behaviors, and the like are combined, an article generation model is used, and an emotion change event is created.

[0454] Suppose that all the emotional values are expressed by six-level evaluation from 0 to 5, as impressive event data, consider the case where "a friend was hit and looked very nasty" is stored in the history data 2222. Here, the friend refers to the user 10, and suppose that the emotion of the user 10 is "disgust" and the value indicating "disgust" is 5. Also, suppose that the emotion of the robot 100 is "unease" and the value indicating "unease" is 4.

[0455] The robot 100, by executing autonomous processing during a period when it does not talk to the user 10, can continue to grow with various parameters. Specifically, as the uppermost event data arranged in order of strong emotional value from the history data 2222, for example, "a friend was hit and looked very nasty" is loaded. In the loaded event data, as the emotion of the robot 100, "unease" of strength 4 is bound, and here, as the emotion of the user 10 who is the friend, "disgust" of strength 5 is bound. If the current emotional value of the robot 100 is "at ease" of strength 3 before loading, after loading, the emotional value of the robot 100 becomes "regret" meaning "sorry" (regretful) sometimes considering the influence of "unease" of strength 4 and "disgust" of strength 5. At this time, because "regret" is an emotion that promotes learning, the robot 100 determines to make a change in emotion event by recalling the event data as a robot behavior. At this time, the information input to the article generation model is a text indicating impressive event data, and in this example, "a friend was hit and looked very nasty". Also, in the emotional map, the emotion of "disgust" is on the innermost side, and as a behavior corresponding thereto, "attack" is predicted on the outermost side, so in this example, a change in emotion event is made to avoid "attack" by the friend on someone.

[0456] For example, if the impressive event data information is used to solve a fill-in-the-blank problem, the following input text can be automatically generated.

[0457] "User was hit. At that time, the user felt very strong disgust. The robot was very uneasy. Please tell me a line that the robot should say next time when meeting the user in 30 words or less. However, please note that it is irrelevant to the time period of the meeting. Also, please avoid direct expression. Give three candidates.

[0458] <Expected format> Candidate 1: (a line that the robot should say to the user) Candidate 2: (a line that the robot should say to the user) Candidate 3: (a line that the robot should say to the user)" At this time, the output of the article generation model is, for example, as follows.

[0459] "Candidate 1: Are you okay? I'm very concerned about yesterday's incident."

[0460] Candidate 2: I am very concerned about yesterday. Do you need me to do something? Candidate 3: I am very worried about you. Can we talk about it? Further, regarding the information obtained in the production of the emotional change event, the robot 100 can also automatically generate the following input text.

[0461] In the case of "the user was slapped", what will the user's mood be when the user is greeted? The user's emotion is in the form of "joy A anger B sadness C happiness D", and the integer from 0 to 5 of the six-level evaluation needs to be filled in from A to D.

[0462] Candidate 1: Are you okay? I am very concerned about yesterday.

[0463] Candidate 2: I am very concerned about yesterday. Do you need me to do something? Candidate 3: I am very worried about you. Can we talk about it? At this time, the output of the article generation model is, for example, as follows.

[0464] "The user's emotion can be as follows.

[0465] Candidate 1: joy 3 anger 1 sadness 2 happiness 2 Candidate 2: joy 2 anger 1 sadness 3 happiness 2 Candidate 3: joy 2 anger 1 sadness 3 happiness 3" In this way, the robot 100 can also perform processing around the idea after producing an emotional change event.

[0466] Finally, the robot 100 can produce an emotional change event using the most pleasant candidate 1 among the multiple candidates, save it in the behavior schedule data 224, and prepare for the next time it meets the user 10.

[0467] As described above, even when there is no conversation with family or friends, the robot 100 continues to determine the emotional value of the robot using the information of the history data 2222 that stores the impressive event data, and when it becomes the above-mentioned emotion that promotes learning, the robot 100 performs autonomous learning without talking to the user 10 according to the emotion of the robot 100, and continues to update the history data 2222 or the behavior schedule data 224.

[0468] The above is an example of using an emotional value, but since emotions can be generated in the emotional map according to the amount of hormone secretion and the type of event, as a value associated with impressive event data, it can also be the type of hormone, the amount of hormone secretion, and the type of event.

[0469] Hereinafter, specific embodiments will be described.

[0470] The robot 100 investigates a topic or information related to an interest of the user, for example, even when there is no conversation with the user.

[0471] The robot 100 investigates information related to a birthday or an anniversary of the user, considering a message of blessing, for example, even when there is no conversation with the user.

[0472] The robot 100 investigates a place or food, a comment on a commodity that the user wants to go, for example, even when there is no conversation with the user.

[0473] The robot 100 investigates weather information, providing a suggestion that matches a schedule or a plan of the user, for example, even when there is no conversation with the user.

[0474] The robot 100 investigates information on a local event or a festival, proposing to the user, for example, even when there is no conversation with the user.

[0475] The robot 100 investigates a result of a sports game or news that the user is interested in, providing a topic, for example, even when there is no conversation with the user.

[0476] The robot 100 investigates and introduces information on music or an artist that the user likes, for example, even when there is no conversation with the user.

[0477] The robot 100 investigates information related to a social issue or news that the user is interested in, providing an opinion, for example, even when there is no conversation with the user.

[0478] The robot 100 investigates information related to a hometown or a birthplace of the user, providing a topic, for example, even when there is no conversation with the user.

[0479] The robot 100 investigates information on a work or a school of the user, providing a suggestion, for example, even when there is no conversation with the user.

[0480] The robot 100 investigates and introduces information on a book or a comic, a movie, a TV drama that the user is interested in, for example, even when there is no conversation with the user.

[0481] The robot 100 investigates information related to health of the user, providing a suggestion, for example, even when there is no conversation with the user.

[0482] The robot 100 investigates information related to a travel plan of the user, providing a suggestion, for example, even when there is no conversation with the user.

[0483] The robot 100 is able to investigate information related to repair and maintenance of a house or a car of the user, providing a suggestion, for example, even when there is no conversation with the user.

[0484] The robot 100 investigates information on beauty or fashion that the user is interested in, for example, even when there is no conversation with the user, and proposes a suggestion.

[0485] The robot 100 investigates information on the pet of the user, for example, even when there is no conversation with the user, and proposes a suggestion.

[0486] The robot 100 investigates information on a competition or event associated with the interest or work of the user, for example, even when there is no conversation with the user, and proposes a suggestion.

[0487] The robot 100 investigates information on a restaurant or a dining place that the user likes, for example, even when there is no conversation with the user, and proposes a suggestion.

[0488] The robot 100 collects information on an important decision related to the life of the user and proposes a suggestion, for example, even when there is no conversation with the user.

[0489] The robot 100 investigates information on a person that the user is concerned about, for example, even when there is no conversation with the user, and proposes a suggestion.

[0490] (Third Embodiment) In the third embodiment, the robot 100 described above is mounted on a stuffed toy, or is applied to a control device that is connected to a control target machine (a speaker or a camera) mounted on a stuffed toy in a wireless or wired manner. In addition, for portions that are the same structure as the second embodiment, the same symbols are marked and the description is omitted.

[0491] Specifically, the third embodiment is configured as follows. For example, the robot 100 is applied to a co-living person (specifically, a stuffed toy 100N shown in Figs. 18 and 19) that promotes a conversation based on information related to the daily life of the user 10, or provides information that matches the interest and hobby of the user 10, while living together with the user 10. In the third embodiment, an example in which the control portion of the robot 100 described above is applied to a smartphone 50 is described. Figure 7 and Figure 8 Specifically, the third embodiment is configured as follows. For example, the robot 100 is applied to a co-living person (specifically, a stuffed toy 100N shown in Figs. 18 and 19) that promotes a conversation based on information related to the daily life of the user 10, or provides information that matches the interest and hobby of the user 10, while living together with the user 10. In the third embodiment, an example in which the control portion of the robot 100 described above is applied to a smartphone 50 is described.

[0492] Figure 12 The functional structure of the stuffed toy 100N is schematically shown. The stuffed toy 100N includes a sensor portion 200A, a sensor module portion 210, a storage portion 220, a control portion 228, and a control target 252A.

[0493] The smartphone 50 housed in the stuffed toy 100N of the present embodiment performs the same processing as the robot 100 of the second embodiment. That is, the smartphone 50 has a sensor portion 200A, a sensor module portion 210, a storage portion 220, a control portion 228, and a control target 252A. Figure 12The functions as the sensor module section 210, the functions as the storage section 220, and the functions as the control section 228 are shown.

[0494] Here, the smartphone 50 is housed in the space section 52 from the outside, and via the USB hub 64 (refer to Figure 7 (B) ) is connected with each of the input / output devices via USB, and thus can have the same functions as the robot 100 of the second embodiment described above.

[0495] In addition, a non-contact type power receiving pad 66 is connected to the USB hub 64. The power receiving pad 66 is assembled with a power receiving coil 66A. The power receiving pad 66 is an example of a wireless power receiving section that receives wireless power supply.

[0496] The power receiving pad 66 is disposed near the leg root portions 68 of the stuffed toy 100N, and is in the position closest to the placement pedestal 70 when the stuffed toy 100N is placed on the placement pedestal 70. The placement pedestal 70 is an example of an external wireless power supply section.

[0497] The stuffed toy 100N placed on this placement pedestal 70 can be enjoyed as an ornament in a natural state.

[0498] In addition, the root portions are formed to have a thinner thickness than the surface layer of the other portions of the stuffed toy 100N, and are held in a state closer to the placement pedestal 70.

[0499] The placement pedestal 70 is provided with a charging pad 72. The charging pad 72 is assembled with a power supply coil 72A that transmits a signal, searches for the power receiving coil 66A of the power receiving pad 66, and if the power receiving coil 66A is found, a magnetic field is generated by a current flowing through the power supply coil 72A, and the power receiving coil 66A starts electromagnetic induction in response to the magnetic field. As a result, a current flows through the power receiving coil 66A, and stores power in the battery (omitted from the drawing) of the smartphone 50 via the USB hub 64.

[0500] That is, by placing the stuffed toy 100N as an ornament on the placement pedestal 70, the smartphone 50 is automatically charged, and thus it is not necessary to take out the smartphone 50 from the space section 52 of the stuffed toy 100N in order to charge.

[0501] In addition, in the third embodiment, the smartphone 50 is housed in the space portion 52 of the stuffed toy 100N, and connection is made by wire (USB connection), but it is not limited thereto. For example, a control device with wireless function (for example, "Bluetooth (registered trademark)") can be housed in the space portion 52 of the stuffed toy 100N, and the control device can be connected to the USB hub 64. In this case, the smartphone 50 is not put into the space portion 52, and the smartphone 50 and the control device communicate wirelessly, and the external smartphone 50 is connected to each input and output device via the control device, whereby the same functions as the robot 100 shown in the above-described second embodiment can be provided. In addition, the control device housed in the space portion 52 of the stuffed toy 100N can be connected to the external smartphone 50 by wire.

[0502] In addition, in the third embodiment, the stuffed toy 100N of a bear is exemplified, but it can be another animal, can be a doll, or can be a shape of a specific character. In addition, it can be changed. Furthermore, the material of the skin is not limited to cloth, but can be another material such as soft plastic, but a soft material is preferable.

[0503] In addition, a monitor can be installed in the skin of the stuffed toy 100N, and a control object 252 that provides information to the user 10 by vision can be added. For example, the eye portion 56 can be a monitor, and joy and sorrow can be expressed by an image reflected in the eye. In addition, a window through which a monitor of a built-in smartphone 50 is provided in the abdomen. Furthermore, the eye portion 56 can be a projector, and joy and sorrow can be expressed by an image projected on a wall surface.

[0504] According to the third embodiment, the existing smartphone 50 is put into the stuffed toy 100N, whereby the camera 203, the microphone 201, the speaker 60, and the like are respectively extended to appropriate positions via USB connection.

[0505] Furthermore, in order to perform wireless charging, the smartphone 50 and the power receiving panel 66 are connected by USB, and the power receiving panel 66 is arranged as much as possible on the outside from the inside of the stuffed toy 100N.

[0506] If wireless charging of the smartphone 50 is to be used, the smartphone 50 must be arranged as much as possible on the outside from the inside of the stuffed toy 100N, and when the stuffed toy 100N is touched from the outside, it becomes uneven.

[0507] Therefore, the smartphone 50 is arranged as much as possible at the center of the stuffed toy 100N, and the wireless charging function (power receiving board 66) is arranged as much as possible at the outside, as viewed from the inside of the stuffed toy 100N. The camera 203, the microphone 201, the speaker 60, and the smartphone 50 receive wireless power supply via the power receiving board 66.

[0508] In addition, other structures and functions of the stuffed toy 100N of the third embodiment are the same as those of the robot 100 of the first embodiment, and thus the description thereof is omitted.

[0509] (Fourth Embodiment) In the above-described second embodiment, the case where the behavior control system is applied to the robot 100 is exemplified, but in the fourth embodiment, the above-described robot 100 is used as an agent for conversing with a user, and the behavior control system is applied to the agent system. In addition, for the portions that are the same structures as those of the second embodiment and the third embodiment, the same symbols are labeled and the description thereof is omitted.

[0510] Figure 13 is a functional block diagram of an agent system 500 configured using part or all of the functions of the behavior control system.

[0511] The agent system 500 is a computer system that performs a series of behaviors in accordance with the intention of the user 10 through a conversation with the user 10. The conversation with the user 10 can be performed through voice or text.

[0512] The agent system 500 includes the sensor section 200A, the sensor module section 210, the storage section 220, the control section 228B, and the control target 252B.

[0513] The agent system 500 can be mounted on, for example, a robot, a mannequin, a stuffed toy, a pendant, a smart watch, a smartphone, a smart speaker, earphones, and a tablet computer. In addition, the agent system 500 can also be implemented by a web server and used via a web browser operated on a communication terminal such as a smartphone held by the user.

[0514] The agent system 500 plays a role of, for example, a housekeeper, a secretary, a teacher, a partner, a friend, a lover, or a teacher for the user 10. The agent system 500 not only converses with the user 10 but also provides advice, guides to a destination, or makes recommendations in accordance with the user's preference, and the like. In addition, the agent system 500 makes a reservation, an order, or a payment for a service provider, and the like.

[0515] The emotion determination section 222 determines the emotion of the user 10 and the emotion of the agent itself, as with the second embodiment described above. The behavior determination section 236 determines the behavior of the robot 100 while taking into account the emotions of the user 10 and the agent. That is, the agent system 500 understands the emotion of the user 10 and provides heartfelt support, assistance, advice, and service by reading between the lines. The agent system 500 also participates in the user 10's consultation of troubles to comfort the user, encourage the user, and cheer up the user. The agent system 500 also plays with the user 10, draws a picture diary, and reminisces about the past. The agent system 500 performs a behavior that increases the happiness of the user 10.

[0516] The control section 228B has a state recognition section 230, an emotion determination section 232, a behavior recognition section 234, a behavior determination section 236, a storage control section 238, a behavior control section 250, a related information collection section 270, a command acquisition section 272, an RPA (Robotic Process Automation) 274, a role setting section 276, and a communication processing section 280.

[0517] The behavior determination section 236 determines the utterance content of the agent for dialogue with the user 10 as the behavior of the agent, as with the second embodiment described above. The behavior control section 250 outputs the utterance content of the agent by at least one of a sound and a text, by a speaker or a display that is a control target 252B.

[0518] The role setting section 276 sets the role of the agent when the agent system 500 dialogues with the user 10, according to a designation from the user 10. That is, the utterance content output from the behavior determination section 236 is output by the agent having the set role. As the role, for example, a famous person or a celebrity who actually exists, such as an actor or an athlete, can be set. In addition, a fictional character who appears in a comic, a movie, or a video can also be set. For example, "Princess Ann" played by "Audrey Hepburn" appearing in the movie "Roman Holiday" can be set as the role of the agent. In a case where the agent role is known, since the voice, the diction, the intonation, and the personality of the role are known, the user 10 only designates a role that the user 10 likes, and the setting of the role in the role setting section 276 is automatically performed. The voice, the diction, the intonation, and the personality of the set role are reflected in the dialogue with the user 10. That is, the behavior control section 250 synthesizes a voice corresponding to the role set by the role setting section 276, and outputs the utterance content of the agent according to the synthesized voice. Thereby, the user 10 can have a feeling of actually dialoguing with the role (for example, a favorite actor) that the user 10 likes.

[0519] When the agent system 500 is mounted on a device having a display such as a smartphone, an icon, a still image, or a video of an agent having a role set by the role setting section 276 can be displayed on the display. For example, an image of the agent is generated using an image synthesis technique such as 3D rendering. In the agent system 500, the image of the agent can perform a gesture corresponding to the emotion of the user 10, the emotion of the agent, and the utterance content of the agent while having a conversation with the user 10. In addition, the agent system 500 can output only a sound without outputting an image while having a conversation with the user 10.

[0520] The emotion determination section 232 determines the emotion value of the user 10 and the emotion value of the agent itself as in the second embodiment. In the present embodiment, the emotion value of the agent is determined instead of the emotion value of the robot 100. The emotion value of the agent itself is reflected in the emotion of the set role. When the agent system 500 has a conversation with the user 10, not only the emotion of the user 10 but also the emotion of the agent is reflected in the conversation. That is, the behavior control section 250 outputs the utterance content in a manner corresponding to the emotion determined by the emotion determination section 232.

[0521] In addition, in a case where the agent system 500 performs a behavior facing the user 10, the emotion of the agent is also reflected. For example, in a case where the user 10 entrusts the agent system 500 with photographing a photo, whether the agent system 500 photographs the photo according to the user's entrustment depends on the degree of the emotion of "sorrow" held by the agent. The role performs a good-willed conversation or behavior to the user 10 in a case where the role holds a positive emotion, and performs a contentious conversation or behavior to the user 10 in a case where the role holds a negative emotion.

[0522] The history data 2222 stores a history of a conversation between the user 10 and the agent system 500 as event data. The holding unit 220 can also be realized by an external cloud storage. The agent system 500 determines a conversation content or a behavior content in a case where it converses with the user 10 or in a case where it performs a behavior facing the user 10, also taking into account the conversation history content saved in the history data 2222. For example, the agent system 500 grasps the interest and the taste of the user 10 based on the conversation history saved in the history data 2222. The agent system 500 generates a conversation content that conforms to the interest and the taste of the user 10, or provides a recommendation. The behavior determination unit 236 determines the utterance content of the agent based on the conversation history saved in the history data 2222. In the history data 2222, personal information of the user 10 such as a name, an address, a telephone number, a credit card number, and the like acquired through a conversation with the user 10 is saved. Here, it can also be an utterance of the agent that spontaneously asks whether or not to register personal information of the user 10, such as "Do you want to register a credit card number?", and the personal information is stored in the history data 2222 according to the answer of the user 10.

[0523] As explained in the above-described second embodiment, the behavior determination unit 236 generates an utterance content based on an article generated using an article generation model. Specifically, the behavior determination unit 236 inputs a text or a sound input by the user 10, the emotions of both the user 10 and the character determined by the emotion determination unit 232, and the conversation history saved in the history data 2222 to the article generation model, and generates an utterance content of the agent. At this time, the behavior determination unit 236 can also input the character personality set by the character setting unit 276 to the article generation model, and generate an utterance content of the agent. In the agent system 500, the article generation model is not located on the front end side that is a contact point with the user 10, but is always used as a tool of the agent system 500.

[0524] The command acquisition unit 272 acquires a command of the agent from a sound or a text issued from the user 10 through a conversation with the user 10 using the output of the utterance understanding unit 212. The command includes a behavior content that the agent system 500 should perform, such as searching for information, reserving a hotel, arranging a ticket, purchasing a product / service, paying a bill, route guidance to a destination, providing a recommendation, and the like.

[0525] The RPA 274 performs a behavior according to the command acquired by the command acquisition unit 272. The RPA 274 performs a behavior related to the use of a service provider, such as searching for information, reserving a hotel, arranging a ticket, purchasing a product / service, paying a bill, and the like.

[0526] The RPA 274 reads and utilizes, from the history data 2222, personal information of the user 10 required for performing a behavior related to the utilization of the service provider. For example, in a case where the agent system 500 makes a commodity purchase according to a request from the user 10, the personal information of the user 10 such as the name, the address, the telephone number, the credit card number, and the like saved in the history data 2222 is read and utilized. It is not friendly to require the user 10 to input the personal information at the initial setting, and it is also unpleasant for the user. In the agent system 500 of the present embodiment, instead of requiring the user 10 to input the personal information at the initial setting, the personal information acquired through the dialogue with the user 10 is stored in advance, and is read and utilized as needed. Thereby, it is possible to avoid giving the user an unpleasant feeling, and to improve the convenience of the user.

[0527] The agent system 500 performs the dialogue processing, for example, by the following steps 1 to 5.

[0528] (Step 1) The agent system 500 sets the role of the agent. Specifically, the role setting section 276 sets the role of the agent when the agent system 500 has a dialogue with the user 10, according to the designation from the user 10.

[0529] (Step 2) The agent system 500 acquires the state of the user 10 including the voice or the text input by the user 10, the emotion value of the user 10, the emotion value of the agent, the history data 2222. Specifically, the same processing as the above steps S100 to S103 is performed, and the state of the user 10 including the voice or the text input by the user 10, the emotion value of the user 10, the emotion value of the agent, and the history data 2222 are acquired.

[0530] (Step 3) The agent system 500 determines the utterance content of the agent. Specifically, the behavior determination section 236 inputs the text or the voice input by the user 10, the emotions of both the user 10 and the role specified by the emotion determination section 232, and the dialogue history saved in the history data 2222 into the article generation model, and generates the utterance content of the agent.

[0531] For example, among the texts representing the text or the voice input by the user 10, the emotions of both the user 10 and the role determined by the emotion determination section 232, and the dialogue history saved in the history data 2222, a fixed sentence such as "At this time, how should the agent answer?" is added, and the article generation model is input, and the utterance content of the agent is acquired.

[0532] As an example, in a case where the text or the voice input by the user 10 is "I would like to make a reservation for a delicious Chinese restaurant near here at 7 o'clock tonight", as the utterance content of the agent, "OK", "This is a recommended restaurant. 1, AAAA. 2, BBBB. 3, CCCC. 4, DDDD" is acquired.

[0533] In addition, in a case where the text or voice input by the user 10 is "the fourth DDDD", the utterance content of the agent is "OK. Try making a reservation. How many seats".

[0534] (Step 4) The agent system 500 outputs the utterance content of the agent. Specifically, the behavior control section 250 synthesizes a voice corresponding to the character set by the character setting section 276, and outputs the utterance content of the agent in accordance with the synthesized voice.

[0535] (Step 5) The agent system 500 determines whether it is a timing to execute the command of the agent. Specifically, the behavior determination section 236 determines whether it is a timing to execute the command of the agent on the basis of the output of the article generation model. For example, in a case where the output of the article generation model includes the command of the agent to execute, it is determined that it is a timing to execute the command of the agent, and the process proceeds to Step 6. On the other hand, in a case where it is determined that it is not a timing to execute the command of the agent, the process returns to Step 2 described above.

[0536] (Step 6) The agent system 500 executes the command of the agent. Specifically, the command acquisition section 272 acquires the command of the agent from the voice or text issued by the user 10 through the dialogue with the user 10. Then, the RPA 274 performs a behavior in accordance with the command acquired by the command acquisition section 272. For example, in a case where the command is "search for information", the search query and the API (Application Programming Interface) obtained through the dialogue with the user 10 are used to search for information through a search site. The behavior determination section 236 inputs the search result into the article generation model, and generates the utterance content of the agent. The behavior control section 250 synthesizes a voice corresponding to the character set by the character setting section 276, and outputs the utterance content of the agent in accordance with the synthesized voice.

[0537] In addition, in a case where the command is "make a reservation for a hotel", the reservation information obtained through the dialogue with the user 10, the hotel information of the reservation destination, and the API are used to make a reservation by making a phone call to the hotel of the reservation destination through a phone software. At this time, the behavior determination section 236 uses the article generation model having a dialogue function, and acquires the utterance content of the agent with respect to the voice input by the counterpart. Then, the behavior determination section 236 inputs the reservation result of the hotel (whether the reservation is correct or not) into the article generation model, and generates the utterance content of the agent. The behavior control section 250 synthesizes a voice corresponding to the character set by the character setting section 276, and outputs the utterance content of the agent in accordance with the synthesized voice.

[0538] Then, the process returns to Step 2 described above.

[0539] Accordingly, the agent system 500 can perform dialogue processing and, as necessary, perform behavior related to utilization of a service provider.

[0540] Figure 14 and 15 An example of the operation of the agent system 500 is shown. Figure 14 An example of the way in which the agent system 500 makes a reservation for a restaurant through dialogue with the user 10 is shown. In Figure 14 In the left side, the content of the utterance of the agent is shown, and in the right side, the content of the utterance of the user 10 is shown. The agent system 500 can understand the preference of the user 10 based on the history of the dialogue with the user 10, provide a list of recommended restaurants that match the preference of the user 10, and perform a reservation for a selected restaurant.

[0541] On the other hand, Figure 15 An example of the way in which the agent system 500 makes a reservation for a restaurant through dialogue with the user 10 is shown. In Figure 15 In the left side, the content of the utterance of the agent is shown, and in the right side, the content of the utterance of the user 10 is shown. The agent system 500 can understand the preference of the user 10 based on the history of the dialogue with the user 10, provide a list of recommended restaurants that match the preference of the user 10, and perform a reservation for a selected restaurant.

[0542] Further, the other structures and functions of the agent system 500 of the fourth embodiment are the same as those of the robot 100 of the second embodiment, and thus the description thereof is omitted.

[0543] Further, in the above-described embodiments, the case in which the robot 100 recognizes the user 10 using the facial image of the user 10 is described, but the disclosed technology is not limited to this aspect. For example, the robot 100 can recognize the user 10 using the voice emitted by the user 10, the mail address of the user 10, the ID of the SNS of the user 10, or an ID card of a built-in wireless IC tag held by the user 10, and the like.

[0544] The robot 100 is an example of an electronic machine provided with a behavior control system. The application target of the behavior control system is not limited to the robot 100, and the behavior control system can be applied to various electronic machines. Further, the functions of the server 300 can be implemented by one or more computers. At least a part of the functions of the server 300 can be implemented by a virtual machine. Further, at least a part of the functions of the server 300 can be implemented by a cloud.

[0545] (Fifth Embodiment) The fifth embodiment is an example configured to apply the response processing and autonomous processing of the behavior control system of the second embodiment and the intelligent agent function of the fourth embodiment to the plush toy of the third embodiment. Hereinafter, parts that have the same structure as the second to fourth embodiments will be labeled with the same symbols and their descriptions will be omitted.

[0546] The robot 100 of this embodiment (in this embodiment, it is equivalent to the smartphone 50 stored in the plush toy 100N) performs the following processing.

[0547] For example, when robot 100 is set up in an event venue, environmental information about the venue is acquired. This environmental information could include the atmosphere of the event venue and the purpose of robot 100. The atmosphere information is expressed numerically as a quiet atmosphere, a bright atmosphere, a somber atmosphere, etc. This atmosphere information can be acquired, for example, by sensor unit 200. The purpose of robot 100 could include, for example, being a warm-up person or a guide. Behavior determination unit 236 adds a fixed sentence such as "What are the lyrics and melody suitable for the current atmosphere?" to the text representing the environmental information, inputs it into the text generation model, and acquires recommended lyrics and melody scores related to the environment in which robot 100 is located.

[0548] In this embodiment, the robot 100 is equipped with a voice synthesis engine. The behavior determination unit 236 inputs the lyrics and melody score obtained from the article generation model into the voice synthesis engine, causing the robot 100 to play music based on the lyrics and melody obtained from the article generation model. Furthermore, the behavior determination unit 236 determines the behavior content of the robot 100 to perform a dance corresponding to the music being played. At this time, the LEDs in the robot 100's eyes may also be made to flash according to the dance.

[0549] Therefore, Robot 100 can improvise music that corresponds to the atmosphere of the event venue and the role of Robot 100, and perform dances that match the music, thus raising the atmosphere of the event venue.

[0550] Furthermore, the processing described in the fifth embodiment can be performed in each of the response processing and autonomous processing in the behavior control system of the second embodiment, and the processing described in the fifth embodiment can also be performed in the agent function of the fourth embodiment.

[0551] (Note 1) A behavior control system, comprising: The status recognition unit identifies user status, including user behavior. The emotion determination department determines the user's or the robot's emotions; and a behavior determination section that determines a behavior of the robot corresponding to the user state and the emotion of the user or the emotion of the robot, based on an article generation model having a conversation function with which the user converses with the robot, The behavior determination section determines a behavior content of the robot to acquire a musical score of a lyric and a melody corresponding to an environment in which the robot is located, based on the article generation model, and play music using a sound synthesis engine and based on the lyric and the melody.

[0552] (Addendum 2) The behavior control system according to Addendum 1, wherein the behavior determination section determines a behavior content of the robot to further perform a motion corresponding to the music.

[0553] (Addendum 3) The behavior control system according to Addendum 1 or 2, wherein the robot is mounted on a stuffed toy, or connected wirelessly or by wire with a control target robot mounted on a stuffed toy.

[0554] (Addendum 4) The behavior control system according to Addendum 3, wherein the control target robot is a speaker, The stuffed toy is mounted with a microphone or a camera.

[0555] (Addendum 5) The behavior control system according to Addendum 4, wherein the camera is installed at an eye portion constituting a face portion of the stuffed toy, the microphone is installed at an ear portion, and the speaker is installed at a mouth portion.

[0556] (Addendum 6) The behavior control system according to Addendum 3, wherein a wireless power reception section that receives wireless power from a wireless power supply section from outside is arranged inside the stuffed toy, The control target robot or the robot is powered via the wireless power reception section.

[0557] (Sixth Embodiment) The robot 100 investigates a topic that the user is interested in or information related to the interest, for example, even when there is no conversation with the user.

[0558] The robot 100 investigates information related to the birthday or the anniversary of the user, and considers a message of a blessing, for example, even when there is no conversation with the user.

[0559] The robot 100 investigates a place or food, a comment on a product that the user wants to go to, for example, even when there is no conversation with the user.

[0560] The robot 100 investigates weather information, for example, even when there is no conversation with the user, and provides a suggestion that fits the user's schedule or plan.

[0561] The robot 100 investigates information on local events or festivals, for example, even when there is no conversation with the user, and proposes to the user.

[0562] The robot 100 investigates and introduces information on a music or artist that the user likes, for example, even when there is no conversation with the user.

[0563] The robot 100 investigates and introduces information on a music or artist that the user likes, for example, even when there is no conversation with the user.

[0564] The robot 100 investigates and introduces information on a music or artist that the user likes, for example, even when there is no conversation with the user.

[0565] The robot 100 investigates and introduces information on a music or artist that the user likes, for example, even when there is no conversation with the user.

[0566] The robot 100 investigates and introduces information on a music or artist that the user likes, for example, even when there is no conversation with the user.

[0567] The robot 100 investigates and introduces information on a music or artist that the user likes, for example, even when there is no conversation with the user.

[0568] The robot 100 investigates and introduces information on a music or artist that the user likes, for example, even when there is no conversation with the user.

[0569] The robot 100 investigates and introduces information on a music or artist that the user likes, for example, even when there is no conversation with the user.

[0570] The robot 100 investigates and introduces information on a music or artist that the user likes, for example, even when there is no conversation with the user.

[0571] The robot 100 investigates and introduces information on a music or artist that the user likes, for example, even when there is no conversation with the user.

[0572] The robot 100 investigates and introduces information on a music or artist that the user likes, for example, even when there is no conversation with the user.

[0573] The robot 100 investigates and introduces information on a music or artist that the user likes, for example, even when there is no conversation with the user.

[0574] The robot 100 investigates information on a restaurant or a dining store that the user likes, for example, even when there is no conversation with the user, and proposes a suggestion.

[0575] The robot 100 collects information on and proposes a suggestion for an important decision related to the user's life, for example, even when there is no conversation with the user.

[0576] The robot 100 investigates information on a person that the user is concerned about, for example, even when there is no conversation with the user, and proposes a suggestion.

[0577] (Seventh Embodiment) The robot 100 of the present embodiment (in the present embodiment, corresponds to the smartphone 50 housed in the stuffed toy 100N) executes the following processing.

[0578] The behavior determination section 236 determines the behavior of the robot 100 corresponding to the user state and the emotion of the user 10 or the emotion of the robot 100, based on an article generation model having a conversation function with which the user 10 and the robot 100 have a conversation. At this time, the behavior determination section 236 generates a life improvement application (hereinafter referred to as a life improvement application) that proposes an improvement in life to the user 10, based on the conversation between the user 10 and the robot 100. That is, the behavior determination section 236 determines the behavior of the robot 100 to function as the generated life improvement application.

[0579] The robot 100 proposes a life improvement plan to the user 10 to improve the life habit of the user 10 that becomes a main cause of causing anxiety or stress, hypertension, or diabetes and the like, based on the exchange, that is, the conversation between the user 10 and the robot 100. Specifically, the robot 100 (in the present embodiment, corresponds to the smartphone 50 housed in the stuffed toy 100N) executes the processing of proposing a life improvement plan by executing a life improvement application by the following steps 1 to 4.

[0580] (Step 1) The robot 100 acquires the state of the user 10, the emotion value of the user 10, the emotion value of the robot 100, and the history data 2222. Specifically, the same processing as the above steps S100 to S103 is performed to acquire the state of the user 10, the emotion value of the user 10, the emotion value of the robot 100, and the history data 2222.

[0581] (Step 2) The robot 100 acquires information related to the life that the user 10 should improve. Specifically, the behavior determination section 236 determines the behavior content of the robot 100 to make the user 10 utter a question such as "How many hours of sleep did you get today?", "Did you exercise today?", and "What is your blood pressure today?" and the like related to the life that should be improved. The behavior control section 250 controls the control object 252 to make the user 10 be questioned about the content related to the life that should be improved. The state recognition section 230 recognizes the information related to the life that the user 10 should improve from the user 10 based on the information (for example, the user's answer) analyzed by the sensor module section 210.

[0582] (Step 3) The robot 100 determines a life improvement plan to be proposed to the user 10. In addition, here, as the life improvement plan, for example, a diet content or sleep and the like for improving lifestyle-related diseases such as hypertension, diabetes, and the like can be cited. Specifically, the behavior determination section 236 adds a fixed sentence such as "What is the life improvement plan to be recommended to the user at this time?" to the text indicating the information related to the life that the user 10 should improve, the user's 10 emotions, the robot's 100 emotions, and the content saved in the history data 2222, and inputs the article generation model to acquire the recommended content related to the life improvement plan. At this time, not only the information related to the life improvement plan of the user 10 is considered, but also the emotional life model of the user 10 is considered based on the emotions of the user 10 or the history data 2222 and the like, whereby the life improvement plan suitable for the user 10 can be proposed.

[0583] (Step 4) The robot 100 proposes the life improvement plan determined in Step 3 to the user 10. Specifically, the behavior determination section 236 determines the utterance to propose the life improvement plan to the user 10 as the behavior of the robot 100, and the behavior control section 250 controls the control object 252 to make the utterance to propose the life improvement plan to the user 10.

[0584] In this way, the robot 100 can function as a life improvement application that proposes a life improvement plan based on the emotional life model of the user 10 and the conversation between the user 10 and the robot 100.

[0585] In addition, the above-described emotion table (refer to Table 2) can also be used to determine the behavior of the robot 100 as with the sixth embodiment. For example, in a case where the user's behavior is the remark "Start the life improvement application", the robot's 100 emotion is the index number "2", and the user's 10 emotion is the index number "3", "The robot is in a very happy state. The user is in a generally happy state. The user has made a remark "Start the life improvement application". How should the robot respond?" is input to the article generation model to acquire the behavior content of the robot. The behavior determination section 236 determines the behavior of the robot according to the behavior content.

[0586] (Note 1) A behavior control system includes: a state recognition unit that recognizes a user state including a behavior of a user; an emotion determination unit that determines an emotion of the user or an emotion of a robot; and a behavior determination unit that determines a behavior of the robot corresponding to the user state and the emotion of the user or the emotion of the robot, based on an article generation model having a conversation function with which the user converses with the robot, the behavior determination unit generates a life improvement application that proposes improvement of life, based on a conversation of the user with the robot.

[0587] (Note 2) The behavior control system according to Note 1, wherein the robot is mounted on a stuffed toy, or connected in a wireless or wired manner with a control target robot mounted on a stuffed toy.

[0588] (Note 3) The behavior control system according to Note 2, wherein the control target robot is a speaker, the stuffed toy is mounted with a microphone or a camera.

[0589] (Note 4) The behavior control system according to Note 3, wherein the camera is installed at an eye portion constituting a face of the stuffed toy, the microphone is installed at an ear portion, and the speaker is installed at a mouth portion.

[0590] (Note 5) The behavior control system according to Note 2, wherein a wireless power reception unit that receives wireless power from an external wireless power supply unit is arranged inside the stuffed toy, the control target robot or the robot is powered via the wireless power reception unit.

[0591] (Eighth Embodiment) The robot 100 (in this embodiment, corresponds to the smartphone 50 housed in the stuffed toy 100N) of this embodiment is linked with a machine class capable of detecting a health state of the user 10 such as a body composition scale and a sphygmomanometer, a machine class that stores foodstuffs such as a refrigerator and a freezer. The robot 100 performs not only scheduled management of the user 10 or a speech of news, but also a suggestion or a recommendation of a dish related to the physical condition of the user 10, a proposal of foodstuffs that should be replenished, and a process of automatic ordering of foodstuffs, and performs diet management based on a user state.

[0592] In a case where the robot 100 performs diet management, data related to the health state of the user 10 is acquired from the server 300 or other external server or the like. For example, in a case where the user 10 measures body composition using a prescribed body composition scale, data related to the body composition is automatically transmitted to the server 300 and stored by date. The "data related to the body composition" described here includes body weight, body fat rate, visceral fat, and muscle mass, and the like. In addition, in a case where the user 10 measures blood pressure using a prescribed sphygmomanometer, data related to the blood pressure is automatically transmitted to the server 300 and stored by date. The robot 100 is configured to be able to grasp changes in the body composition and changes in the blood pressure of the user 10 by acquiring data related to the body composition and the blood pressure of the user 10 for a prescribed period from the server 300.

[0593] The robot 100 provides advice on the physical condition to the user 10 on the basis of the changes in the body composition and the changes in the blood pressure of the user 10 and the emotion of the user 10 or the emotion of the robot 100. For example, in a case where the user 10 has a tendency to decrease in body weight, the robot 100 makes a speech to the user 10 via the speaker 60 of content that it is better to increase the food intake because of the tendency to decrease in the body weight. At this time, the robot 100 can determine the language to say to the user 10 using an article production model, and the robot 100 can express a sad emotion or an uneasy emotion by the eyes 56. In addition, for example, in a case where the user 10 has a tendency to increase in body weight and the user 10 has a tendency to increase in blood pressure, the robot 100 can also make a speech to the user 10 via the speaker 60 of content that attention should be paid to the intake of a large number of calories. At this time, the robot 100 can also express a worried emotion by the eyes 56. In addition, the robot 100 can also make a speech to the user 10 via the speaker 60 of content that recommends exercise.

[0594] Further, in a case where the robot 100 proposes a recommended dish, proposes a food material that should be supplemented, and automatically orders a food material, the robot 100 acquires data of the food material stored in the refrigerator and the freezer. For example, it can also be configured to provide a camera inside the refrigerator and inside the freezer, acquire information related to the food material stored inside the refrigerator and inside the freezer on the basis of image data captured by the camera, and store in the server 300. The information related to the food material can include information such as a consumption period.

[0595] The robot 100 proposes a recipe and proposes a food material that should be replenished, based on information of food materials stored in the server 300. For example, the robot 100 infers a dish that the user 10 has recently eaten, a dish that the user 10 wants to eat, and the like, based on a conversation with the user 10, proposes a recipe to the user 10 via the speaker 60, taking into account the health status of the user 10. At this time, the priority of a recipe that can make more use of food materials stored in the refrigerator and the freezer can also be increased. In addition, the robot 100 can also analyze a dish that the user 10 has eaten, based on the content captured by the 2D camera 203 in the diet of the user 10, thereby grasping a dish that the user 10 has recently eaten.

[0596] In addition, the robot 100 can also propose a food material that is insufficient or a food material predicted to be insufficient to the user 10 in a case where the user 10 agrees to the proposed recipe. Furthermore, if permission for automatic ordering is given in advance by the user 10, the robot 100 purchases an insufficient food material on a prescribed food material sales website. In this case, the robot 100 conveys information of the purchased food material to the user. In addition, in a case where the robot 100 proposes a recipe, the preference of the user 10 is taken into account.

[0597] The robot 100 (in the present embodiment, corresponds to the smartphone 50 housed in the stuffed toy 100N) executes a process of determining a proposed recipe, based on the preference of the user, the status of the user, and the reaction of the user, by the following steps 1 to 5-2.

[0598] (Step 1) The robot 100 acquires the status of the user 10, the emotional value of the user 10, the emotional value of the robot 100, and the history data 2222. Specifically, the same processes as steps S100 to S103 described above are performed, and the status of the user 10, the emotional value of the user 10, the emotional value of the robot 100, and the history data 2222 are acquired.

[0599] (Step 2) The robot 100 acquires the preference of the user 10 with respect to food. Specifically, the behavior determination unit 236 determines a speech that asks the user 10 about the preference with respect to food as a behavior of the robot 100, and the behavior control unit 250 controls the control target 252 to ask the user 10 about the preference with respect to food or food materials. The status recognition unit 230 recognizes the preference of the user 10 with respect to food, based on information (for example, the answer of the user) analyzed by the sensor module unit 210.

[0600] (Step 3) The robot 100 determines the contents of the recipe to be proposed to the user 10. Specifically, the behavior determination section 236 adds a fixed sentence such as "What food is recommended to the user at this time?" to the text representing the contents stored in the history data 2222 regarding the user's 10 preference for food, the user's 10 emotion, the robot's 100 emotion, and the article generation model, and acquires recommended contents regarding food. At this time, by considering not only the user's 10 preference for food but also the user's 10 emotion or the history data 2222, it is possible to make a proposal suitable for the user 10. In addition, by considering the robot's 100 emotion, it is possible to make the user 10 feel that the robot 100 has an emotion.

[0601] (Step 4) The robot 100 proposes the recipe determined in Step 3 to the user 10 and acquires the user's 10 reaction. Specifically, the behavior determination section 236 determines the utterance to be proposed to the user 10 as the behavior of the robot 100, the behavior control section 250 controls the control object 252, and the utterance to be proposed to the user 10 is made. The state recognition section 230 recognizes the state of the user 10 on the basis of the information analyzed by the sensor module section 210, and the emotion determination section 232 determines the emotional value representing the emotion of the user 10 on the basis of the information analyzed by the sensor module section 210 and the state of the user 10 recognized by the state recognition section 230. The behavior determination section 236 judges whether the reaction of the user 10 is positive or not on the basis of the state of the user 10 recognized by the state recognition section 230 and the emotional value representing the emotion of the user 10.

[0602] (Step 5-1) In the case where the reaction of the user 10 is positive, the robot 100 performs the process of confirming the ingredients required for the proposed recipe.

[0603] (Step 5-2) In the case where the reaction of the user 10 is not positive, the robot 100 determines another recipe to be proposed to the user 10. Specifically, in the case where another recipe to be proposed to the user 10 is determined as the behavior of the robot 100, the behavior determination section 236 adds a fixed sentence such as "Is there another food recommended to the user?" to the text representing the contents stored in the history data 2222 regarding the user's 10 preference for food, the user's 10 emotion, the robot's 100 emotion, and the article generation model, and acquires recommended contents regarding food. Then, the above-described Step 4 is returned to, and the above-described Steps 4 to 5-2 are repeated until the process of confirming the ingredients required for the recipe to be proposed to the user 10 is determined to be performed.

[0604] Alternatively, the same sentiment table (refer to Table 1 above) can be used to determine the behavior of robot 100 as in the second embodiment. For example, if the user's behavior is to say "I'll make the dish you suggested today," and robot 100's sentiment is index number "2" and user 10's sentiment is index number "3," then the following input is used to generate an article model to obtain the robot's behavior content: "The robot is in a very happy state. The user is in a normally happy state. The user suggested 'I'll make the dish you suggested today.' How should the robot respond?" The behavior determination unit 236 determines the robot's behavior based on this behavior content.

[0605] Furthermore, the processing described above in the fifth embodiment can be performed in each of the response processing and autonomous processing in the behavior control system of the first embodiment, or the processing described above in the fifth embodiment can be performed in the agent function of the fourth embodiment.

[0606] (Note 1) A behavior control system, comprising: The status recognition unit identifies user status, including user behavior. The emotion determination unit determines the emotions of the user or the electronic device; and The behavior determination unit, based on an article generation model that enables dialogue between the user and the electronic machine, determines the behavior of the electronic machine corresponding to the user's state and the user's emotion, or the behavior of the electronic machine corresponding to the user's state and the electronic machine's emotion. The behavior determination unit performs dietary management based on the user's status.

[0607] (Note 2) According to the behavior control system described in Appendix 1, the behavior determination unit makes at least one of the following suggestions to the user: suggestions related to physical condition, suggestions of recipes, and suggestions of ingredients to be supplemented.

[0608] (Note 3) According to the behavior control system described in Appendix 1, the behavior determination unit orders insufficient ingredients or predicts that there will be insufficient ingredients.

[0609] (Note 4) According to the behavior control system described in App...

Claims

1. A behavior control system, comprising: The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, generates behavioral content for the robot based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavioral content. The behavior determination unit has the following features: The image acquisition unit is capable of capturing images of the competition space, where a specific competition can be carried out. and The competitor emotion analysis unit analyzes the emotions of multiple competitors engaged in competition within the competition space, as captured by the image acquisition unit. The robot's behavior is determined based on the analysis results of the competitor's emotion analysis unit.

2. The behavior control system according to claim 1, wherein, The competitor sentiment analysis unit analyzes the sentiments of competitors belonging to a specific team among the multiple competitors.

3. The behavior control system according to claim 1 or claim 2, wherein, The robot is mounted on a plush toy, or connected wirelessly or via a wired connection to a controllable machine mounted on a plush toy.

4. A behavior control system, comprising: The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, generates behavioral content for the robot based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavioral content. The behavior determination unit has the following features: The image acquisition unit is capable of capturing images of a competition space where a specific competition can be conducted; and The feature determination unit determines the features of multiple competitors engaged in competition within the competition space captured by the image acquisition unit. The robot's behavior is determined based on the determination result of the feature determination unit.

5. A behavior control system, comprising: The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, generates behavioral content for the robot based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavioral content. When the behavior determination unit is activated at a specified time, it obtains a summary of the previous day's history by adding a fixed sentence to the text representing the previous day's historical data and inputting it into the article generation model, and then speaks the content of the obtained summary.

6. A behavior control system, comprising: The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, generates behavioral content for the robot based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavioral content. When the behavior determination unit is activated at a specified time, it obtains a summary of the previous day's history by adding a fixed sentence to the text representing the previous day's historical data and inputting it into the article generation model. Then, it obtains an image summarizing the previous day's history by inputting the obtained summary of the previous day's history into the image generation model and displays the obtained image.

7. A behavior control system, comprising: The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, generates behavioral content for the robot based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavioral content. When the behavior determination unit is activated at a predetermined time, it determines the robot's emotion corresponding to the previous day's history by adding a fixed sentence to the text representing the previous day's historical data and inputting it into the article generation model.

8. A behavior control system, comprising: The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on the dialogue function that enables the user to converse with the robot, generates robot behavior content based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavior content. The behavior determination unit determines the emotion based on the user's previous day's history by adding historical data, including the user's behavior and emotions from the previous day, to a fixed sentence that asks the user about their emotions based on the historical data and inputting it into the dialogue function when the user wakes up.

9. A behavior control system, comprising: The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on the dialogue function that enables the user to converse with the robot, generates robot behavior content based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavior content. The behavior determination unit acquires a summary of historical data, including the user's behavior and emotions from the previous day, at the time the user wakes up, acquires music based on the summary, and plays the music.

10. A behavior control system, comprising: The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, generates behavioral content for the robot based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavioral content. When the behavior determination unit is engaged in a battle game with the user, in which a winner or loser is determined, it determines the user level, which represents the strength of the user in the battle game, and sets a robot level, which represents the strength of the robot, based on the determined user level.

11. A behavior control system, comprising: The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on the dialogue function that enables the user to converse with the robot, generates robot behavior content based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavior content. When the behavior determination unit receives a question from the user about which of two or more things to choose, it selects at least one of the two or more things and answers the user, based at least on historical information related to the user.

12. A behavior control system, comprising: The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on the dialogue function that enables the user to converse with the robot, generates robot behavior content based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavior content. The behavior determination unit stores the types of behaviors performed by the user within the home as specific information corresponding to the timing of performing the behavior. Based on the specific information, it determines the timing of the behavior when the user should perform it and notifies the user.

13. A behavior control system, comprising: The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, generates behavioral content for the robot based on the user's behavior and the user's or robot's emotions, and determines the robot's behavior corresponding to the behavioral content. The behavior determination unit receives the statements of multiple users who are having a conversation, outputs the topic of the conversation, and outputs other topics based on the emotions of at least one of the users who are having the conversation to determine the robot's behavior.

14. A behavior control system, comprising: The status recognition unit identifies user status, including user behavior. The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, determines the robot's behavior corresponding to the user's state and the user's or robot's emotions. The behavior determination unit determines the behavior content of the robot, obtains the musical score of lyrics and melody corresponding to the environment in which the robot is located based on the article generation model, and uses a voice synthesis engine to play music based on the lyrics and melody.

15. A behavior control system, comprising: The status recognition unit identifies user status, including user behavior. The emotion determination department determines the emotions of the user or the robot. and The behavior determination unit, based on an article generation model that enables dialogue between the user and the robot, determines the robot's behavior corresponding to the user's state and the user's or robot's emotions. The behavior determination unit generates life improvement applications that propose improvements to the user's life based on the dialogue between the user and the robot.

16. A behavior control system, comprising: The status recognition unit identifies user status, including user behavior. The emotion determination unit determines the emotions of users or electronic devices; and The behavior determination unit, based on an article generation model that enables dialogue between the user and the electronic machine, determines the behavior of the electronic machine corresponding to the user's state and the user's emotion, or the behavior of the electronic machine corresponding to the user's state and the electronic machine's emotion. The behavior determination unit performs dietary management based on the user's status.

17. A behavior control system, comprising: The status recognition unit identifies the user status, including user behavior, and the status of electronic devices. An emotion determination unit determines the emotion of the user or the emotion of the electronic device; and The behavior determination unit determines the behavior of the electronic device based on at least one of the user's state, the state of the electronic device, the user's emotion, and the electronic device's emotion. When the behavior determination unit determines that answering a user's question is the behavior of the electronic machine... Obtain the vector representing the user's question; and From a database storing combinations of questions and answers, a question with a vector corresponding to the retrieved vector is retrieved. Using the answer to the retrieved question and an article generation model with dialogue functionality, an answer is generated for the user's question.

18. A control system, comprising: The input section receives user input. The processing department uses an article generation model that generates articles based on input data to perform specific processing. and The output unit controls the behavior of the electronic device to output the result of the specific processing. The processing unit determines whether the predetermined triggering conditions meet the conditions for the content to be displayed in the user's meeting. When the triggering condition is met, and when at least the email records, reservation records, and meeting speech items obtained from user input within a specific period are used as the input data, the output of the article generation model is used to obtain and output responses related to the content presented in the meeting, as the result of the specific processing.

19. An information processing system, comprising: The input section receives user input. The processing unit performs specific processing using a generative model that generates results based on the input data; and The output unit controls the behavior of the electronic device to output the result of the specific processing. When the processing unit takes text indicating earthquake-related information as input data, it uses the output of the generation model to obtain the result of the specific processing.

20. A behavior control system, comprising: The status recognition unit identifies the user status, including the user's behavior, and the robot's status. An emotion determination unit determines the emotion of the user or the emotion of the robot; and The behavior determination unit, at a predetermined time, uses at least one of the user's state, the robot's state, the user's emotion, and the robot's emotion, along with a behavior determination model, to determine any one of a variety of machine behaviors, including not performing any behavior, as the robot's behavior.

21. A behavior control system, comprising: The status recognition unit identifies the user status, including user behavior, and the status of electronic devices. An emotion determination unit determines the emotion of the user or the emotion of the electronic device; The behavior determination unit uses at least one of the user state, the electronic machine state, the user's emotion, and the electronic machine's emotion, along with a behavior determination model, at a predetermined time to determine any one of a variety of machine actions, including not performing any action, as the robot's behavior. and The storage control unit stores event data in historical data, which includes sentiment values ​​determined by the sentiment determination unit and data including the user's behavior. The machine actions include proposal activities. When the behavior determination unit determines that the proposed activity is an action of the electronic machine, it determines, based on the event data, the user's proposed action.

22. A behavior control system, comprising: The status recognition unit identifies the user status, including user behavior, and the status of electronic devices. An emotion determination unit determines the emotion of the user or the emotion of the electronic device; The behavior determination unit uses at least one of the user state, the state of the electronic machine, the user's emotion, and the electronic machine's emotion, along with a behavior determination model, at a predetermined time to determine any one of a variety of machine actions, including not performing any action, as the behavior of the electronic machine. and The storage control unit stores event data in historical data, which includes sentiment values ​​determined by the sentiment determination unit and data including the user's behavior. The machine's actions include facilitating communication with others. When the behavior determination unit determines that promoting communication with others is the behavior of the electronic machine, it determines at least one of the communication partner or communication method based on the event data.

23. A behavior control system, comprising: The status recognition unit identifies the user status, including user behavior, and the status of electronic devices. An emotion determination unit determines the emotion of the user or the emotion of the electronic device; and The behavior determination unit, at a predetermined time, uses at least one of the user state, the state of the electronic machine, the user's emotion, and the electronic machine's emotion, along with a behavior determination model, to determine any one of a variety of machine actions, including inaction, as the behavior of the electronic machine. The machine's actions include providing suggestions related to the specific competition to the user participating in that competition. The behavior determination unit includes: The image acquisition unit is capable of capturing images of the competition space, in which the specific competition in which the user participates can be carried out; and The competitor analysis unit analyzes the emotions of multiple competitors performing the specific competition in the competition space captured by the image acquisition unit. When the electronic machine determines that making suggestions related to the specific competition to the user participating in the specific competition is an action of the specific competition, suggestions are made to the user based on the analysis results of the competitor analysis unit.

24. A behavior control system, comprising: The status recognition unit identifies the user status, including user behavior, and the status of electronic devices. An emotion determination unit determines the emotion of the user or the emotion of the electronic device; and The behavior determination unit, at a predetermined time, uses at least one of the user state, the state of the electronic machine, the user's emotion, and the electronic machine's emotion, along with a behavior determination model, to determine any one of a variety of machine actions, including inaction, as the behavior of the electronic machine. The machine's actions include providing suggestions related to the specific competition to the user participating in that competition. The behavior determination unit has the following features: The image acquisition unit is capable of capturing images of the arena, in which the specific competition in which the user participates can be conducted; and The feature determination unit determines the features of multiple competitors engaged in competition within the competition space captured by the image acquisition unit. When the electronic machine determines that making suggestions related to the specific competition to the user participating in the specific competition is the behavior of the specific competition, suggestions are made to the user based on the determination result of the feature determination unit.

25. A behavior control system, comprising: The status recognition unit identifies the user status, including user behavior, and the status of electronic devices. An emotion determination unit determines the emotion of the user or the emotion of the electronic device; The behavior determination unit uses at least one of the user state, the electronic machine state, the user's emotion, and the electronic machine's emotion, along with a behavior determination model, at a predetermined time to determine any one of a variety of machine actions, including not performing any action, as the robot's behavior. and The storage control unit stores event data, including the emotion value determined by the emotion determination unit and data including the user's behavior, in historical data. The machine actions include setting a first action to correct the user's behavior. The behavior determination unit spontaneously or periodically detects the user's behavior, and if, based on the detected user behavior and pre-stored specific information, it determines that correcting the user's behavior is the behavior of the electronic machine, it executes the first behavior content.

26. A behavior control system, comprising: The status recognition unit identifies the user status, including user behavior, and the status of electronic devices. An emotion determination unit determines the emotion of the user or the emotion of the electronic device; The behavior determination unit uses at least one of the user state, the state of the electronic machine, the user's emotion, and the electronic machine's emotion, along with a behavior determination model, at a predetermined time to determine any one of a variety of machine actions, including not performing any action, as the behavior of the electronic machine. and The storage control unit stores event data in historical data, which includes sentiment values ​​determined by the sentiment determination unit and event data including data on the user's behavior. The machine's actions include providing users with home-related advice. When the behavior determination unit determines that making suggestions to the user related to the home is the behavior of the electronic machine, it uses an article generation model based on the data related to the machine in the home stored in the historical data to suggest suggestions or recommended dishes, ingredients that should be supplemented, etc., related to the user's physical condition.

27. A behavior control system, comprising: The status recognition unit identifies the user status, including user behavior, and the status of electronic devices. An emotion determination unit determines the emotion of the user or the emotion of the electronic device; and The behavior determination unit, at a predetermined time, uses at least one of the user state, the state of the electronic machine, the user's emotion, and the electronic machine's emotion, along with a behavior determination model, to determine any one of a variety of machine actions, including inaction, as the behavior of the electronic machine. The machine's actions include providing the user with advice related to labor issues. When the behavior determination unit determines that making suggestions related to labor issues to the user is an action of the electronic machine, it determines to make suggestions related to labor issues to the user based on the user's behavior.

28. A behavior control system, comprising: The status recognition unit identifies the user status, including user behavior, and the status of electronic devices. An emotion determination unit determines the emotion of the user or the emotion of the electronic device; The behavior determination unit uses at least one of the user state, the state of the electronic machine, the user's emotion, and the electronic machine's emotion, along with a behavior determination model, at a predetermined time to determine any one of a variety of machine actions, including not performing any action, as the behavior of the electronic machine. and The storage control unit stores event data in historical data, which includes sentiment values ​​determined by the sentiment determination unit and data including the user's behavior. The machine's actions include suggesting actions that the user within the household might take. The storage control unit establishes a correspondence between the types of behaviors performed by the user within the home and the timing of those behaviors, and stores this correspondence in the historical data. When the behavior determination unit spontaneously or periodically determines, based on the historical data, a suggestion to remind the user in the household of a possible behavior as an action of the electronic device, it executes the suggestion to remind the user of that behavior at the appropriate time when the user should perform that behavior.

29. A behavior control system, comprising: The status recognition unit identifies the user status, including user behavior, and the status of electronic devices. An emotion determination unit determines the emotion of the user or the emotion of the electronic device; The behavior determination unit uses at least one of the user state, the state of the electronic machine, the user's emotion, and the electronic machine's emotion, along with a behavior determination model, at a predetermined time to determine any one of a variety of machine actions, including not performing any action, as the behavior of the electronic machine. and The storage control unit stores event data in historical data, which includes sentiment values ​​determined by the sentiment determination unit and data including the user's behavior. The machine actions include providing process support for the users in the meeting. When the meeting is in a predetermined state, the behavior determination unit will determine the process support of the meeting output to the user in the meeting as the behavior of the electronic machine, and output the process support of the meeting.

30. A behavior control system, comprising: The testing department is responsible for testing the occurrence of events that are in accordance with regulations. and The output control unit controls the robot equipped with the article generation model to output information corresponding to the events detected by the detection unit to the user.

31. A behavior control system, comprising: The collection department collects status information indicating the user's condition; and The output control unit controls a robot equipped with an article generation model to suggest outfits to the user that correspond to the situation information collected by the collection unit.

32. A control system, comprising: The diagnostic results acquisition department acquires diagnostic results including at least one of the following: color diagnosis, skeletal diagnosis, and facial style diagnosis of the user who is conversing with the electronic machine; The user feature acquisition unit acquires user features including at least one of the user's voice volume, voice pitch, and facial expression; The user expectation acquisition department acquires user expectations that include at least one of the user's desired career and self-image. The suggestion content generation unit generates suggestion content for the user based on the diagnostic results, the user characteristics, and the user expectations; and The control unit causes the electronic device to output the proposed content to the user.

Images