Information processing system and program

The information processing system addresses the challenge of optimizing chat responses for multiple users by associating user emotion values with conversation content, ensuring positive emotional outcomes through real-time adjustment.

JP2024158646APending Publication Date: 2024-11-08SOFTBANK GROUP CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023074002
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-28
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Existing chat systems, particularly those using large language models (LLM), struggle to provide optimal responses in one-on-many communication scenarios, failing to account for individual user emotions and past interactions, leading to negative experiences for some users.

Method used

An information processing system that includes emotion recognition and data generation units to associate user emotion values with conversation content, adjusting content in real-time to ensure positive emotional responses for multiple users, leveraging neural networks and emotion engines to optimize communication.

Benefits of technology

The system effectively generates and adjusts conversation content to maintain positive emotional responses across multiple users, enhancing communication quality and user satisfaction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024158646000001_ABST
    Figure 2024158646000001_ABST
Patent Text Reader

Abstract

To provide an information processing system and a program which make an optimum reply to each of a plurality of users in one-on-n communication with one robot and the plurality of users in the same subject at the same timing.SOLUTION: In a system in which a server controls a plurality of robots via a communication network, a server 100 includes a processing section 110 consisting of: a control section for controlling a robot 200 in a manner to make it converse with users; an emotion determination section for, when the robot converses with the plurality of users on one subject, determining user emotion values indicating emotions of the users for each of the plurality of users; an association data generation section for generating association data associating the user emotion values of the plurality of users with conversation contents; and a conversation content generation section for generating the conversation contents corresponding to the one subject on the basis of the plurality of association data generated by the association data generation section.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing system and a program. [Background technology]

[0002] Patent Document 1 describes a behavior control system that determines an appropriate robot behavior for the state of a user who is a conversation partner of the robot. Patent Document 2 describes an emotion identification system that identifies the emotion of a robot. [Prior art document] [Patent documents] [Patent Document 1] JP 2016-012341 A [Patent Document 2] JP 2017-199319 A Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment of the present invention, an information processing system is provided. The information processing system may include a control unit that controls an electronic device to converse with a user. The information processing system may include an emotion determination unit that determines a user emotion value indicating the emotion of each of the multiple users when the electronic device is conversing with multiple users about a single subject. The information processing system may include a correspondence data generation unit that generates correspondence data in which the user emotion values ​​of the multiple users are associated with conversation content of the conversation. The information processing system may include a conversation content generation unit that generates conversation content corresponding to the single subject based on the multiple correspondence data generated by the correspondence data generation unit.

[0004] In the information processing system, the conversation content generation unit may generate the conversation content suitable for a plurality of users when having a conversation with the plurality of users about the one subject. The conversation content generation unit may generate the conversation content such that none of the user emotion values ​​of the plurality of users becomes a negative value when having a conversation with the plurality of users about the one subject.

[0005] Any of the information processing systems may further include a conversation content adjustment unit that automatically adjusts conversation content corresponding to the one subject, and the control unit may control the electronic device so that, when the electronic device converses with a plurality of users about the one subject, the electronic device converses with the plurality of users in accordance with the conversation content adjusted by the conversation content adjustment unit corresponding to the one subject, and the correspondence data generation unit may generate correspondence data that associates the user emotion values ​​of the plurality of users with the conversation content when the electronic device is conversing with the plurality of users in accordance with the adjusted conversation content.

[0006] In any of the information processing systems, the conversation content generation section may generate the conversation content corresponding to the one subject by giving priority to newer association data among the plurality of association data.

[0007] Any of the information processing systems may further include a user state recognition unit that recognizes user behavior for each of the multiple users who are conversing with the electronic device about the one subject, and the correspondence data generation unit may generate the correspondence data that associates the user emotion values ​​and user behavior information of the multiple users with the content of the conversation.

[0008] In any of the information processing systems, the emotion determination unit may further determine an electronic device emotion value indicating an emotion of the electronic device when the electronic device is conversing with the multiple users, and the conversation content generation unit may adjust the content of the conversation based on the user emotion values ​​of the multiple users and the electronic device emotion value when the electronic device and the multiple users are conversing.

[0009] According to one embodiment of the present invention, there is provided a program for causing a computer to function as the information processing system.

[0010] The above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions. [Brief description of the drawings]

[0011] [Figure 1] 1 shows an example of the overall configuration of a system 10 according to the present embodiment. [Diagram 2] 1 shows a schematic functional block configuration of a robot 200 and a server 100. [Diagram 3] An example of the hardware configuration of a computer 1200 that functions as a part of the server 100 or the robot 200 is shown in schematic form. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0013] FIG. 1 illustrates an example of an overall configuration of a system 10 according to the present embodiment. The system 10 includes a server 100 and a robot 200. The robot 200 may be a humanoid robot. The robot 200 may be a non-humanoid robot. The robot 200 may be an example of an electronic device. The system 10 may include a plurality of robots 200. The server 100 may function as an information processing system. The robot 200 may be an example of an object to be controlled by a control system. The system 10 may be an example of an information processing system.

[0014] The robot 200 is placed in an arbitrary location. The user 20 is a person who uses the robot 200.

[0015] The robot 200 is placed, for example, in a home. The user 20 may be a family member of the home in which the robot 200 is placed, or a stranger visiting the home. The robot 200 is placed, for example, at the reception desk of a store or office. The user 20 may be a customer visiting the store or office. The robot 200 is placed, for example, in a hospital or clinic. The user 20 may be a patient visiting a hospital or clinic. The robot 200 is placed, for example, in public transportation. The user 20 may be a user of the public transportation. The usage forms of the robot 200 are not limited to these usage forms.

[0016] The server 100 is provided remotely from the robot 200. The server 100 can control the robot 200 through a communication network 50. For example, the server 100 acquires sensor information detected by the robot 200 through the communication network 50, determines an action to be performed by the robot 200 based on the acquired sensor information, and transmits control information through the communication network 50 to instruct the robot 200.

[0017] For example, the robot 200 acquires various types of information detected by sensors, such as the voice and image of the user 20 and information on an external force received by the robot 200, and transmits the information to the server 100. The server 100 generates control information for controlling the robot 200 based on the information acquired from the robot 200. The server 100 generates control information for controlling the robot 200 based on the information acquired from the robot 200, for example, by using a neural network (NN).

[0018] In the system 10, the server 100 may execute processes for processing various information to determine the behavior of the robot 200, and processes for determining the behavior of the robot 200 based on the results of processing the various information. Note that part or all of these processes may be executed by the robot 200. In this case, part of the robot 200 functions as an information processing system.

[0019] In recent years, the use of chat systems using large language models (LLMs) has become widespread. Existing chat systems using LLMs are generated on the assumption that they will be used on personal computers (PCs) or smartphones, and when incorporating LLMs into communication devices, including humanoid robots, there is a problem in that it is not possible to provide optimal responses to each user in one-to-n communications between a robot and multiple users on the same subject at the same time. In addition, with existing chat systems using LLMs, it is difficult to respond to users based on the results of past communications.

[0020] The system 10 according to the present embodiment may include a function for performing user recognition for a plurality of users 20 who are conversation partners of the robot 200, and recording the content of the conversation for each user and transitions of a user emotion value indicating the user's emotion, a function for generating a conversation, and a function for recognizing the emotion of the user 20 and instructing a communication device to express the emotion. In Fig. 1, an example is shown in which the server 100 includes an emotion transition record DB 101 that performs user recognition and records the content of the conversation for each user and transitions of a user emotion value indicating the user's emotion, an LLM 102 that can generate a conversation, and an emotion engine 103 that can recognize the emotion of the user 20 and instruct the robot 200 to express the emotion, but the implementation form of the plurality of functions is not limited to this.

[0021] The emotion engine 103 may recognize and record transitions of emotions and actions of the multiple users 20 when the robot 200 is conversing with the multiple users 20. The emotion engine 103 may generate emotions for the robot 200 in accordance with the situation, and cause the robot 200 to communicate with the multiple users 20 using expressions according to the generated emotions.

[0022] The emotion transition record DB 101 may manage the IDs of a plurality of users 20 by face authentication, and record the conversation content and emotion transition of each user 20, and the results of communication with the robot 200.

[0023] The server 100 may use an emotion communication model to evaluate the behavioral results of the robot 200, the content of communication between the robot 200 and each user 20, and emotion transitions. When each user 20 participates in a conversation on the same subject, the server 100 may use the evaluation results to enable expressions and responses that are optimal for each user 20 or that are not inappropriate for all participants in the conversation.

[0024] In a clean slate with no past interactions or individual data, each user 20 will express themselves in a positive way, and there will be cases where not everyone will be positive, but by accumulating and learning from such history and going through a certain amount of trial and error, the content will become one that makes everyone positive. Furthermore, by recognizing users, combinations of users 20 can also be evaluated, for example, if a child becomes positive, the parents who are with them will also become positive.

[0025] The emotion transition record DB101 may record on a yearly / monthly / weekly basis, and adjust the correction level of the response depending on the time difference. The larger the time difference, the smaller the correction level, and the smaller the time difference, the larger the correction level. This allows for optimal reflection of the influence of past conversation results over time, while realizing natural communication that takes into consideration each individual user 20 in communication between multiple users 20 and the robot 200. Imagine a conversation between humans, but something that was felt positively one month ago may have faded in emotion one week ago, and may not feel positive today. The emotion transition record DB101 may be adjusted by weighting according to time so that the influence of older records is not too large.

[0026] 2 shows an example of a functional block configuration of the robot 200 and the server 100. First, the functional block configuration of the robot 200 will be described. The robot 200 has a sensor unit 210, an information processing unit 220, a control target 230, and a communication unit 240. The information processing unit 220 may be a processor such as an MPU. The communication unit 240 is responsible for communication with the server 100. The communication unit 240 may be a communication device such as a network IF.

[0027] The control target 230 includes a speaker. The control target 230 includes a display unit. The display unit may be disposed at any position of the robot 200, such as the chest of the robot 200. The control target 230 includes LEDs installed in the eyes or the like. The control target 230 includes motors and the like that drive movable parts of the robot 200, such as the limbs and head.

[0028] The sensor unit 210 has various sensors such as a microphone 211, a gyro sensor 212, a motor sensor 213, a camera 214, an infrared sensor 215, and a battery remaining amount sensor 216. It is not essential that the sensor unit 210 has all of these sensors, and the sensor unit 210 may have sensors other than these.

[0029] The microphone 211 acquires surrounding sounds. For example, the microphone 211 acquires the voice of the user 20. The gyro sensor 212 detects the angular velocity of the entire robot 200 and each part of the robot 200. The motor sensor 213 detects the rotation angle of the drive shaft of the motor that drives the movable part of the robot 200. The camera 214 captures images using visible light and generates image information of moving images and still images. The infrared sensor 215 detects surrounding objects using infrared rays. The battery remaining capacity sensor 216 detects the remaining capacity of the battery equipped in the robot 200.

[0030] The sensor unit 210 outputs various sensor data, such as voice data acquired by the microphone 211, an image captured by the camera 214, an angular velocity detected by the gyro sensor 212, a rotation angle detected by the motor sensor 213, a remaining capacity detected by the battery remaining amount sensor 216, and object information detected by the infrared sensor 215, to the information processing unit 220. The information processing unit 220 supplies the acquired sensor signals to the communication unit 240, and transmits them to the server 100. Furthermore, based on the control information acquired from the server 100, the information processing unit 220 causes the speaker of the robot 200 to speak, causes an LED of the robot 200 to emit light, and causes limbs and the like of the robot 200 to operate.

[0031] Next, the functional block configuration of the server 100 will be described. The server 100 has a processing unit 110, a communication unit 120, and a storage unit 130. The processing unit 110 has a control unit 111, a user state recognition unit 112, an emotion determination unit 113, a corresponding data generation unit 114, a conversation content generation unit 115, and a conversation content adjustment unit 116. It is not essential that the processing unit 110 has all of these. The communication unit 120 has an information acquisition unit 122.

[0032] The communication unit 120 is responsible for communication with the robot 200. The communication unit 120 may be a communication device such as a network IF. The information acquisition unit 122 acquires various information. For example, the information acquisition unit 122 acquires information detected by the sensor unit 210 of the robot 200 via the communication network 50.

[0033] The storage unit 130 has a storage medium such as a hard disk device, a flash memory, etc. The storage unit 130 also has a volatile storage device such as a RAM. The storage unit 130 stores program codes and various temporary data that the processing unit 110 reads out during execution, as well as information necessary for the processing of the processing unit 110. The storage unit 130 stores information acquired by the communication unit 120 from the robot 200.

[0034] The control unit 111 determines an action of the robot 200 using the information stored in the storage unit 130 and the information acquired by the information acquisition unit 122, and transmits control information for executing the determined action to the robot 200 via the communication unit 120. The control unit 111 controls the robot 200 to converse with the user 20, for example.

[0035] The user state recognition unit 112 recognizes the state of the user 20 based on the information acquired by the information acquisition unit 122. The user state recognition unit 112 may analyze a captured image captured by the camera 214 to recognize the face of the user 20. For example, the storage unit 130 stores face images of a plurality of people, and the user state recognition unit 112 recognizes the face of the user 20 by matching the captured image with the stored face images. The user state recognition unit 112 may analyze a captured image captured by the camera 214 to recognize the facial expression of the user 20. The user state recognition unit 112 may analyze the voice of the user 20 detected by the microphone 211 to recognize the contents of the utterance of the user 20. The user state recognition unit 112 may perform voice emotion recognition by analyzing the voice of the user 20 detected by the microphone 211. For example, the user state recognition unit 112 extracts feature amounts such as frequency components of the voice, and recognizes the emotion of the user 20 based on the extracted feature amounts.

[0036] The user state recognition unit 112 may recognize the behavior of the user 20 based on the information acquired by the information acquisition unit 122. For example, the user state recognition unit 112 stores in advance a neural network that outputs the behavior of the user 20 from the information acquired by the information acquisition unit 122, which is learned using the information acquired by the information acquisition unit 122 and the actual behavior of the user 20 at the time when the information acquisition unit 122 acquired the information as training data, and specifies the behavior of the user 20 by inputting the information acquired by the information acquisition unit 122 into the neural network. The neural network may be configured to receive the facial expression of the user 20 and the contents of the user 20's voice recognized by the user state recognition unit 112. The method of recognizing the user's behavior is not limited to this, and other known methods may be used. The user state recognition unit 112 may generate user behavior information indicating the behavior of the user 20.

[0037] The emotion determination unit 113 determines a user emotion value indicating the emotion of the user 20 conversing with the robot 200. The emotion determination unit 113 may determine the user emotion value of the user 20 using information acquired by the information acquisition unit 122. The emotion determination unit 113 may determine the user emotion value of the user 20 using a user state recognized by the user state recognition unit 112. When voice emotion recognition is executed by the user state recognition unit 112, the emotion determination unit 113 may determine the user emotion value of the user 20 using information acquired by the information acquisition unit 122 and a voice emotion recognition result.

[0038] The user emotion value may include a type of emotion of the user 20, and a value indicating the positive or negative emotion and the degree of the emotion of the user 20. Examples of the types of emotion of the user 20 include "joy", "pleasure", "comfort", "relief", "excitement", "relief", "fulfillment", "anger", "sorrow", "discomfort", "anxiety", "sadness", "concern", and "emptiness". Note that these are merely examples and are not limited to these. The value indicating the positive or negative emotion of the user 20 indicates a positive value if the emotion of the user 20 is a cheerful emotion accompanied by a pleasant feeling or comfort, such as "joy", "pleasure", "comfort", "relief", "excitement", "relief", and "fulfillment", and the brighter the emotion, the larger the value. If the emotion of the user is an unpleasant emotion, such as "anger", "sorrow", "discomfort", "anxiety", "sadness", "concern", and "emptiness", the value indicates a negative value, and the more unpleasant the emotion, the larger the absolute value of the negative value. If the emotion of the user is not any of the above ("normal"), the value of 0 is indicated.

[0039] For example, the emotion determination unit 113 pre-stores a neural network that outputs a user emotion value of the user 20 from the information acquired by the information acquisition unit 122, the neural network having been trained using the information acquired by the information acquisition unit 122 and a user emotion value indicating the actual emotion of the user 20 at the time the information acquisition unit 122 acquired the information as training data, and acquires the user emotion value by inputting the information acquired by the information acquisition unit 122 into the neural network. The neural network may be configured to accept the facial expression of the user 20 and the vocal content of the user 20 recognized by the user state recognition unit 112. The method of determining the user emotion value is not limited to this, and other known methods may be used.

[0040] The robot 200 according to the present embodiment may have an emotion. The emotion determination unit 113 may determine a robot emotion value indicating the emotion of the robot 200. The robot emotion value may be an example of an electronic device emotion value. The robot emotion value may include a type of emotion of the robot 200, and a value indicating the positive or negative emotion and the degree of the emotion of the robot 200. Examples of the types of emotions of the robot 200 include "joy", "pleasure", "comfort", "relief", "excitement", "relief", "fulfillment", "anger", "sorrow", "discomfort", "anxiety", "sorrow", "worry", and "emptiness". Note that these are examples and are not limited to these. The value indicating the positive or negative emotion of the robot 200 indicates a positive value if the emotion of the robot 200 is a bright emotion accompanied by a pleasant feeling or comfort, such as "joy", "pleasure", "comfort", "relief", "excitement", "relief", and "fulfillment", and the brighter the emotion, the larger the value. If the emotion of the robot 200 is an unpleasant emotion such as "anger", "sorrow", "discomfort", "anxiety", "sorrow", "worry" or "emptiness", it shows a negative value, and the more unpleasant the emotion, the larger the absolute value of the negative value becomes. If the emotion of the robot 200 is not any of the above ("neutral"), it shows a value of 0.

[0041] The emotion determination unit 113 may determine the robot emotion value based on the information acquired by the information acquisition unit 122 and the state of the user 20 recognized by the user state recognition unit 112. Specifically, the emotion determination unit 113 may determine the robot emotion value according to a rule for updating the robot emotion value, which is determined in association with the information acquired by the information acquisition unit 122 and the state of the user 20 recognized by the user state recognition unit 112. For example, when the user state recognition unit 112 recognizes that the user 20 looks lonely, the emotion determination unit 113 increases the emotion value of "sadness" of the robot 200. Furthermore, when the user state recognition unit 112 recognizes that the user 20 has smiled, the emotion determination unit 113 increases the emotion value of "joy" of the robot 200. Note that the emotion determination unit 113 may determine the robot emotion value by further considering the state of the robot 200. For example, the emotion determination unit 113 may increase the emotion value of "sadness" of the robot 200 when the battery level of the robot 200 is low or when the room is pitch black. For example, the emotion determination unit 113 may increase the emotion value of "anger" in the case of a user 20 who continues to talk to the robot 200 despite the battery level being low.

[0042] The storage unit 130 may store reaction rules for determining the behavior of the robot 200 based on the user emotion value of the user 20, who is the conversation partner, the behavior of the user 20, and the robot emotion value of the robot 200. In the reaction rules, for example, when the behavior of the user 20 is "laughing", the behavior of the robot 200 is determined as the behavior of the robot 200. In addition, in the reaction rules, for example, when the behavior of the user 20 is "anger", the behavior of "apologizing" is determined as the behavior of the robot 200. In addition, in the reaction rules, for example, when the behavior of the user 20 is "asking a question", the behavior of "answering" is determined as the behavior of the robot 200. In addition, in the reaction rules, for example, when the behavior of the user 20 is "sad", the behavior of "calling out" is determined as the behavior of the robot 200. The control unit 111 may determine the behavior of the robot 200 based on the reaction rules and cause the robot 200 to execute the determined behavior.

[0043] The storage unit 130 may store the contents of a conversation when the robot 200 converses with the multiple users 20 for each of the multiple subjects. The control unit 111 may cause the robot 200 to converse with the multiple users 20 according to the contents of the conversation stored in the storage unit 130. The emotion determination unit 113 may determine a user emotion value for each of the multiple users 20 when the robot 200 converses with the multiple users 20 about one subject.

[0044] When the robot 200 is conversing with multiple users 20 about one subject, the correspondence data generation unit 114 generates correspondence data in which the content of the conversation is associated with the user emotion values ​​of the multiple users 20. The correspondence data generation unit 114 stores the generated correspondence data in the storage unit 130. The content of the conversation may be composed of multiple stages, and the correspondence data generation unit 114 may associate the user emotion values ​​of the multiple users 20 at each of the multiple stages. The correspondence data generation unit 114 may associate the user emotion values ​​of the multiple users 20 at multiple timings at each stage.

[0045] The conversation content generation unit 115 generates conversation content corresponding to one subject based on the plurality of pieces of correspondence data generated by the correspondence data generation unit 114. Generating the conversation content may include generating new conversation content, or may include updating existing conversation content.

[0046] For example, when a conversation is held with the multiple users 20 about one subject, the conversation content generation unit 115 generates conversation content suitable for the multiple users 20. As a specific example, when a conversation is held with the multiple users 20 about one subject, the conversation content generation unit 115 generates conversation content such that none of the user emotion values ​​of the multiple users 20 become negative values.

[0047] The robot 200 can accumulate various correspondence data by having multiple conversations with multiple users 20 about a certain subject.

[0048] In the case of one-to-one communication between the robot 200 and a user 20, it is possible to communicate with the user 20 in an appropriate manner by adjusting the content of the conversation in accordance with the emotional transitions of the user 20. However, in the case of one-to-n communication, for example, adjusting the content of the conversation in accordance with the emotional transitions of any one of the multiple users 20 may cause the other users 20 to feel negative emotions.

[0049] In contrast, the system 10 according to the present embodiment can learn, by using various correspondence data, what kind of conversation content will make more users 20 feel less negative emotions and more positive emotions when the robot 200 communicates with multiple users 20 on a one-to-n basis. Even when talking about the same subject, the way the multiple users 20 feel will change depending on the wording and expressions used, and the conversation content generation unit 115 can continuously perform the above-mentioned generation to make the conversation content such that all of the multiple users 20 who are the conversation partners will feel positive.

[0050] The conversation content adjustment unit 116 automatically adjusts the conversation content of the robot 200 corresponding to one subject. The conversation content adjustment unit 116 may automatically adjust the conversation content every time the robot 200 has a conversation with multiple users 20 about one subject. The control unit 111 may control the robot 200 so that, when the robot 200 converses with multiple users 20 about one subject, the robot 200 converses with the multiple users 20 according to the conversation content adjusted by the conversation content adjustment unit 116 corresponding to the one subject. Then, the correspondence data generation unit 114 may generate correspondence data in which the user emotion values ​​of the multiple users 20 and the conversation content are associated when the robot 200 converses with the multiple users 20 according to the adjusted conversation content. In this way, the server 100 collects dialogue data while appropriately adjusting the conversation content, thereby collecting various variations of dialogue data. As a result, when the robot 200 converses with multiple users 20 for each of multiple subjects, it can automatically seek out conversation content suitable for the multiple users 20, and can automatically adjust the conversation content so that expressions and responses are optimal for all of the multiple users 20 or are not inappropriate for all of the multiple users 20.

[0051] The conversation content generation unit 115 may generate conversation content corresponding to each subject by prioritizing newer corresponding data among the multiple corresponding data stored in the storage unit 130. For example, the conversation content generation unit 115 applies a higher priority to newer corresponding data to generate conversation content. For example, the conversation content generation unit 115 applies a heavier weight to corresponding data with a higher priority to generate conversation content based on the corresponding data. For example, the conversation content generation unit 115 generates conversation content based only on corresponding data whose priority is higher than a predetermined threshold value.

[0052] For example, in the correspondence data from one month ago, a user emotion value with a positive emotion and a high degree of emotion is associated with the conversation content, in the correspondence data from one week ago, a user emotion value with a positive emotion and a low degree of emotion is associated with the conversation content, and in the correspondence data from one day ago, a positive emotion is not associated with the conversation content, in which case the conversation content generating unit 115 prioritizes the newer correspondence data and generates the conversation content assuming that the multiple users 20 do not have a positive emotion in the conversation content. When imagining a conversation between humans, a content that was felt positively one month ago may have become less emotional one week ago and may not be felt positive today, and such a feeling of a conversation between humans can be applied to the conversation content of the robot 200.

[0053] As a specific example, a situation will be described in which the subject is an infectious disease, and when the robot 200 and the multiple users 20 have a conversation about the infectious disease at a certain time (sometimes referred to as a first time), most of the multiple users 20 feel fear or anxiety, whereas after that (sometimes referred to as a second time), most of the multiple users 20 feel neither fear nor anxiety. In the first time, since there is a large amount of conversation data in which fear or anxiety is associated with the conversation content, the conversation content generation unit 115 adjusts the conversation content so as to reduce the fear or anxiety. In contrast, in the second time, the conversation content generation unit 115 generates the conversation content by prioritizing the corresponding data of the second time over the first time, and therefore adjusts the conversation content by prioritizing other elements rather than adjusting the conversation content so as to reduce the fear or anxiety.

[0054] If the corresponding data in the first period and the corresponding data in the second period are treated equally, the conversation content generation unit 115 will adjust the conversation content to reduce fear or anxiety even in the second period, which may result in the conversation content feeling unnatural to the user 20 who is not feeling fear or anxiety. In contrast, by preferentially treating the corresponding data in the second period, it is possible to generate conversation content that is more suited to the current situation.

[0055] When the robot 200 is having a conversation with multiple users 20 about one subject, the correspondence data generating unit 114 may associate the relationship between the multiple users 20 with the conversation content of the conversation in addition to the user emotion values ​​of the multiple users 20. Examples of the relationship between the multiple users 20 include, but are not limited to, parent-child, family, friends, and work colleagues. When the robot 200 is having a conversation with multiple users 20 about one subject, the correspondence data generating unit 114 may generate the conversation content using correspondence data in which a relationship identical to or similar to the relationship between the multiple users 20 who are the conversation partners is associated. Depending on the relationship between the multiple users 20, there may be a difference in whether the user feels positive or negative emotions toward the conversation content. For example, if a user speaks in a childish manner or imitates an anime character when talking to a group of work colleagues, there is a high possibility that the user will feel negative emotions. On the other hand, when talking with a group of parents and children, if the child speaks in a childish manner or imitates an anime character, the child's emotions are likely to become positive, and the parent's emotions are likely to become positive accordingly. The correspondence data generating unit 114 can generate conversation content that reflects such tendencies by using correspondence data in which the same or similar relationships as the relationships between multiple users 20 who are conversation partners are associated.

[0056] When the robot 200 is having a conversation with multiple users 20 about one subject, the correspondence data generating unit 114 may associate the user behavior information of the multiple users 20 with the conversation content of the conversation in addition to the user emotion values ​​of the multiple users 20. This makes it possible to generate conversation content suitable for the multiple users 20 by using the behavior of the multiple users 20 in addition to the transition of emotions of the multiple users 20 when the robot 200 has a conversation with the multiple users 20 according to the conversation content included in the correspondence data. For example, when the correspondence data indicates that many users 20 are yawning in the middle of a conversation according to the conversation content, the conversation content generating unit 115 adjusts the conversation content so that a statement that attracts the attention of the users 20 is made at a timing before many users 20 yawn.

[0057] When the robot 200 and the multiple users 20 are having a conversation, the conversation content generation unit 115 may adjust the conversation content based on the robot emotion value of the robot 200 in addition to the user emotion values ​​of the multiple users 20. For example, the storage unit 130 stores reaction rules for determining the conversation content of the robot 200 based on the emotions of the multiple users 20 and the emotion of the robot 200, and the conversation content generation unit 115 adjusts the conversation content using the reaction rules. For example, the reaction rules stipulate that when the emotion of the robot 200 is "normal" and the emotion of most of the multiple users 20 is "anger", the emotion of the robot 200 is changed to "sadness" and the expression of the conversation content is made more polite. In this way, the conversation content can be generated using the correspondence data so that the emotions of the multiple users 20 are positive overall, while the conversation content can be finely adjusted using the robot emotion value, which can contribute to realizing a conversation suitable for the multiple users 20.

[0058] The functions of the server 100 described above may be implemented by one or more computers. At least some of the functions of the server 100 may be implemented by a virtual machine. At least some of the functions of the server 100 may be implemented in the cloud. In the above embodiment, the robot 200 has been mainly given as an example of an electronic device, but this is not limited to this. Examples of electronic devices include smartphones, tablet terminals, PCs (Personal Computers), smart speakers, stuffed toys, vehicles such as automobiles and motorcycles, home appliances, etc. Note that these are merely examples, and electronic devices are not limited to these.

[0059] 3 is a schematic diagram showing an example of a hardware configuration of a computer 1200 functioning as a part of the server 100 or the robot 200. The computer 1200 may function as the server 100, for example. A program installed on the computer 1200 may cause the computer 1200 to function as one or more "parts" of the apparatus according to the present embodiment, or cause the computer 1200 to execute operations or one or more "parts" associated with the apparatus according to the present embodiment, and / or cause the computer 1200 to execute a process or steps of the process according to the present embodiment. Such a program may be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0060] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are connected to each other by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive 1226, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive 1226 may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid state drive, or the like. The computer 1200 also includes a legacy input / output unit such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0061] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0062] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive 1226 reads programs or data from a DVD-ROM 1227 or the like and provides the programs or data to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0063] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or a program that depends on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0064] The programs are provided by a computer-readable storage medium such as a DVD-ROM 1227 or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by implementing operations or processing of information according to the use of the computer 1200.

[0065] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, the DVD-ROM 1227, or a recording medium such as an IC card, and transmits the read transmission data to a network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0066] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, the DVD drive 1226 (DVD-ROM 1227), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0067] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0068] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0069] The blocks in the flowcharts and block diagrams in the present embodiment may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0070] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0071] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0072] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0073] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0074] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order. [Explanation of symbols]

[0075] 10 system, 20 user, 50 communication network, 100 server, 101 emotion transition record DB, 102 LLM, 103 emotion engine, 110 processing unit, 111 control unit, 112 user state recognition unit, 113 emotion determination unit, 114 corresponding data generation unit, 115 conversation content generation unit, 116 conversation content adjustment unit, 120 communication unit, 122 information acquisition unit, 130 memory unit, 200 robot, 210 sensor unit, 211 microphone, 212 gyro sensor, 213 motor sensor, 214 camera, 215 infrared sensor, 216 battery remaining sensor, 220 information processing unit, 230 control target, 240 communication unit, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 I / O controller, 1222 communication interface, 1224 storage device, 1226 DVD drive, 1227 DVD-ROM, 1230 ROM, 1240 I / O chip

Claims

1. A control unit that controls the electronic device in a manner similar to a conversation with a user; an emotion determining unit that determines a user emotion value indicating an emotion of each of the plurality of users when the electronic device is having a conversation with the plurality of users about one subject; a correspondence data generating unit that generates correspondence data in which the user emotion values ​​of the plurality of users are associated with conversation content of the conversation; a conversation content generation unit that generates a conversation content corresponding to the one subject based on the plurality of pieces of correspondence data generated by the correspondence data generation unit; An information processing system comprising:

2. The information processing system according to claim 1 , wherein the conversation content generation unit generates the conversation content suitable for a plurality of users when a conversation is held with the plurality of users about the one subject.

3. The information processing system according to claim 2 , wherein the conversation content generation unit generates the conversation content in which, when a conversation is held with a plurality of users about the one subject, none of the user emotion values ​​of the plurality of users becomes a negative value.

4. A conversation content adjustment unit that automatically adjusts conversation content corresponding to the one subject. Further equipped with the control unit controls the electronic device so that, when the electronic device converses with a plurality of users about the one subject, the electronic device converses with the plurality of users according to the conversation content adjusted by the conversation content adjustment unit corresponding to the one subject; 4. The information processing system according to claim 1, wherein the correspondence data generation unit generates correspondence data that associates the user emotion values ​​of the multiple users with the conversation content when the electronic device is conversing with the multiple users in accordance with the adjusted conversation content.

5. The information processing system according to claim 1 , wherein the conversation content generation unit generates the conversation content corresponding to the one subject by giving priority to newer correspondence data among the plurality of correspondence data.

6. a user state recognition unit that recognizes the behavior of each of the plurality of users who are having a conversation with the electronic device about the one subject; Further equipped with The information processing system according to claim 1 , wherein the corresponding data generation unit generates the corresponding data in which the user emotion values ​​of the plurality of users and user behavior information indicating the behavior of the users are associated with the conversation content of the conversation.

7. the emotion determination unit further determines an electronic device emotion value indicating an emotion of the electronic device when the electronic device is conversing with the plurality of users; 4. The information processing system according to claim 1, wherein the conversation content generation unit adjusts the conversation content of the conversation based on the user emotion values ​​of the multiple users and the electronic device emotion value when the electronic device and the multiple users are conversing.

8. The information processing system according to claim 1 , further comprising the electronic device.

9. A program for causing a computer to function as the information processing system according to any one of claims 1 to 3.