Control system and program

Abstract

To output the stored information according to the emotion associated with the relevant information.SOLUTION: A control system according to an embodiment controls actions of a robot, and comprises an emotion determination unit, a storage unit, and an output control unit. The emotion determination unit determines emotion of either or both of a user and a robot on the basis of at least a part of information detected by a sensor continuously detecting the user. The storage unit stores, in association with the determined emotion of either or both of the determined user and the robot, information obtained by processing at least the part of the information detected by the user. When the information stored in the storage unit is output by the robot, the output control unit perform control of changing either or both of the actions of the robot and output timing of the stored information according to the emotion associated with the stored information.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a control system and a program. [Background technology]

[0002] A terminal is known that learns a conversation between a user and a call partner and stores the call partner's replies to questions from the user in a reply table (see, for example, Patent Document 1). Also, an emotion generation device is known that includes a neural network that inputs user information, device information, and its own current emotional state and outputs the next emotional state (see, for example, Patent Document 2). Also, a technology is known that stores a spatiotemporal pattern in an associative memory that includes multiple electronic neurons having a layer-neural network relationship with directional artificial synaptic connectivity (see, for example, Patent Document 3). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-253389 A [Patent Document 2] Japanese Patent Application Publication No. 10-254592 [Patent Document 3] Special Publication No. 2013-535067 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology leaves room for improvement in terms of outputting stored information.

[0005] The present invention has been made in consideration of the above, and has an object to provide a control system and a program that can output stored information in accordance with an emotion associated with the information. [Means for solving the problem]

[0006] A control system according to one aspect of the embodiment is a control system for controlling the behavior of a robot, and includes an emotion determination unit, a memory unit, and an output control unit. The emotion determination unit determines the emotion of one or both of the user and the robot based on at least a portion of information detected by a sensor that continuously detects the user. The memory unit stores information obtained by processing at least a portion of the information detected by the sensor, in association with the determined emotion of one or both of the user and the robot. The output control unit, when outputting the information stored in the memory unit by the robot, performs control to change one or both of the robot's behavior and the output timing of the information in accordance with the emotion associated with the information. Effect of the Invention

[0007] According to one aspect of the embodiment, the stored information may be output according to an emotion associated with the information. [Brief description of the drawings]

[0008] [Figure 1A] FIG. 1A is a schematic diagram illustrating an example of a system 20 according to the present embodiment. [Figure 1B] FIG. 1B is a diagram showing an example of emotion value changes. [Diagram 2] FIG. 2 is a diagram showing a schematic block configuration of the server 200 and the robot 40. As shown in FIG. [Diagram 3] FIG. 3 is a schematic diagram of a neural network 300. [Figure 4] FIG. 4 is a diagram showing a schematic diagram of the parameters of the neural network in a table format. [Diagram 5] FIG. 5 is a diagram illustrating an outline of the operation flow of the server 200 when the robot 40 is started or reset. [Figure 6] FIG. 6 is a diagram for explaining the outline of the calculation of the connection weights of the artificial synapses. [Figure 7] FIG. 7 is a diagram showing an outline of the time evolution of the coupling coefficient when a function ht ij is defined as an increase / decrease parameter of the coupling coefficient. [Figure 8] FIG. 8 is a diagram showing a schematic diagram of the time evolution of the coupling coefficient when further simultaneous firing occurs at time t2. [Figure 9] FIG. 9 is a diagram illustrating an outline of effect definition information that defines a chemical effect applied to a parameter. [Figure 10] FIG. 10 is a flowchart showing the calculation of the internal state and status. [Figure 11] FIG. 11 is a diagram for explaining an example of calculation of the internal state when the artificial neuron does not fire. [Figure 12] FIG. 12 is a diagram for explaining an example of calculation of an output when an artificial neuron fires. [Figure 13] FIG. 13 is a diagram showing a schematic diagram of the time evolution of the coupling coefficient when a function is defined as an increase / decrease parameter of the artificial neuron. [Figure 14] FIG. 14 is a diagram showing an example of rules 1400 stored in the recording format switching rule 290 in table form. [Figure 15] FIG. 15 is a flowchart showing the operation control of the robot 40. [Figure 16] FIG. 16 is a flowchart showing control of the data storage period. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0010] Fig. 1A shows an example of a system 20 according to the present embodiment. Fig. 1B shows an example of an emotional value change. The system 20 includes a server 200, and a robot 40a and a robot 40b. The robots 40a and 40b communicate with the server 200 via a communication network 90 to exchange information.

[0011] The user 30a is a user of the robot 40a. The user 30b is a user of the robot 40b. The robot 40b has substantially the same functions as the robot 40a. Therefore, the system 20 will be described with the robots 40a and 40b collectively referred to as the robot 40.

[0012] The robot 40 performs various actions according to the situation, such as moving its head or limbs, conversing with the user 30, and providing an image to the user 30. At this time, the robot 40 decides the action in cooperation with the server 200. For example, the robot 40 transmits to the server 200 detection information such as a facial image of the user 30 acquired by a camera function and a voice of the user 30 acquired by a microphone function. The server 200 analyzes the detection information received from the robot 40, decides the action to be taken by the robot 40, and transmits action information representing the decided action to the robot 40. The robot 40 operates according to the action information received from the server 200.

[0013] The robot 40 has an emotion value that represents its own emotion. For example, the robot 40 has an emotion value that represents the strength of each emotion, such as "happy," "fun," "sad," "fear," and "excitement." The emotion value of the robot 40 is determined by the server 200. The server 200 causes the robot 40 to perform an action according to the determined emotion. For example, when the robot 40 has a high emotion value of excitement and is talking to the user 30, the server 200 causes the robot 40 to speak quickly. In this way, the robot 40 can express its own emotion through actions, etc.

[0014] The server 200 updates the current state of the robot 40 using a neural network based on the detection information received from the robot 40. The state of the robot 40 includes the emotion of the robot 40. Therefore, the server 200 determines the emotion of the robot 40 using the neural network.

[0015] The robot 40 also acquires various information related to the user 30. For example, the robot 40 uses a camera function to take a picture of the user 30 and acquires an image of the user 30. The robot 40 also acquires various information related to the surroundings by a sensor provided in the robot 40. For example, the robot 40 acquires surrounding information such as information on objects present in the surroundings, as well as odors, temperature, humidity, position information, weather, the number of people in the surroundings, and their facial expressions. The robot 40 also detects an event from various information related to the user 30 and the surroundings. For example, the robot 40 recognizes the presence of the user 30 from the voice and image of the user 30, recognizes that the user 30 is "Mr. A", and generates event information such as "Mr. A is here". The robot 40 also generates event information such as "I was petted". The robot 40 transmits the acquired various information related to the user 30 and the surroundings to the server 200. When an event is detected, the robot 40 transmits event information related to the detected event to the server 200. The event may be detected in the server 200.

[0016] The server 200 stores various pieces of information related to the user 30 and the surroundings received from the robot 40 in association with the emotions of the robot 40. For example, the server 200 stores various pieces of information related to the user 30 and the surroundings by tagging them with information on the emotion value of the robot 40. This allows the server 200 to record the emotions at the time the information was acquired in association with various pieces of information related to the user 30 and the surroundings.

[0017] Furthermore, when an event is detected from various information related to the user 30 and the surroundings, the server 200 associates the degree of emotion of the robot 40 with the event and stores the associated information. The server 200 stores the event information and the emotion value information of the robot 40 with tags. This allows the server 200 to record the emotion at the time the information was acquired in association with the event.

[0018] When outputting stored information from the robot 40, the server 200 performs control to change one or both of the behavior of the robot 40 and the output timing of the information according to the emotion associated with the information. This allows the user 30 to feel the emotion associated with the information from the behavior and output timing of the robot 40 when outputting the information. In this way, the server 200 can cause the robot 40 to output the stored information according to the emotion associated with the information.

[0019] In addition, the server 200 controls the storage period of the stored information according to the emotion associated with the information. For example, the server 200 controls so that information associated with a strong emotion is stored for a long period of time, and information associated with a weak emotion is stored for a short period of time. This allows information that leaves an impression on the user 30 to be stored for a long period of time, and information that leaves a poor impression on the user 30 is quickly deleted. In this way, the server 200 can appropriately store information according to the associated emotion.

[0020] The robot 40 also records the video data of the user 30 acquired by the camera function in the server 200. The robot 40 acquires the video data from the server 200 as necessary and provides it to the user 30. The robot 40 generates video data with a larger amount of information as the strength of its own emotion increases and records it in the server 200. For example, when the robot 40 records information in a highly compressed format such as skeletal data, the robot 40 switches to recording information in a low compression format such as HD video when the emotion value of the robot 40 exceeds a threshold. According to the system 20, high-definition video data when the emotion of the robot 40 increases can be recorded. The robot 40 also generates video data with a larger amount of information as the strength of the emotion of the user 30 increases and records it in the server 200. Specifically, when the robot 40 records information in a highly compressed format such as skeletal data, the robot 40 switches to recording information in a low compression format such as HD video when the emotion value of the user 30 exceeds a threshold. According to the system 20, high-definition video data of a moment when the user 30 is emotionally excited can be left as a record.

[0021] In addition, even if the robot 40's or the user's emotion is not strong as in state 2 of FIG. 1B, when the robot 40's or the user's emotion changes suddenly, the robot 40 generates video data with a large amount of information and records it in the server 200. For example, when the robot 40 records information in a highly compressed format such as skeletal data, the robot 40 switches to recording information in a low compression format such as HD video in response to the change in the emotion value exceeding the threshold even if the emotion value of the excitement of the robot 40 or the user is equal to or less than the threshold. According to the system 20, high-definition video data when the emotion of the robot 40 or the user 30 changes drastically can be recorded. In addition, the robot 40 may change the amount of information of the video data according to the degree of emotion change. Specifically, the robot 40 lowers the compression rate of the video data as the degree of change in the emotion of the robot 40 or the user increases. As another example, the robot 40 may change the format of the information to be recorded in the order of decreasing degree of change in the emotion of the robot 40 or the user, such as text only → image only → voice and image → low-quality video → high-quality video. Furthermore, the robot 40 may change the length or amount of video data, not just the compression rate or the format of information. This allows the robot 40 or the user 30 to record higher-definition video data when the emotions of the robot 40 or the user 30 change more dramatically.

[0022] Also, the robot 40 may change the information to be stored according to the continuity of the change in emotion of the robot 40 or the user 30, as in state 1 of FIG. 1B. For example, when the cumulative value of the emotion value within a predetermined time exceeds a threshold, the robot 40 generates video data with a large amount of information and records it in the server 200. Also, the robot 40 may change the information to be stored according to the combination of the speed of the change (slope) of emotion within a predetermined time or the degree of the slope. For example, the robot 40 generates video data with a large amount of information and records it in the server 200 when the slope indicating the change in the emotion value is a combination of a sudden rise → a gradual rise for a certain period of time → a sudden fall within a predetermined time. Thereby, for example, in a tense situation due to pressure or the like, video data of a situation in which the absolute value of the emotion is not relatively large but the time feels long (an emotion value above a certain value continues for a certain period of time) can be recorded. Change

[0023] In addition, the robot 40 tags the information to be recorded when the emotional value of itself or the robot 40 exceeds a threshold value or when a change in the emotional value exceeds a threshold value. Specifically, the robot 40 tags the video data with information detected by a sensor provided in the robot 40 and information on the emotional value of the user or itself. The tagged information includes, for example, surrounding information (information on objects present in the vicinity, etc.), smell, temperature, humidity, location information, weather, the number of people in the vicinity, their expressions and emotions, etc. This allows the user 30 to clearly recall the situation at the time from the tagged information when checking the recorded information. Furthermore, by tagging, the information can be easily found when searching for information on a specific situation from the recorded information.

[0024] FIG. 2 shows a schematic block configuration of the server 200 and the robot 40. The robot 40 has a sensor unit 156, a processing unit 152, a control target 155, a communication unit 158, and a display unit 157. The server 200 has a processing unit 202, a storage unit 280, and a communication unit 208. The processing unit 202 includes an initial value setting unit 210, an external input data generating unit 230, a parameter processing unit 240, an action determining unit 250, a switching control unit 260, and a recording control unit 270. The storage unit 280 stores an action determining rule 282, definition information 284, a parameter initial value 286, the latest parameter 288, a recording format switching rule 290, and recorded data 292. The storage unit 280 functions as a memory unit that stores information in association with an emotion.

[0025] In the robot 40, the sensor unit 156 has sensors such as a microphone 161, a 2D camera 163, a 3D depth sensor 162, and a distance sensor 164. Each sensor of the sensor unit 156 continuously detects information. The sensor information detected by the sensor unit 156 is output to the processing unit 152. The 2D camera 163 is an example of an image sensor that continuously captures an object, and captures an object with visible light to generate video information. The 3D depth sensor 162 detects the contour of the object by continuously irradiating an infrared pattern and analyzing the infrared pattern from the infrared images continuously captured by the infrared camera. In addition, the sensor unit 156 may include various other sensors such as a clock, a gyro sensor, a touch sensor, a sensor for motor feedback, a sensor for detecting remaining battery capacity, an odor sensor, a temperature sensor, a humidity sensor, a GPS (Global Positioning System) sensor, etc.

[0026] The processing unit 152 is formed of a processor such as a CPU. The processing unit 152 transmits sensor information continuously detected by each sensor of the sensor unit 156 to the server 200 through the communication unit 158. For example, the processing unit 152 transmits various information related to the user 30 detected by each sensor of the sensor unit 156 to the server 200 through the communication unit 158. For example, the processing unit 152 transmits an image of the user 30 captured by the 2D camera 163 to the server 200 through the communication unit 158. In addition, the processing unit 152 transmits surrounding information such as information on objects present in the surroundings detected by each sensor of the sensor unit 156, odors, temperature, humidity, position information, weather, the number of people in the surroundings, and their expressions to the server 200 through the communication unit 158. In addition, the processing unit 152 processes at least a part of the sensor information continuously detected by each sensor of the sensor unit 156 to generate information for recording. The processing unit 152 generates information in the first recording format or information in the second recording format having a larger amount of information than the information in the first recording format. The information in the first recording format is, for example, information in a high compression format, and the information in the second recording format is, for example, information in a low compression format. For example, the processing unit 152 generates shape data such as skeletal data of an object as information in the first recording format based on skeletal information continuously detected by the 3D depth sensor 162. The processing unit 152 also generates full HD video data and audio data from video information captured by the 2D camera 163 and audio information detected by the microphone 161. Full HD video data is an example of video data having more information than the shape data of an object. The processing unit 152 detects an event from various information related to the user 30 and the surroundings detected by each sensor of the sensor unit 156. When an event is detected, the processing unit 152 causes the server 200 to transmit event information related to the detected event through the communication unit 158. The event may be detected by the processing unit 202 of the server 200.

[0027] The communication unit 158 ​​transmits various information related to the user 30 detected by each sensor of the sensor unit 156 to the server 200. For example, the communication unit 158 ​​transmits an image of the user 30 captured by the 2D camera 163 to the server 200. The communication unit 158 ​​also transmits to the server 200 surrounding information such as information on objects present in the surroundings detected by each sensor of the sensor unit 156, as well as odors, temperature, humidity, position information, weather, the number of people in the surroundings, and their expressions. The communication unit 158 ​​also transmits to the server 200 information in the first recording format or information in the second recording format generated by the processing unit 152. In addition, when an event is detected, the communication unit 158 ​​transmits event information related to the detected event to the server 200.

[0028] Furthermore, in the robot 40, the communication unit 158 ​​acquires information stored in the recording data 292 from the server 200. The communication unit 158 ​​functions as a recorded information receiving unit that acquires information in the second recording format, including video data, recorded by the recording control unit 270. The processing unit 152 generates an image to be presented to the user 30, based on the video data included in the information in the second recording format received by the communication unit 158. The processing unit 152 functions as an image generating unit that generates an image to be presented to the user 30.

[0029] Furthermore, the communication unit 158 ​​receives operation information indicating the operation content from the server 200. The processing unit 152 controls the control target 155 based on the operation content received by the communication unit 158. The control target 155 includes a speaker, motors for driving each part of the robot 40 such as the limbs, a light emitting device, and the like. When information indicating the vocalization content is received from the server 200, the processing unit 152 outputs a sound from the speaker according to the received vocalization content. Furthermore, the processing unit 152 can control a part of the behavior of the robot 40 by controlling the drive motors of the limbs. Furthermore, the processing unit 152 can express a part of the emotion of the robot 40 by controlling these motors.

[0030] In the server 200 , the communication unit 208 outputs the information received from the robot 40 to the processing unit 202 .

[0031] In the processing unit 202, the initial value setting unit 210 stores the initial values ​​of the parameters indicating the initial state of the neural network received by the communication unit 208 in the parameter initial value 286 in the storage unit 280. Note that the initial values ​​of the parameters of the neural network may be predetermined in the server 200, or may be changeable by the user 30 via the communication network 90.

[0032] The external input data generating unit 230 processes at least a part of the sensor information received by the communication unit 208 to generate input information from outside the neural network, and outputs the generated information to the parameter processing unit 240. The parameter processing unit 240 performs calculations on the neural network based on the input information and current parameters 288 and definition information 284 of the neural network stored in the storage unit 280.

[0033] The artificial neurons of the neural network include a plurality of artificial neurons in which the situation of the robot 40 is defined, a plurality of emotion artificial neurons in which a plurality of emotions of the robot 40 itself is defined, and a plurality of endocrine artificial neurons in which the production state of the endocrine substance of the robot 40 itself is defined. The parameter processing unit 240 calculates parameters representing the internal states of the plurality of artificial neurons in the neural network based on the input information generated by the external input data generating unit 230. For example, the parameter processing unit 240 updates the parameters of the current internal states of the plurality of artificial neurons in which the situation of the robot 40 is defined, based on the input information generated by the external input data generating unit 230. In addition, the parameter processing unit 240 calculates parameters of the internal states of other artificial neurons in the neural network. As a result, for example, the parameters of the internal state of the emotion artificial neuron in which the emotion "happy" is defined are calculated. The parameters of the internal state of this emotion artificial neuron are an example of an index representing the strength of the emotion "happy". Therefore, the parameter processing unit 240 can determine the strength of the emotion in the control system based on the internal state of the emotion artificial neuron. In this way, the parameter processing unit 240 functions as an emotion determination unit that determines an emotion using a neural network based on at least a portion of the information detected by the sensor of the sensor unit 156. For example, the parameter processing unit 240 determines the type and strength of an emotion based on at least a portion of the information detected by the sensor of the sensor unit 156.

[0034] Furthermore, the parameter processing unit 240 determines an emotion based on information newly detected by a sensor included in the sensor unit 156 provided in the robot 40 and an emotion previously determined by the parameter processing unit 240 associated with information previously detected by a sensor included in the sensor unit 156 provided in the robot 40. For example, the parameter processing unit 240 can determine a new emotion similar to the emotion "happy" previously determined for the information "person A" newly detected by the sensor included in the sensor unit 156 provided in the robot 40, based on the information "person A" "patted on the head" previously detected by the sensor included in the sensor unit 156 provided in the robot 40 and the emotion "happy" previously determined by the parameter processing unit 240.

[0035] That is, when the newly detected information includes a specific element (for example, "Person A"), the parameter processing unit 240 determines the emotion for the newly detected information based on the emotion associated with the specific element. That is, the parameter processing unit 240 suppresses to some extent the strong appearance of an emotion completely opposite to an emotion that occurred in the past in a situation that has something in common with a situation experienced in the past. This makes it possible to generate a variety of emotions according to the information of people and events learned in the past and the current situation.

[0036] Further, the parameter processing unit 240 determines the emotion using the emotion information updated by the recording control unit 270. For example, the parameter processing unit 240 determines the emotion based on the information newly detected by the sensor of the sensor unit 156 provided in the robot 40 and the emotion updated by the recording control unit 270 associated with the information previously detected by the sensor of the sensor unit 156 provided in the robot 40. More specifically, the parameter processing unit 240 can determine a new emotion similar to the emotion "fear" updated by the recording control unit 270 associated with the information "Person A" "patted" newly detected by the sensor of the sensor unit 156 provided in the robot 40 and the element "Person A" included in the information "Person A" "hit" previously detected by the sensor of the sensor unit 156 provided in the robot 40. In other words, the parameter processing unit 240 suppresses to some extent the strong expression of an emotion completely opposite to the updated emotion in a situation that has something in common with a situation previously experienced, taking into account the change in the relationship. This makes it possible to generate a variety of emotions based on previously learned people and event situations, the current situation, and changes in the relationship with the subject.

[0037] The neural network parameters calculated by the parameter processing unit 240 are supplied to the switching control unit 260 , the recording control unit 270 and the operation determination unit 250 .

[0038] The recording control unit 270 associates the emotion of the robot 40 with the information detected by the sensors of the sensor unit 156, and stores the information obtained by processing at least a part of the information in the storage unit 280. For example, the recording control unit 270 associates the emotion of the robot 40 with various information received by the communication unit 208 and stores the information in the recorded data 292. For example, the recording control unit 270 associates the emotion of the robot 40 with various information related to the user 30, peripheral information, and event information, and stores the information in the recorded data 292. For example, the recording control unit 270 associates the emotion of the robot 40 with an emotion value of the robot 40 as the emotion of the robot 40 and stores the information in the recorded data 292.

[0039] For example, the recording control unit 270 associates information detected by a sensor included in the sensor unit 156 included in the robot 40 with an emotion determined by the parameter processing unit 240, and stores the associated information in the storage unit 280. For example, the recording control unit 270 associates information "Person A" and "patted on the head" detected by a sensor included in the sensor unit 156 included in the robot 40 with an emotion "happy" determined by the parameter processing unit 240, and stores the associated information in the recorded data 292. That is, the recording control unit 270 associates information on people, events, etc. with information on emotions, and stores the associated information.

[0040] Furthermore, the recording control unit 270 stores the information in the first recording format or the information in the second recording format received by the communication unit 208 from the robot 40 in the recording data 292. The recording control unit 270 stores the information received from each robot 40 in the recording data 292 in association with information identifying each robot 40.

[0041] In addition, based on the emotion newly determined by the parameter processing unit 240, the recording control unit 270 updates the emotion previously determined by the parameter processing unit 240, which is associated with information previously detected by a sensor provided in the robot 40 and which includes elements common to the information newly detected by the sensor provided in the robot 40, to the emotion newly determined by the parameter processing unit 240.

[0042] For example, when the strength of an emotion newly determined by the parameter processing unit 240 exceeds a predetermined threshold, the recording control unit 270 updates the emotion previously determined by the parameter processing unit 240, which is associated with information previously detected by a sensor provided in the robot 40 and which includes elements common to information newly detected by a sensor provided in the robot 40, to the emotion newly determined by the parameter processing unit 240.

[0043] More specifically, when the emotion "very happy" newly determined by the parameter processing unit 240 exceeds a predetermined threshold, the recording control unit 270 updates the emotion "happy" previously determined by the parameter processing unit 240, associated with the element "person A" included in the information "person A" "patted" previously detected by the sensor of the sensor unit 156 provided in the robot 40, including the element "person A" common to the information "person A" "hugged" newly detected by the sensor provided in the robot 40, to the emotion "very happy" newly determined by the parameter processing unit 240. That is, in the above example, the recording control unit 270 updates the emotion associated with "person A" from "happy" to "very happy".

[0044] In addition to a fixed value, the predetermined threshold value can be any value or rate of change according to the purpose, such as any value or rate of change for the strength of the current emotion, any value or rate of change for the type of the current emotion, etc. This makes it possible to generate a variety of emotions by considering changes in the relationship with the target and using the determined intensity of emotion according to the situation of people and events learned in the past and the current situation.

[0045] In the above example, when the recording control unit 270 updates the emotion associated with a specific element included in the detected information, the recording control unit 270 may update the emotion associated with other elements detected together with the common element. More specifically, when the emotion "very fun" newly determined by the parameter processing unit 240 exceeds a predetermined threshold, the recording control unit 270 updates the emotion "boredom" previously determined by the parameter processing unit 240, which is associated with the element "amusement park" included in the information "person A" and "amusement park" previously detected by the sensor of the sensor unit 156 provided in the robot 40, including the element "person A" common to the information "person A", "hugged", and "park" newly detected by the sensor provided in the robot 40, to the emotion "very fun" newly determined by the parameter processing unit 240. That is, in the above example, the recording control unit 270 updates the emotion associated with the "amusement park" related to "person A" from "boredom" to "very fun" using the common element "person A" as a key. That is, when updating the emotion associated with the common element "Person A" from "bored" to "very fun", the recording control unit 270 can also update the emotion associated with the element "Amusement Park" previously detected together with the common element "Person A" from "bored" to "very fun". In this way, for example, when the emotion toward a person changes from negative to positive, the emotion toward an event in which the person was involved also changes from negative to positive. This makes it possible to propagate the change in the relationship with the target to other targets in accordance with the situation of the person or event learned in the past and the current situation, and to generate a variety of emotions in consideration of the changed relationship.

[0046] Furthermore, for example, when the type of emotion determined by the parameter processing unit 240 undergoes a predetermined change, the recording control unit 270 updates the emotion previously determined by the parameter processing unit 240, which is associated with the information previously detected by the sensor of the sensor unit 156 included in the robot 40 and which includes an element common to the information newly detected by the sensor of the sensor unit 156 included in the robot 40, to the emotion newly determined by the parameter processing unit 240. More specifically, when the emotion "fear" newly determined by the parameter processing unit 240 is a predetermined change, the recording control unit 270 updates the emotion "happy" previously determined by the parameter processing unit 240, which is associated with the element "person A" included in the information "person A" and "patted" previously detected by the sensor of the sensor unit 156 included in the robot 40 and which includes the element "person A" common to the information "person A" and "hit" newly detected by the sensor of the sensor unit 156 included in the robot 40, to the emotion "fear" newly determined by the parameter processing unit 240.

[0047] That is, in the above example, the recording control unit 270 updates the emotion associated with "Person A" from "Happy" to "Scary." Note that the predetermined change can be any change according to the purpose, in addition to the change to the opposing emotion given in the above example. This makes it possible to generate a variety of emotions by considering the change in the relationship with the target and using the determined emotion change according to the situation of the person or event learned in the past and the current situation.

[0048] Furthermore, for example, the recording control unit 270 can perform an update process by combining the strength and type of emotion determined by the parameter processing unit 240. For example, when the type of emotion determined by the parameter processing unit 240 undergoes a predetermined change and the strength of the emotion exceeds a predetermined threshold, the recording control unit 270 updates the emotion previously determined by the parameter processing unit 240, which is associated with information previously detected by the sensor of the sensor unit 156 provided in the robot 40 and includes elements common to information newly detected by the sensor of the sensor unit 156 provided in the robot 40, to the emotion newly determined by the parameter processing unit 240. More specifically, when the emotion "very scary" newly determined by the parameter processing unit 240 is a predetermined change and exceeds a predetermined threshold, the recording control unit 270 updates the emotion "happy" previously determined by the parameter processing unit 240, which is associated with the element "person A" included in the information "person A" "patted" previously detected by the sensor of the sensor unit 156 provided in the robot 40, including the element "person A" common to the information "person A" "hit" newly detected by the sensor of the sensor unit 156 provided in the robot 40, to the emotion "very scary" newly determined by the parameter processing unit 240. That is, in the above example, the recording control unit 270 updates the emotion associated with "person A" from "happy" to "very scary". This makes it possible to generate a variety of emotions by considering the change in the relationship with the target using the change and strength of the determined emotion in accordance with the situation of the person or event learned in the past and the current situation.

[0049] The target of the update by the recording control unit 270 is an emotion associated with information including elements common to information newly detected by a sensor of the sensor unit 156 provided in the robot 40. That is, an emotion associated with information having at least one common element among information such as surrounding information, location information, date and time, weather, temperature, scenery, smell, sound, number of people in the vicinity, people, facial expressions, behavior of people, and other situations may be the target of the update by the recording control unit 270. This makes it possible to identify commonalities between the situations of people and events learned in the past and the current situation, and generate various emotions by using the determined emotion and taking into account changes in the relationship with the target.

[0050] The switching control unit 260 determines the recording format of the information generated by the processing unit 152 of the robot 40 based on the parameters supplied from the parameter processing unit 240. When it is necessary to switch the recording format of the information generated by the processing unit 152, the switching control unit 260 causes the robot 40 to transmit an instruction to switch the recording format via the communication unit 208. In the robot 40, the processing unit 152 switches the recording format according to the instruction received from the server 200.

[0051] For example, when the switching control unit 260 is causing the processing unit 152 to generate information in the first recording format, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format in response to an increase in the intensity of the emotion determined by the parameter processing unit 240. In this way, the switching control unit 260 switches the recording format of the information recorded by the recording control unit 270 from the first recording format to the second recording format in response to an increase in the intensity of the emotion determined by the parameter processing unit 240. This allows the information when the emotion of the robot 40 increases to be recorded in detail. In addition, in the robot 40, the processing unit 152 acquires the video data in the second recording format acquired from the server 200 and generates a video to be presented to the user 30. Therefore, the user 30 can enjoy the information when the emotion of the robot 40 increases as a video.

[0052] In addition, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format in response to an increase in the intensity of the emotion of the user 30. In this way, the switching control unit 260 switches the recording format of the information to be recorded by the recording control unit 270 from the first recording format to the second recording format in response to an increase in the intensity of the emotion determined by the parameter processing unit 240. This allows information when the emotion of the user 30 is heightened to be recorded in detail. In addition, in the robot 40, the processing unit 152 acquires video data in the second recording format acquired from the server 200 and generates a video to be presented to the user 30. Therefore, the user 30 can enjoy information when his / her emotion is heightened as a video. The emotion of the user 30 can be detected, for example, based on a voice or facial expression detected from the user 30. Furthermore, the switching control unit 260 may recognize a change in the emotion of the user 30 based on the detection of an impact by a touch sensor (not shown) included in the sensor unit 156. For example, when an impact is detected by a touch sensor included in the sensor unit 156, the switching control unit 260 may recognize that the emotion of the user 30 has worsened.

[0053] When the switching control unit 260 is causing the processing unit 152 to generate information in the second recording format, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the second recording format to the first recording format in response to a decrease in the intensity of the emotion of the robot 40 or the emotion of the user determined by the parameter processing unit 240. In this way, the switching control unit 260 switches the recording format of the information recorded by the recording control unit 270 from the second recording format to the first recording format in response to a decrease in the intensity of the emotion of the robot 40 or the emotion of the user determined by the parameter processing unit 240.

[0054] Furthermore, in response to an increase in the emotional change of the user 30 or the robot 40, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information generated by the processing unit 152 from the first recording format to the second recording format. This allows detailed information about when the emotion of the user 30 or the robot 40 increases to be recorded. Furthermore, in the robot 40, the processing unit 152 acquires the video data in the second recording format acquired from the server 200, and generates a video to be presented to the user 30. Thus, the user 30 can enjoy information about when the emotion of the user 30 or the robot 40 increases as a video.

[0055] When the switching control unit 260 is causing the processing unit 152 to generate information in the second recording format, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the second recording format to the first recording format in response to the change in the emotion of the robot 40 or the emotion of the user subsiding (the amount of change becomes equal to or less than the threshold) or the emotion of the robot 40 or the emotion of the user returning to the emotion before the change. In this way, the switching control unit 260 switches the recording format of the information recorded by the recording control unit 270 from the second recording format to the first recording format in response to the change in the emotion of the robot 40 or the emotion of the user subsiding.

[0056] The switching control unit 260 may also switch the recording format according to the continuity of changes in the emotions of the robot 40 or the user 30. For example, when the cumulative value of the emotion value within a predetermined time exceeds a threshold, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format. The switching control unit 260 may also switch the recording format according to a combination of the speed of the change (slope) of the emotion within a predetermined time or the degree of the slope. For example, when the slope indicating the change in the emotion value is a combination of a sudden rise → a gradual change for a certain period of time → a sudden fall within a predetermined time, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format. This makes it possible to record video data of a situation in which, for example, in a tense scene due to pressure or the like, the absolute value of the emotion is not relatively large but the time feels long (an emotion value above a certain value continues for a predetermined time).

[0057] Furthermore, when the recording control unit 270 switches the recording format of the information generated by the processing unit 152 from the first recording format to the second recording format, it transmits an instruction to the robot 40 to tag the information in the second recording format. This allows the user 30 to clearly recall the situation at the time from the tagged information when checking the recorded information. Furthermore, by tagging, the information can be easily found when searching for information on a specific situation from among the recorded information.

[0058] However, people may remember information for different periods of time depending on the emotions they felt when they memorized the information. For example, people tend to remember information associated with strong emotions for a long time, and information associated with weak emotions for a long time.

[0059] Therefore, the recording control unit 270 controls the storage period of the information stored in the recording data 292 of the storage unit 280 according to the emotion associated with the information. For example, the recording control unit 270 controls so as to store information associated with a strong emotion for a long period of time, and store information associated with a weak emotion for a short period of time. For example, the recording control unit 270 determines whether the emotion value associated with the information is equal to or greater than a predetermined threshold value that is regarded as a strong emotion, and determines the emotion as strong if the emotion value is equal to or greater than the threshold value, and determines the emotion as strong if the emotion value is less than the threshold value. Note that the recording control unit 270 may set the threshold value in multiple stages and determine the emotion in multiple stages such as strong, normal, and weak.

[0060] The recording control unit 270 sets the storage period according to the strength of the emotion. For example, the recording control unit 270 sets a long storage period for storing information associated with a strong emotion, and sets a short storage period for information associated with a weak emotion. The recording control unit 270 periodically determines whether the storage period has elapsed from the date and time when the information was stored in the recording data 292 for each piece of information stored in the recording data 292. The recording control unit 270 deletes information whose storage period has elapsed from the recording data 292. This allows the information to be appropriately stored according to the emotion at the time the information was stored. For example, information associated with a strong emotion can be stored longer than information associated with a weak emotion.

[0061] The recording control unit 270 may change the information associated with a weak emotion so that it is stored for a longer period of time if similar information is consecutive than if it is not consecutive. For example, the recording control unit 270 changes the information to be stored for a longer period of time if the information of the same person, event, etc. is stored a predetermined number of times within a predetermined period of time. This allows information associated with a weak emotion to be stored for a longer period of time if it is not consecutive.

[0062] When switching the recording format of the information generated by the processing unit 152 from the first recording format to the second recording format, the recording control unit 270 transmits an instruction to the robot 40 to tag the information in the second recording format. This allows the user 30 to clearly recall the situation at the time from the tagged information when checking the recorded information. Furthermore, by tagging, the information can be easily found when searching for information on a specific situation from the recorded information.

[0063] However, people may tend to generalize memorized information over time. For example, people tend to gradually forget the details of memorized information over time, and the information becomes generalized. The last memory that remains is a higher-level conceptualization.

[0064] Therefore, the recording control unit 270 may update the information stored in the storage unit 280 so as to generalize the stored information as time passes from the timing at which the information was stored. For example, the recording control unit 270 updates the information stored in the recording data 292 to information of a higher concept as time passes from the timing at which the information was stored. For example, the recording control unit 270 reads out the information stored in the recording data 292 every time a predetermined period of time passes from the timing at which the information was stored. The predetermined period of time may be a fixed period of time, or may be determined in accordance with the period of time during which memory fades. For example, the predetermined period of time may be determined to be one month, three months, six months, one year, three years, etc., in order of increasing length. The recording control unit 270 identifies a higher concept of the read out information. For example, the recording control unit 270 identifies general information about the detailed information included in the information. For example, the recording control unit 270 identifies a wider area including the location, or a wider time period, date, or time including the date and time, for the date and time or location information included in the information. Furthermore, the recording control unit 270 erases a part of the detailed contents of the read information. Then, the recording control unit 270 updates the information stored in the recording data 292 corresponding to the read information to the specified higher-level concept information. For example, if the recording data 292 stores "I went to a theme park with a character", the recording control unit 270 updates it to "I went to a theme park" first and then to "I went to an amusement park" over time. For example, if the recording data 292 stores "Sunny with about 30% clouds", the recording control unit 270 updates it to "Sunny with light clouds" first and then to "Sunny" and then to "Good weather" over time. For example, if the recording data 292 stores "I was stroked by a child", the recording control unit 270 updates it to "Someone stroked me" first and then to "I was stroked" over time. Also, for example, if "traveled to Thailand" is stored, the record control unit 270 first updates it to "traveled to Thailand", then to "traveled to Southeast Asia", then to "traveled abroad", and then to "traveled" over time. This allows the stored information to be generalized, compressed, or deleted over time.

[0065] Additionally, people tend to experience weaker emotions associated with memorized information over time.

[0066] Therefore, the recording control section 270 may update the emotion associated with the information stored in the storage section 280 over time. For example, the recording control section 270 updates the information stored in the recording data 292 so as to weaken the intensity of the emotion associated with the information stored in the recording data 292 over time from the timing at which the information was stored. For example, the recording control section 270 updates the emotion value associated with the information stored in the recording data 292 to a smaller value by a predetermined rate or a predetermined value each time a predetermined period of time has passed since the timing at which the information was stored. The predetermined period of time may be a fixed period of time, or may be determined corresponding to the period of time during which memory fades. This makes it possible to weaken the intensity of the emotion associated with the stored information over time.

[0067] The action decision rule 282 defines an action to be performed by the robot 40 in association with the state of the robot 40. For example, the action decision rule 282 defines an action to be performed by the robot 40 in association with the internal state of an artificial neuron of a neural network. For example, the action decision rule 282 defines an action of vocalizing a word expressing happiness in association with the condition that an emotion artificial neuron defining the emotion of "happy" is high. The action decision rule 282 also defines an action to be taken when the robot becomes sleepy in association with the condition that an internal state of an endocrine artificial neuron defining an endocrine substance corresponding to sleepiness is high.

[0068] Incidentally, endocrine substances refer to substances such as neurotransmitters and hormones that are secreted in the body and transmit signals. Also, endocrine means that endocrine substances are secreted in the body. However, the endocrine substances of the robot 40 itself are one piece of information that affects the behavior of the robot 40, and do not mean that the robot 40 actually generates endocrine substances. Similarly, the emotions of the robot 40 itself are one piece of information that affects the behavior of the robot 40, and do not mean that the robot 40 actually has emotions.

[0069] The movement determination unit 250 determines the movement of the robot 40 based on the movement defined in the movement determination rules 282 in association with the activation state or internal state of each artificial neuron determined by the parameter processing unit 240.

[0070] The action determination unit 250 reads out information stored in the recorded data 292 of the storage unit 280 according to the action determined. For example, when the action determined in the action determination rule 282 requires output of information stored in the recorded data 292, the action determination unit 250 reads out necessary information from the recorded data 292. Also, for example, when the action determination unit 250 receives an output instruction from the user 30 regarding information stored in the recorded data 292, the action determination unit 250 reads out information instructed to be output from the recorded data 292. Also, when the action determination unit 250 has a conversation with the user 30, responds to an inquiry from the user 30, or asks a question to the user 30, the action determination unit 250 reads out information indicating the contents of the voice from the recorded data 292.

[0071] The motion determination unit 250 generates motion information for the robot 40 so that the robot 40 performs the determined motion, and outputs the generated motion information to the robot 40 together with the read information. When the motion determination unit 250 outputs the information stored in the recording data 292 of the storage unit 280 to the robot 40, the motion determination unit 250 performs control to change one or both of the behavior of the robot 40 and the output timing of the information according to the emotion associated with the information. For example, the motion determination unit 250 associates the degree of emotion with an event, and provides a lag in the timing of outputting information according to the degree of emotion. For example, the motion determination unit 250 controls to output information associated with a weak emotion over a longer period of time, or to output information with a filler word, compared to information associated with a strong emotion. For example, the motion determination unit 250 controls to output information associated with a weak emotion over a longer period of time. For example, the motion determination unit 250 controls to output information associated with a weak emotion with a filler word such as "um" or "hmm" inserted in the conversation, with a pause between the information. This allows the stored information to be output according to the emotion associated with the information. It also allows for natural conversation.

[0072] Also, for example, the operation determination unit 250 controls to output information in small amounts even if the information is associated with a strong emotion. The operation determination unit 250 controls to output information associated with a strong emotion little by little, compared with information associated with a weak emotion. Also, the operation determination unit 250 detects the surrounding situation when the information associated with a strong emotion is output from the robot 40 by a sensor, and controls one or both of the behavior of the robot 40 and the output timing of the information according to the detected surrounding situation. For example, when a strong emotion is associated, the robot 40 is controlled to insert filler words such as "Eh~" and "That's~" into the conversation, and to read and output the peripheral information in detail while thinking about the next line to be spoken. This makes it possible to output the stored information according to the emotion associated with the information. Also, it is possible to buy time to grasp the peripheral information. Also, since the peripheral information can be grasped and the next line to be spoken can be output, it is possible to lead to a natural conversation.

[0073] The motion information indicating the motion determined by the motion determination unit 250 is transmitted from the communication unit 208 to the robot 40. In the robot 40, the processing unit 152 controls the control target 155 to cause the robot 40 to perform the motion indicated by the information received from the server 200. This enables the robot 40 to perform an appropriate motion according to the current emotion of the robot 40.

[0074] 3 is a schematic diagram of a neural network 300. The neural network 300 is an example neural network for explaining the operation of the parameter processing unit 240. The neural network 300 includes a plurality of artificial neurons including artificial neuron 1, artificial neuron 2, artificial neuron 3, artificial neuron 4, artificial neuron 5, artificial neuron 6, artificial neuron 7, artificial neuron 8, artificial neuron 9, artificial neuron a, artificial neuron b, and artificial neuron c. The neural network 300 includes a plurality of artificial synapses including artificial synapse 301, artificial synapse 302, artificial synapse 303, artificial synapse 304, artificial synapse 305, artificial synapse 306, artificial synapse 307, artificial synapse 308, artificial synapse 309, artificial synapse 310, artificial synapse 311, artificial synapse 312, artificial synapse 313, artificial synapse 314, artificial synapse 315, artificial synapse 316, artificial synapse 317, artificial synapse 318, and artificial synapse 319. The artificial neuron corresponds to a neuron in a living organism. The artificial synapse corresponds to a synapse in a living organism.

[0075] The artificial synapse 301 connects the artificial neuron 4 and the artificial neuron 1. The artificial synapse 301 is an artificial synapse that connects in one direction, as shown by the arrow of the artificial synapse 301. The artificial neuron 4 is an artificial neuron that is connected to the input of the artificial neuron 1. The artificial synapse 302 connects the artificial neuron 1 and the artificial neuron 2. The artificial synapse 302 is an artificial synapse that connects in two directions, as shown by the arrows on both ends of the artificial synapse 302. The artificial neuron 1 is an artificial neuron that is connected to the input of the artificial neuron 2. The artificial neuron 2 is an artificial neuron that is connected to the input of the artificial neuron 1.

[0076] In this embodiment, an artificial neuron may be represented by N, and an artificial synapse may be represented by S. When identifying each artificial neuron, a superscripted reference symbol may be used as an identification letter. When identifying an arbitrary artificial neuron, i or j may be used as an identification letter. For example, N i represents an arbitrary artificial neuron.

[0077] In addition, an artificial synapse may be identified by the identification numbers i and j of the two artificial neurons connected to the artificial synapse. For example, S 41 is N 1 and N 4 It represents an artificial synapse that connects ij is N i The output of N j represents an artificial synapse that inputs to S ji is N j The output of N i represents an artificial synapse input to

[0078] In FIG. 3, A to J indicate the defined states of the robot 40. The states of the robot 40 include the emotions of the robot 40, the state of production of endocrine substances, the situation of the robot 40, and the like. As an example, 4 , N 6 , and N 7 is a conceptual artificial neuron in which a concept that represents the situation of the robot 40 is defined. For example, N4 is a conceptual artificial neuron that has been assigned the situation "the bell rang". 6 is a conceptual artificial neuron that has been assigned the state "charging has started". 7 is a conceptual artificial neuron to which the state “charge level is below the threshold” is assigned.

[0079] N 1 , N 3 , N b and N c is an emotion artificial neuron in which the emotion of the robot 40 is defined. 1 is an emotional artificial neuron to which the emotion "happy" is assigned. 3 is an emotional artificial neuron to which the emotion "sad" is assigned. b is an emotional artificial neuron to which the emotion "fear" is assigned. c is an emotional artificial neuron to which the emotion “fun” is assigned.

[0080] N 2 , N 5 and N a is an endocrine artificial neuron with a defined endocrine state of the robot 40. 5 is an endocrine artificial neuron that is assigned the developmental state of dopamine. Dopamine is an example of an endocrine substance involved in the reward system. 5 is an example of an endocrine artificial neuron involved in the reward system. 2 is an endocrine artificial neuron that is assigned the generation state of serotonin. Serotonin is an example of an endocrine substance involved in the sleep system. 2 is an example of an endocrine artificial neuron involved in the sleep system. a is an endocrine artificial neuron that has been assigned the developmental state of noradrenaline. Noradrenaline is an example of an endocrine substance involved in the sympathetic nervous system. a is an endocrine artificial neuron involved in the sympathetic nervous system.

[0081] The definition information 284 in the storage unit 280 stores information defining the state of the robot 40 as described above for each of the multiple artificial neurons that make up the neural network. Thus, the neural network 300 includes conceptual artificial neurons, emotional artificial neurons, and endocrine artificial neurons. Conceptual artificial neurons, emotional artificial neurons, and endocrine artificial neurons are artificial neurons in which the meanings of concepts, emotions, endocrine, etc. are explicitly defined. In contrast, N 8 Or N 9 is an artificial neuron with no defined state for the robot 40. 8 Or N 9 is an artificial neuron whose meanings, such as concepts, emotions, and endocrine functions, are not explicitly defined.

[0082] The parameters of the neural network 300 are i I, which is the input to t i And, from outside the neural network, N i E, which is the input to t i And, N i The parameters of and S i and parameters.

[0083] N i The parameters of N i S stands for status t i And, N i represents the internal state of the artificial neuron, V i m t And, N i T represents the threshold for firing i t And, N i The time when the last firing occurred is t f and the artificial neuron N at the final firing time i V represents the internal state of i m tf and the output increase / decrease parameter a t i , b ti , h t i The output increase / decrease parameter is an example of a parameter that determines the time evolution of the output when the artificial neuron fires. In this embodiment, the subscript t indicates that the parameter can be updated as time progresses. i m t is information corresponding to the membrane potential of the artificial neuron, and is an example of a parameter representing the internal state or output of the artificial neuron.

[0084] S ij The parameters of S ij BS represents the coupling coefficient of the artificial synapse t ij And, S. ij N connected i and N j The time when the two last fired simultaneously is t cf and BS, which represents the coupling coefficient at the time of the last simultaneous firing. ij tcf and a, which is the coupling coefficient increase / decrease parameter t ij , b t ij , h t ij The increase / decrease parameter of the coupling coefficient is an example of a parameter that determines the time evolution of the coupling coefficient after the two artificial neurons connected by the artificial synapse last fired simultaneously.

[0085] The parameter processing unit 240 updates the above-mentioned parameters based on the input from the external input data generating unit 230 and the neural network to determine the activation state of each artificial neuron. The operation determining unit 250 determines the operation of the robot 40 based on the internal state or activation state of at least some of the artificial neurons determined by the parameter values ​​of at least some of the artificial neurons among the multiple artificial neurons in the neural network, and the state defined for at least some of the artificial neurons by the definition information 284. Note that the activation state may be an activated state or an inactivated state. In this embodiment, activation may be called "firing", and inactivation may be called "not firing". Note that, as described later, the "firing" state is divided into an "upward phase" and a "downward phase" depending on whether the internal state is rising or not. "Not firing", "upward phase", and "downward phase" are represented by the status Sti.

[0086] Figure 4 shows the parameters of the neural network in a schematic table format. Each neuron N is assigned a threshold T t and the increase / decrease parameter h t , a t and b t Each artificial synapse has a coupling coefficient BS t and the increase / decrease parameter h t , a t and b t The parameters are as follows: i Every time, an artificial synapse is used, i In each row, the parameters of all the artificial neurons directly connected to the corresponding artificial synapse are shown, as well as the parameters of the corresponding artificial synapse.

[0087] 5 shows an outline of the operation flow of the server 200 when the robot 40 is started or reset. When the server 200 receives a notification that the robot 40 has been started or reset, the parameter processing unit 240 performs initial setting of the parameters of the neural network. For example, the parameter processing unit 240 acquires initial values ​​of the parameters from the storage unit 280 and generates parameter data of the neural network in a predetermined data structure (S502). In addition, the parameter processing unit 240 sets the values ​​of the parameters of the neural network at time t0. When the initial setting is completed, in S504, a loop for time t is started.

[0088] In S510, the parameter processing unit 240 calculates the time step t n+1 Calculate the parameters corresponding to the changes due to the electrical influence of the artificial synapse in . t ij Calculate.

[0089] In S520, the parameter processing unit 240 calculates the time step t n+1 Calculate the parameters corresponding to the changes due to the chemical effects of endocrine substances in N i and S ij More specifically, we calculate the change in the parameters of n+1 Artificial neuron N affected by endocrine artificial neuron i The internal state of the S ij Calculate the coupling coefficient increase / decrease parameters and coupling coefficients.

[0090] In S530, the parameter processing unit 240 acquires an input from outside the neural network. Specifically, the parameter processing unit 240 acquires the output of the external input data generating unit 230.

[0091] In S540, the parameter processing unit 240 calculates the time step t n+1 In N iCalculate the internal state of V i m tn+1 and Status S tt i Then, in S550, at time t n+1 The values ​​of the parameters at time t are stored in the parameter 288 of the storage unit 280. n+1 The control unit 250 outputs the values ​​of the parameters in the step S100 to the operation determination unit 250, the switching control unit 260, and the recording control unit 270.

[0092] In S560, the recording control unit 270 associates emotions with the various types of information received by the communication unit 208 and stores them in the recorded data 292. For example, the recording control unit 270 associates emotions with various types of information about the user 30, peripheral information, and event information and stores them in the recorded data 292. For example, the recording control unit 270 associates values ​​of parameters (emotion values) representing the robot 40 as emotions of the robot 40 and stores them in the recorded data 292.

[0093] In S506, the parameter processing unit 240 judges whether or not to end the loop. For example, it judges to end the loop when the time represented by the time step reaches a predetermined time or when sensor information from the robot 40 is not received for a predetermined time. If the loop is not to be ended, the process returns to S510 and the calculation of the next time step is further performed. If the loop is to be ended, this flow is ended.

[0094] FIG. 6 is a diagram for explaining the calculation of the coupling coefficient of the artificial synapse. Here, the constant a is used as the initial value of the increase / decrease parameter. ij and b ij This section describes the cases where is defined.

[0095] time t n At the time step of S ij N on both ends of i and N j are firing, the parameter processing unit 240 n+1 in B.S. tn+1 ij ,BStn+1 ij =BS tn ij +a tn ij ×(t n+1 -t n ) is calculated. On the other hand, at time t n At time steps S i and S j If none of them fires at time t n+1 Coupling coefficient BS tn+1 ij ,BS tn+1 ij =BS tn ij +b tn ij ×(t n+1 -t n ) is calculated. Also, BS tn+1 ij If is negative, BS tn+1 ij is set to 0. Note that BS ij is a positive value of S ij So, a t ij is a positive value, and b t ij is a negative value. ij S is negative ij So, a t ij is a positive value, and b t ij is a negative value.

[0096] As shown in Figure 6, at time t0, the artificial neurons at both ends fire simultaneously. t ij is a per unit time t0 ij In addition, since there are no simultaneous firings at time t1, BS t ij is |b per unit time t1 ij In addition, due to simultaneous firing at time t4, BS t ij is a per unit time t4 ij increases with.

[0097] Figure 7 shows the function h t ij The time evolution of the coupling coefficient when is defined is shown below. t ij t cf The time elapsed from Δt (= tt cf ) ≧0. t ij is a function of at least Δt and takes real values.

[0098] The function 700 shown in FIG. t ij The function 700 is an example of cf Coupling coefficient BS tcf ij and Δt. The function 700 monotonically increases when Δt is less than a predetermined value, and monotonically decreases and gradually decreases toward 0 when Δt is greater than the predetermined value. The function 700 has a value BS tcf ij Take.

[0099] FIG. 7 shows that a function 700 is defined as a parameter for increasing or decreasing the coupling coefficient. i and N j The parameter processing unit 240 calculates the coupling coefficients of the BSs at the times t1 to t6 based on the function 700 and Δt. t ij In the time range from time t1 to time t6, N i and N j do not fire simultaneously. Therefore, for example, after time t2, the coupling coefficient decreases monotonically.

[0100] Figure 8 shows N i and N j The time evolution of the coupling coefficient when the neurons fire simultaneously is shown in Fig. The coupling coefficient is calculated in the same manner as in FIG. 7 from time t0 to time t2. At time t2, N i and N jWhen these two fire simultaneously, the parameter processor 240 t ij (t-t2,BS t2 ij ) and calculates the coupling coefficient at each of times t3 to t6. In this way, the coupling coefficient increases each time simultaneous firing is repeated. This has the effect of strengthening the artificial synaptic connection, as in Hebb's law in living organisms. On the other hand, as shown in Figures 6 and 7, when the time without simultaneous firing becomes long, it has the effect of weakening the artificial synaptic connection.

[0101] Fig. 9 shows a schematic diagram of effect definition information that defines a chemical effect given to a parameter. This effect definition information is used to calculate a change in the parameter in S520 of Fig. 5. The definition information includes a condition regarding the internal state of the endocrine artificial neuron, information specifying the artificial neuron or artificial synapse that exerts an effect, and an equation that determines the effect content.

[0102] In the example of FIG. 9, the endocrine artificial neuron N 2 is an endocrine artificial neuron to which the endocrine substance responsible for sleepiness is assigned. Endocrine artificial neuron N 2 For definition information about Vm tn 2 >T tn 2 "Conditions for Endocrine Artificial Neuron N 2 The artificial neuron N 1 and N 3 ", and "T tn+1 i =T tn i Parameter processing unit 240 determines Vm tn 2 T tn 2 If it exceeds t n+1 Emotional Artificial Neuron N 1 and N 3The threshold is increased by 10%. This makes it difficult for the emotional artificial neuron to fire when drowsiness occurs. For example, the conceptual artificial neuron N 7 The output of the endocrine artificial neuron N 2 By defining a neural network connected to the input of the above, it is possible to embody the phenomenon in which emotions become less likely to increase as the battery level becomes low.

[0103] In addition, the endocrine artificial neuron N 5 is an endocrine artificial neuron to which dopamine is assigned. Endocrine artificial neuron N 5 The first definition information for "Vm tn 5 >T tn 5 and Vm tn 4 >T tn 4 "Conditions for Endocrine Artificial Neuron N 5 As an artificial synapse that is influenced by 49 and S 95 ", and "a" is the formula that determines the effect. tn+1 ij =a tn ij The formula "Vm × 1.1" is defined. tn 5 T tn 5 Exceeds Vm tn 4 T tn 4 If it exceeds t n+1 Artificial Synapse S 49 and S 95 Increases the increase / decrease parameters by 10%.

[0104] This allows the concept of artificial neuron N to be defined as the situation in which the reward system's endocrine artificial neuron fires, resulting in the "bell ringing" condition. 4 is firing, the implicit artificial neuron N 9 Conceptual artificial neuron N via 4and N 5 This allows the binding of the artificial neuron N to the reward system when the "bell" is rung. 5 becomes more likely to ignite.

[0105] In addition, the endocrine artificial neuron N 5 The second definition of "Vm tn 5 >T tn 5 "Conditions for Endocrine Artificial Neuron N 5 The artificial neuron that is influenced by 1 ", and "T tn+1 i =T tn i The formula "Vm × 1.1" is defined. tn 5 T tn 5 If it exceeds t n+1 Artificial neuron N 1 This reduces the increase / decrease parameter of the reward system endocrine artificial neuron N 5 When this emotion is ignited, it becomes easier for the emotion of happiness to be ignited.

[0106] According to the definition of the effect of such a reward system on endocrine artificial neurons, it is possible to implement an implementation in which, if the action of charging the robot 40 while ringing a bell is repeated, the robot 40 will behave in a way that expresses happiness just by ringing the bell.

[0107] The influence definition information is not limited to the example of FIG. 9. For example, a condition may be defined that the internal state of the artificial neuron is equal to or lower than a threshold. A condition regarding the status of the artificial neuron, for example, a condition regarding an ascending phase, a descending phase, or non-firing, may be defined. The influence range may be defined by directly specifying an artificial neuron or an artificial synapse, or by defining "all artificial synapses connected to a specific artificial neuron." Regarding the influence formula, in the case where the target is an artificial neuron, in addition to multiplying the threshold by a constant, a formula may be defined in which a constant is added to the threshold or an increase / decrease parameter of the internal state is multiplied by a constant. In the case where the target is an artificial synapse, in addition to multiplying the increase / decrease parameter by a constant, an formula may be defined in which a coupling coefficient is multiplied by a constant.

[0108] The influence definition information is stored in the definition information 284 of the storage unit 280. Thus, the storage unit 280 stores influence definition information that defines the influence of at least one of the internal state and firing state of the endocrine artificial neuron on the parameters of at least one of the other artificial neurons and artificial synapses that are not directly connected to the endocrine artificial neuron via an artificial synapse. Then, the parameter processing unit 240 updates the parameters of at least one of the other artificial neurons and artificial synapses that are not directly connected to the endocrine artificial neuron via an artificial synapse based on at least one of the internal state and firing state of the endocrine artificial neuron and the influence definition information. Furthermore, the parameters of the other artificial neurons that are influenced by at least one of the internal state and firing state of the endocrine artificial neuron can include at least one of the parameters that define the threshold value, firing state, and time development of the output at the time of firing of the other artificial neuron. Furthermore, the parameters of the artificial synapse influenced by at least one of the internal state and the firing state of the endocrine artificial neuron can include at least one of the coupling coefficient of the artificial synapse and a parameter that determines the time evolution of the coupling coefficient after the two artificial neurons linked by the artificial synapse last fired simultaneously. Furthermore, the influence definition information includes information that determines the influence that the firing state of the endocrine artificial neuron associated with the reward system has on the threshold of the emotional artificial neuron, and the parameter processing unit 240 updates the threshold of the emotional artificial neuron in accordance with the influence definition information when the endocrine artificial neuron fires.

[0109] Figure 10 shows the V tn+1 i and S tn+1 i 5. The process of this flowchart can be applied to a part of the process in S540 in FIG. 5. In S1100, the parameter processing unit 240 calculates S tn i indicates non-firing.

[0110] S tn i If indicates non-firing, the parameter processing unit 240 i Input to Itn+1 i Specifically, when the input from outside the neural network is N i If not connected to I tn+1 i =Σ j B.S. tn+1 ji ×VM tn j ×f(S tn j ) is calculated by the following formula: i If connected to I tn+1 i =Σ j B.S. tn+1 ji × VM tn j ×f(S tn j )+E tn+1 i Here, E tn i is the time t from outside the neural network. n This is the input in

[0111] In addition, f(S) returns 0 when S is a value that indicates no firing, and returns 1 when S is a value that indicates an ascending or descending phase. This model corresponds to a model in which a synapse transmits an action potential only when a neuron fires. Note that f(S)=1 may also be returned. This corresponds to a model in which a membrane potential is transmitted regardless of the firing state of the neuron.

[0112] In S1112, the parameter processing unit 240 tn+1 i T tn+1 i Determine whether it exceeds I tn+1 i T tn+1 i If it exceeds Vm tn+1 i is calculated based on the increase / decrease parameters, and Vm tn+1 i Depending on S tn+1i is set to a value indicating the rising phase or the falling phase (S1114), and this flow ends.

[0113] In the S1100, tn i When Vm is in the rising phase or the falling phase, the parameter processing unit 240 tn+1 i Then, the parameter processing unit 240 calculates t n+1 By Vm t i When Vmin is reached, S tn+1 i Set to the unfired value and t n+1 By Vm t i If Vmin is not reached, S tn+1 i The parameter processing unit 240 sets t n+i By Vm t i When Vmax is reached, S tn+1 i Set the value of the falling phase to t n+1 By Vm t i If Vmax is not reached, S tn+1 i Set the value of the rising phase to

[0114] In this way, N i If is firing, even if the output falls below the threshold, N i The output of is independent of the input. Such a period corresponds to the absolute refractory period in biological neurons.

[0115] Figure 11 shows the N i V when does not ignite t i FIG. 13 is a diagram for explaining an example of calculation of

[0116] At time step t0, N i is not fired. I at time t1 t1 i Tt1 i If V at time t1 is equal to or less than V t1 i , V t1 i =I t1 i Calculate V from time t0 to t1. t i , V t i =I t0 i Similarly, the parameter processing unit 240 calculates the time step t n The calculated V t n maintains the value of until the next time step, and V tn+1 In I tn+1 Change to.

[0117] Figure 12 shows the N i V when ignition occurs i t FIG. 12 is a diagram for explaining an example of calculation of the constant a i and b i This is a calculation example when is defined.

[0118] At time step t0, N i is not fired. I at time t1 t1 i T t1 i If the parameter processing unit 240 exceeds V t1 i , V t1 i =I t1 i Calculate V from time t0 to t1. t i , V t i =I t0 i Here, the calculation is performed using I at time t1. t1 i is less than Vmax. I at time t1 t1 iIf exceeds Vmax, I t1 i =Vmax.

[0119] As shown in FIG. 12, after time t1, the parameter processing unit 240 t i Until the time when V reaches Vmax, t i per unit time t ij In addition, the parameter processing unit 240 increases N i Status S t i is determined to be the ascending phase.

[0120] Also, V t i When V reaches Vmax, V t i Until V reaches Vmin, V t i per unit time |b t i In addition, the parameter processing unit 240 reduces N i The status of V is determined to be in the descending phase. t i When Vmin is reached, V at the next time t6 i , V t6 i =I t6 i Calculate by V t i After reaching Vmin, the status is determined as non-firing.

[0121] In addition, N i If the status of is in the declining phase, the calculated Vm t i T t i Even if it falls below Vm t i I t i The parameter processing unit 240 does not depend on Vm t i Tt i Even if it falls below Vm t i Vm increases or decreases according to the parameter until it reaches Vmin. t i Calculate.

[0122] Figure 13 shows the N i As a parameter of the function h t i The time evolution of the coupling coefficient when is defined is shown below. In general, h t i is the firing time t f The time elapsed from Δt (= tt f ) ≧0. t i is at least a function of Δt. t i takes real values, and h t i The value range is from Vmin to Vmax.

[0123] The function 1300 shown in FIG. t i The function 1300 is an example of f In tf i and Δt. The function 1300 increases monotonically when Δt is less than a predetermined value, and decreases monotonically when Δt is greater than a predetermined value. The function 1300 has a value Vm tf i Take.

[0124] FIG. 13 shows a case where a function 1400 is defined as an increase / decrease parameter of the internal state, and N i The parameter processing unit 240 calculates the output when the function 1400, Δt and Vm f i Based on this, Vmti is calculated for each time from time t1 to time t5. t i Since Vmin is reached at time t5, Vm i =I t6i It becomes.

[0125] FIG. 14 shows an example of a rule 1400 stored in the recording format switching rule 290 in the form of a table. The rule 1400 is 1 , N 3 , N b and N c When at least the first condition is satisfied that any one of the Vmti exceeds a threshold, the information recording format is "switched to a low compression format". In this way, when information is recorded in a high compression format, the switching control unit 260 determines to switch the information recording format to a low compression format when the first condition is satisfied from a state in which the first condition is not satisfied. Note that, as the threshold, each of N j The threshold value may be higher than Tit.

[0126] Also, rule 1400 states that 5 and N a VM t i If the second condition is met, that is, the total value of the data exceeds the threshold, the data recording format is switched to a low compression format. " is set. In this way, when information is recorded in a high compression format, the switching control unit 260 determines to switch the information recording format to a low compression format when the second condition is satisfied from a state in which the second condition is not satisfied. Note that, as a threshold value, j The total value of Vmax of each is multiplied by a constant 0.9. j T i t may be higher than the sum of the above.

[0127] N 1 , N 3 , N b and N care emotion artificial neurons in which emotions "happy", "sad", "fearful" and "fun" are defined, respectively. Therefore, the parameter processing unit 240 determines the intensity of the emotion based on the internal state of the emotion artificial neuron, and when the determined intensity of the emotion exceeds a predetermined threshold, the recording format can be switched to a low compression format.

[0128] N5 and Na are endocrine artificial neurons in which the endocrine substances "dopamine" and "noradrenaline" are defined, respectively. The sum of the parameters of the internal states of these endocrine artificial neurons is an example of an index representing the intensity of the emotion of "excitement." Therefore, the parameter processing unit 240 determines the intensity of the emotion based on the internal states of the endocrine artificial neurons, and when the determined intensity of the emotion exceeds a predetermined threshold, the recording format can be switched to a low compression format.

[0129] Also, rule 1400 states that 1 , N 3 , N b and N c Any VM t i is also equal to or smaller than the first threshold, and N 5 and N a VM t i When a third condition is satisfied, that is, the sum of the first and second thresholds is equal to or less than the second threshold, the operation of "switching the data recording format to a high compression format" is defined. Therefore, when information is recorded in a low compression format, the switching control unit 260 determines to switch the information recording format to a high compression format when the third condition is satisfied from a state in which the third condition is not satisfied. In this way, the recording format can be switched to a high compression format in response to the intensity of the emotion becoming equal to or less than the predetermined threshold.

[0130] The first threshold of the third condition is j The second threshold value of the third condition is the value obtained by multiplying the Vmax of each N jThe value is obtained by multiplying the total value of Vmax of the three conditions by the constant 0.8. In this manner, the case is exemplified in which the first threshold value of the third condition is smaller than the threshold value of the first condition, and the second threshold value of the third condition is smaller than the threshold value of the second condition. However, the first threshold value may be the same as the threshold value of the first condition, and the second threshold value may be the same as the threshold value of the second condition. Also, the first threshold value of the third condition is set to a value obtained by multiplying the total value of Vmax of the three conditions by the constant 0.8. j T i t The second threshold of the third condition may be higher than each N j T i t In addition, the threshold value for each condition is not limited to these examples, and various values ​​can be applied.

[0131] Next, a description will be given of the motion control of the robot 40. Fig. 15 is a diagram showing a flowchart relating to the motion control of the robot 40. When the motion determination unit 250 determines the motion of the robot 40, it reads out information stored in the recorded data 292 of the storage unit 280 according to the determined motion (S600).

[0132] In S610, the movement determination unit 250 changes one or both of the behavior of the robot 40 and the output timing of the information according to the emotion associated with the read information. Then, in S620, the movement determination unit 250 generates movement information for the robot 40 so as to perform the determined movement, outputs it to the robot 40 together with the read information, and ends this flow. In this way, the system 20 can cause the robot 40 to output the stored information according to the emotion associated with the information.

[0133] Next, the deletion of information stored in recorded data 292 will be described. Fig. 16 is a diagram showing a flowchart relating to control of the storage period of data. Recording control unit 270 judges the strength of the emotion associated with the information stored in recorded data 292 of storage unit 280 (S700). For example, recording control unit 270 judges whether the emotion value associated with the information is equal to or greater than a predetermined threshold value at which the emotion is deemed to be strong, and judges the emotion to be strong if the emotion value is equal to or greater than the threshold value, and judges the emotion to be strong if the emotion value is less than the threshold value.

[0134] In S710, the recording control unit 270 sets a storage period according to the strength of the emotion. For example, the recording control unit 270 sets a long storage period for storing information associated with a strong emotion, and sets a short storage period for information associated with a weak emotion.

[0135] In S720, the recording control unit 270 deletes from the recording data 292, any information for which the retention period has elapsed since the date and time when the information was stored in the recording data 292, and ends this flow. This allows the system 20 to appropriately store information according to the associated emotion. For example, information associated with a strong emotion can be stored longer than information associated with a weak emotion.

[0136] According to the system 20, the robot 40 continuously transmits highly compressed information such as skeletal data to the server 200 during a period when the emotion of the robot 40 is not significantly high, and the server 200 records the information. The continuous information such as skeletal data recorded in the server 200 can be used when analyzing the memory of the robot 40. When the emotion of the robot 40 significantly increases, the robot 40 starts transmitting full HD video data and audio data, and during a period when the emotion is strong enough to exceed a certain value, the server 200 records low-compression information including the full HD video data and audio data in addition to the skeletal data. When the robot 40 is requested by the user 30 to provide a video of the robot 40's memories, the robot 40 requests the server 200 to transmit the full HD video data and audio data, and provides the user 30 with the video data and audio data received from the server 200.

[0137] In this way, according to the system 20, high-quality video data of scenes in which the robot 40 feels strong emotions can be stored in the server 200. On the other hand, when the robot 40 does not feel strong emotions, simplified information such as skeletal data can be stored in the server 200. In this way, the robot 40, like a human being, can retain clear memories of times when it felt strong emotions, while retaining simplified memories of times when it did not feel strong emotions.

[0138] In this embodiment, "happy", "sad", "fearful", "fun" and "excited" are described as emotions, but the emotions handled by the system 20 are not limited to these. In addition, in this embodiment, "dopamine", "serotonin" and "noradrenaline" are described as endocrine substances, but the endocrine substances handled by the system 20 are not limited to these.

[0139] In addition, the parameter processing unit 240 has been described as determining the emotion of the robot 40 using a neural network, but the present invention is not limited to this. The emotion may be determined by any method. In addition, the parameter processing unit 240 has been described as determining the emotion of the robot 40, but the parameter processing unit 240 may determine the emotions of the robot 40 and the user 30, respectively, or may determine the emotion of the user 30. The record control unit 270 may associate the emotions of the robot 40 and the user 30 with various information received by the communication unit 208 and store them in the record data 292, or may associate the emotions of the user 30 with the various information and store them in the record data 292.

[0140] The functions of the server 200 may be implemented by one or more computers. At least some of the functions of the server 200 may be implemented by a virtual machine. At least some of the functions of the server 200 may be implemented in a cloud. Among the functions of the server 200, the functions of the components other than the storage unit 280 can be realized by a CPU operating based on a program. For example, at least some of the processing described as the operation of the server 200 can be realized by a processor controlling each piece of hardware (e.g., a hard disk, a memory, etc.) possessed by the computer according to a program. In this way, at least some of the processing of the server 200 can be realized by the processor operating according to a program to control each piece of hardware, so that each piece of hardware including the processor, the hard disk, the memory, etc. and the program operate in cooperation with each other. That is, the program can make the computer function as each component of the server 200. Similarly, among the components of the robot 40, the functions of the components other than the control target 155 and the sensor unit 156 can be realized by a CPU operating based on a program. That is, the program can make the computer function as each component of the robot 40. The computer may load a program that controls the execution of the above-mentioned processing, and operate according to the loaded program to execute the processing. The computer can read the program from a computer-readable recording medium that stores the program. The program can also be supplied to the computer through a communication line, and the computer can read the program through the communication line. The program may be read from a source connected via a communication line.

[0141] In the embodiment described above, the server 200 separate from the robot 40 is responsible for processing the neural network. Also, the server 200 separate from the robot 40 stores information such as video data. However, the robot 40 itself may perform the functions of the server 200, such as processing the neural network. Also, the robot 40 itself may store information such as video data. Also, the robot 40 is an example of a device to be controlled by the server 200. The device to be controlled is not limited to the robot 40, and various devices such as home appliances, vehicles, toys, etc. can be applied as the controlled devices.

[0142] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0143] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order. [Explanation of symbols]

[0144] 20 Systems 30 users 40 Robot 90 Communication Network 152 Processing section 155 Control Object 156 Sensor section 158 Communications Department 161 Mike 162 3D Depth Sensor 163 2D Camera 164 Distance Sensor 200 Servers 202 Processing section 208 Communications Department 210 Initial value setting section 230 External input data generation unit 240 Parameter Processing Section 250 Action Decision Unit 260 Switching control section 270 Recording control section 280 Storage Unit 282 Action Decision Rules 284 Definition information 286 Parameter Initial Values 288 Parameters 290 Recording format switching rules 292 Recorded Data 300 Neural Networks

Claims

1. A control system for controlling the behavior of a robot, comprising: an emotion determining unit that determines an emotion of one or both of the user and the robot based on at least a part of information detected by a sensor that continuously detects the user; a storage unit that stores information obtained by processing at least a part of the information detected by the sensor in association with the determined emotion of one or both of the user and the robot; an output control unit that performs control to change one or both of an action of the robot and an output timing of the information stored in the storage unit according to an emotion associated with the information when the information is output by the robot; A control system comprising:

2. The emotion determination unit determines an intensity of an emotion of one or both of the user and the robot, The storage unit stores information obtained by processing at least a portion of the information detected by the sensor in association with the determined emotion intensity; The output control unit controls to output information associated with a weak emotion over a longer period of time compared with information associated with a strong emotion, or to output the information together with a filler word. The control system of claim 1 .

3. The emotion determination unit determines an intensity of an emotion of one or both of the user and the robot, The storage unit stores information obtained by processing at least a portion of the information detected by the sensor in association with the determined emotion intensity; The output control unit controls so as to output information associated with a strong emotion little by little in comparison with information associated with a weak emotion. The control system of claim 1 .

4. The output control unit detects a surrounding situation by a sensor when the robot outputs information associated with a strong emotion, and controls one or both of the behavior of the robot and the output timing of the information according to the detected surrounding situation. The control system of claim 1 .

5. The storage unit stores information including any one of surrounding information, smell, temperature, humidity, location information, weather, the number of people in the vicinity, facial expressions of the people in the vicinity, and emotions of the people in the vicinity. The control system of claim 1 .

6. A storage control unit that updates the information stored in the storage unit so as to generalize the stored information as time passes from the time when the information was stored. The control system of claim 1 further comprising:

7. a memory control unit that updates the information stored in the memory unit so as to weaken the intensity of the emotion associated with the stored information as time passes from the time when the information was stored. The control system of claim 1 further comprising:

8. A program for controlling the behavior of a robot, determining emotions of the user and the robot based at least in part on information detected by a sensor that continuously detects the user; storing information obtained by processing at least a portion of the information detected by the sensor in association with the determined emotion of one or both of the user and the robot; When outputting the stored information from the robot, changing one or both of the behavior of the robot and the output timing of the information according to the emotion associated with the information; Programs including.

Images