Control system, system, and program

Abstract

To properly record data according to emotions.SOLUTION: A control system according to an embodiment is a control system for controlling behaviors of a robot and includes a recording control unit and a switch control unit. The recording control unit processes at least a part of information detected continuously by a sensor to generate and record, one of information of a first recording format and information of a second recording format more massive than the information of the first recording format. The switch control unit switches the recording format of the recording control unit from the first recording format to the second recording format if the emotion of a robot or a user exceeds a threshold value when the information of the first recording format is being generated.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a control system, a system, and a program. [Background technology]

[0002] A terminal is known that learns a conversation between a user and a call partner and stores the call partner's replies to questions from the user in a reply table (see, for example, Patent Document 1). Also, an emotion generation device is known that includes a neural network that inputs user information, device information, and its own current emotional state and outputs the next emotional state (see, for example, Patent Document 2). Also, a technology is known that stores a spatiotemporal pattern in an associative memory that includes multiple electronic neurons having a layer-neural network relationship with directional artificial synaptic connectivity (see, for example, Patent Document 3). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-253389 A [Patent Document 2] Japanese Patent Application Publication No. 10-254592 [Patent Document 3] Special Publication No. 2013-535067 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional techniques have the problem that data may not be recorded appropriately according to the emotions of the user or robot.

[0005] The present invention has been made in consideration of the above, and has an object to provide a control system, a system, and a program that can appropriately record data according to emotions. [Means for solving the problem]

[0006] A control system according to one aspect of the embodiment is a control system for controlling the behavior of a robot, and includes a recording control unit and a switching control unit. The recording control unit processes at least a portion of information continuously detected by a sensor, and generates and records information in a first recording format or information in a second recording format having a larger amount of information than the information in the first recording format. When the information in the first recording format is being generated, the switching control unit switches the recording format used by the recording control unit from the first recording format to the second recording format if an emotion of the robot or a user exceeds a threshold. Effect of the Invention

[0007] According to one aspect of the embodiment, data can be appropriately recorded according to emotions. [Brief description of the drawings]

[0008] [Figure 1A] FIG. 1A is a schematic diagram illustrating an example of a system 20 according to the present embodiment. [Figure 1B] FIG. 1B is a diagram showing an example of emotion value changes. [Diagram 2] FIG. 2 is a diagram showing a schematic block configuration of the server 200 and the robot 40. As shown in FIG. [Diagram 3] FIG. 3 is a schematic diagram of a neural network 300. [Figure 4] FIG. 4 is a diagram showing a schematic diagram of the parameters of the neural network in a table format. [Diagram 5] FIG. 5 is a diagram illustrating an outline of the operation flow of the server 200 when the robot 40 is started or reset. [Figure 6] FIG. 6 is a diagram for explaining the outline of the calculation of the connection weights of the artificial synapses. [Figure 7] FIG. 7 is a diagram showing an outline of the time evolution of the coupling coefficient when a function ht ij is defined as an increase / decrease parameter of the coupling coefficient. [Figure 8]FIG. 8 is a diagram showing a schematic diagram of the time evolution of the coupling coefficient when further simultaneous firing occurs at time t2. [Figure 9] FIG. 9 is a diagram illustrating an outline of effect definition information that defines a chemical effect applied to a parameter. [Figure 10] FIG. 10 is a flowchart showing the calculation of the internal state and status. [Figure 11] FIG. 11 is a diagram for explaining an example of calculation of the internal state when the artificial neuron does not fire. [Figure 12] FIG. 12 is a diagram for explaining an example of calculation of an output when an artificial neuron fires. [Figure 13] FIG. 13 is a diagram showing a schematic diagram of the time evolution of the coupling coefficient when a function is defined as an increase / decrease parameter of the artificial neuron. [Figure 14] FIG. 14 is a diagram showing an example of rules 1400 stored in the recording format switching rule 290 in table form. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0010] Fig. 1A shows an example of a system 20 according to the present embodiment. Fig. 1B shows an example of an emotional value change. The system 20 includes a server 200, and a robot 40a and a robot 40b. The robots 40a and 40b communicate with the server 200 via a communication network 90 to exchange information.

[0011] The user 30a is a user of the robot 40a. The user 30b is a user of the robot 40b. The robot 40b has substantially the same functions as the robot 40a. Therefore, the system 20 will be described with the robots 40a and 40b collectively referred to as the robot 40.

[0012] The robot 40 performs various actions according to the situation, such as moving its head or limbs, conversing with the user 30, and providing an image to the user 30. At this time, the robot 40 decides the action in cooperation with the server 200. For example, the robot 40 transmits to the server 200 detection information such as a facial image of the user 30 acquired by a camera function and a voice of the user 30 acquired by a microphone function. The server 200 analyzes the detection information received from the robot 40, decides the action to be taken by the robot 40, and transmits action information representing the decided action to the robot 40. The robot 40 operates according to the action information received from the server 200.

[0013] The robot 40 has an emotion value that represents its own emotion. For example, the robot 40 has an emotion value that represents the strength of each emotion, such as "happy," "fun," "sad," "fear," and "excitement." The emotion value of the robot 40 is determined by the server 200. The server 200 causes the robot 40 to perform an action according to the determined emotion. For example, when the robot 40 has a high emotion value of excitement and is talking to the user 30, the server 200 causes the robot 40 to speak quickly. In this way, the robot 40 can express its own emotion through actions, etc.

[0014] The server 200 updates the current state of the robot 40 using a neural network based on the detection information received from the robot 40. The state of the robot 40 includes the emotion of the robot 40. Therefore, the server 200 determines the emotion of the robot 40 using the neural network.

[0015] The robot 40 also records the video data of the user 30 acquired by the camera function in the server 200. The robot 40 acquires the video data from the server 200 as necessary and provides it to the user 30. The robot 40 generates video data with a larger amount of information as the strength of its own emotion increases and records it in the server 200. For example, when the robot 40 records information in a highly compressed format such as skeletal data, the robot 40 switches to recording information in a low compression format such as HD video when the emotion value of the robot 40 exceeds a threshold. According to the system 20, high-definition video data when the emotion of the robot 40 increases can be recorded. The robot 40 also generates video data with a larger amount of information as the strength of the emotion of the user 30 increases and records it in the server 200. Specifically, when the robot 40 records information in a highly compressed format such as skeletal data, the robot 40 switches to recording information in a low compression format such as HD video when the emotion value of the user 30 exceeds a threshold. According to the system 20, high-definition video data of a moment when the user 30 is emotionally excited can be left as a record.

[0016] Furthermore, when the strength of the emotion of the robot 40 or the user exceeds a threshold, the robot 40 generates video data with a large amount of information and records it in the server 200. For example, when the robot 40 is recording information in a highly compressed format such as skeletal data, the robot 40 switches to recording information in a low-compression format such as HD video when the emotion value of the excitement of the robot 40 or the user exceeds a threshold. According to the system 20, it is possible to leave a record of high-definition video data when the intensity of the emotion of the robot 40 or the user 30 exceeds a threshold.

[0017] For example, suppose that the robot 40 has an interaction (e.g., a conversation) with the user and the user's emotion value of anger exceeds a threshold. In other words, suppose that the behavior of the robot 40 has made the user angry. In such a case, the robot 40 records data with a large amount of information in the server 200, and the system 20 can learn based on detailed information of when the robot 40 made the user angry.

[0018] In this case, the system 20 learns information regarding the flow of conversation between the user and the robot 40, remarks, and expressions, as well as the facial expressions and attitudes of the user and the robot 40. Then, the system 20 can perform control such as refraining from the behavior (conversation, gestures, etc.) of the robot 40 that made the user uncomfortable in the same situation from the next time onwards.

[0019] As a specific example, in the system 20, the robot 40 remains calm and records the flow of conversation, remarks, facial expressions, etc. in a scene in which the user becomes emotional. Based on this information, in a similar situation, the robot 40 is restricted in its behavior when the user becomes emotional, and is controlled to take a different action.

[0020] On the other hand, when performing such control, it is conceivable that, for example, the user may not be particularly concerned about the restricted behavior of the robot 40, and therefore, the behavior of the robot 40 may be excessively restricted.

[0021] Therefore, the system 20 may perform, for example, a time weighting on the behavior of the robot 40 that makes the user feel uncomfortable. For example, the system 20 may gradually release restrictions on the behavior of the robot 40 that makes the user feel uncomfortable as time passes.

[0022] Furthermore, the system 20 may record the behavior that the robot 40 performed toward the user and the emotional value of the user at that time in association with each other, and may determine whether the robot 40 will perform the same behavior after leveling out the emotional value of the user when the robot 40 performs the same behavior. In this case, the system 20 may cause the server 200 to record data with a large amount of information when the robot 40 performs a specific behavior (for example, a behavior that angers the user), regardless of the magnitude of the emotional value.

[0023] Furthermore, the robot 40 generates video data with a large amount of information and records it in the server 200 when triggered by transmission of the user's emotion to the robot, or when transmission of the robot's emotion to the user.

[0024] Here, emotion propagation refers to the emotion value of a specific emotion of the user or robot 40 that is the source of emotion exceeding a threshold, and then the emotion value of the same emotion increases in the user or robot 40 that is the destination of emotion. For example, emotion propagation refers to the emotion value of the anger of the robot increasing after the emotion value of the user exceeds a threshold. Note that when emotion propagation occurs, the emotion value of the anger of the user or robot 40 that is the destination of emotion propagation is below a predetermined value, that is, the user or robot 40 that is the destination of emotion propagation is not angry.

[0025] Specifically, as shown in FIG. 1B, after the user's anger emotion value A1 exceeds a threshold, the user's emotion is transmitted to the robot 40 through an interaction between the user and the robot (e.g., conversation, etc.), the emotion of the robot 40 changes from calm to anger, and the anger emotion value A2 exceeds a threshold.

[0026] In such a case, the robot 40 causes the server 200 to record data with a large amount of information, triggered by the user's emotion value A1 exceeding the threshold and then the robot's emotion value A2 exceeding the threshold.

[0027] In such a case, the server 200 can record detailed information when the user's emotion is transmitted to the robot 40. By collecting detailed information on such a case, the system 20 can improve the behavior of the robot 40 toward the user when the user becomes emotional, such as when the user's emotion value exceeds a threshold value.

[0028] 1B shows a case where the user's emotion value and the robot's emotion value exceed the threshold, but the present invention is not limited to this. Emotion propagation may be such that, after the user's anger emotion value exceeds the threshold, when the robot's anger emotion value exceeds the threshold, the user's anger emotion value is equal to or less than the threshold. Emotion propagation also includes a case where the robot's anger emotion value does not exceed the threshold if the robot's anger emotion value increases after the user's anger emotion value exceeds the threshold.

[0029] Furthermore, when a strong emotion of the robot 40 is transmitted to the user, the robot 40 also records a video or the like with a large amount of information in the server 200. As a specific example, it is assumed that after the emotion value of the robot 40, anger, exceeds a threshold, the emotion of the user changes from calm to anger through a conversation between the robot 40 and the user, and the emotion value of anger exceeds the threshold.

[0030] In such a case, the system 20 can collect detailed data when the robot 40's feeling of anger is transmitted to the user, and can use such data to improve the behavior of the robot 40.

[0031] For example, the system 20 can control the emotions of the robot 40 so that the emotions of the robot 40 do not change to anger due to comments made by the user, or change the behavior (tone of voice, expressions, gestures, etc.) of the robot 40 when it is angry to behavior that does not change the user's emotions to anger.

[0032] Furthermore, the system 20 may switch the detection trigger for starting recording of data with a large amount of information depending on the situation. For example, the system 20 switches the detection trigger between the emotion value of the robot 40 or the user exceeding a threshold, the transmission of the user's emotion to the robot 40, and the transmission of the robot's emotion to the user depending on the situation.

[0033] For example, the system 20 switches the detection trigger for starting recording using the topic of the conversation between the user and the robot 40, specific keywords or phrases in the conversation, etc. as keys. For example, when the system 20 detects a phrase or phrase in an utterance by the robot 40 that caused the user's emotion value to exceed a threshold in the past, the system 20 switches the detection trigger to the fact that the user's emotion value has exceeded the threshold. In other words, the system 20 switches the detection trigger using the rehashing of past conversations by the user or the robot 40 as a key.

[0034] In such a state, even if the emotion value of the anger of the robot 40 exceeds the threshold, the recording may not be started. That is, in this case, the system 20 starts recording by focusing on the emotion value of the user after the robot 40 uses a specific phrase or expression.

[0035] In addition, when the system 20 detects a phrase or expression in a user's speech that caused the robot 40's anger emotion value to exceed a threshold in the past, the system 20 switches to a detection trigger using the fact that the robot 40's anger emotion value exceeded the threshold as a key.

[0036] In addition, the system 20 switches the detection trigger to the transmission of the user's emotion to the robot 40, using as a key when the conversation is on a specific topic, such as when the user and the robot 40 are having a conversation about the topic of the conversation when the user's emotion was transmitted to the robot 40 in the past.

[0037] For example, in this case, the system 20 starts recording when it detects that the emotion value of the robot 40 has exceeded the threshold value after the emotion value of the user's anger has exceeded the threshold value. In addition, when switching the detection trigger, the system 20 may switch the detection trigger by combining conditions such as weather, temperature, the number of people in the vicinity, location, time of day, facial expression, and emotion.

[0038] In addition, the robot 40 tags information to be recorded when a current detection trigger is detected, such as when the emotion value of the robot 40 or the robot 40 exceeds a threshold value. Specifically, the robot 40 tags information detected by a sensor provided in the robot 40 and information on the emotion value of the user or the robot 40 to the video data. The tagged information includes, for example, surrounding information (information on objects present in the vicinity, etc.), smell, temperature, humidity, location information, weather, the number of people in the vicinity, their expressions and emotions, etc. This allows the user 30 to clearly recall the situation at the time from the tagged information when checking the recorded information. Furthermore, tagging makes it easy to find information when searching for information on a specific situation from the recorded information.

[0039] 2 is a schematic block diagram of the server 200 and the robot 40. The robot 40b has a sensor unit 156, a processing unit 152, a control target 155, a communication unit 158, and a display unit 157. The server 200 has a processing unit 202, a storage unit 280, and a communication unit 208. The processing unit 202 includes an initial value setting unit 210, an external input data generating unit 230, a parameter processing unit 240, an action determining unit 250, a switching control unit 260, and a recording control unit 270. The storage unit 280 stores action determining rules 282, definition information 284, parameter initial values ​​286, the latest parameters 288, a recording format switching rule 290, and recorded data 291. Store 2.

[0040] In the robot 40, the sensor unit 156 has sensors such as a microphone 161, a 2D camera 163, a 3D depth sensor 162, and a distance sensor 164. Each sensor of the sensor unit 156 continuously detects information. The sensor information detected by the sensor unit 156 is output to the processing unit 152. The 2D camera 163 is an example of an image sensor that continuously captures an object, and captures an object with visible light to generate video information. The 3D depth sensor 162 detects the contour of the object by continuously irradiating an infrared pattern and analyzing the infrared pattern from the infrared images continuously captured by the infrared camera. In addition, the sensor unit 156 may include various other sensors such as a clock, a gyro sensor, a touch sensor, a sensor for motor feedback, a sensor for detecting remaining battery capacity, an odor sensor, a temperature sensor, a humidity sensor, a GPS (Global Positioning System) sensor, etc.

[0041] The processing unit 152 is formed of a processor such as a CPU. The processing unit 152 transmits sensor information continuously detected by each sensor of the sensor unit 156 to the server 200 through the communication unit 158. The processing unit 152 also processes at least a part of the sensor information continuously detected by each sensor of the sensor unit 156 to generate information for recording. The processing unit 152 generates information in a first recording format or information in a second recording format having a larger amount of information than the information in the first recording format. The information in the first recording format is, for example, information in a high compression format, and the information in the second recording format is, for example, information in a low compression format. For example, the processing unit 152 generates shape data such as skeleton data of an object as information in the first recording format based on skeleton information continuously detected by the 3D depth sensor 162. The processing unit 152 also generates full HD video data and audio data from video information captured by the 2D camera 163 and audio information detected by the microphone 161. The full HD video data is an example of video data having more information than the shape data of an object.

[0042] The communication unit 158 ​​transmits the information in the first recording format or the information in the second recording format generated by the processing unit 152 to the server 200. In the server 200, the recording control unit 270 stores the information in the first recording format or the information in the second recording format received by the communication unit 208 from the robot 40 in the recording data 292. The recording control unit 270 stores the information received from each robot 40 in the recording data 292 in association with information identifying each robot 40.

[0043] Furthermore, in the robot 40, the communication unit 158 ​​acquires information stored in the recording data 292 from the server 200. The communication unit 158 ​​functions as a recorded information receiving unit that acquires information in the second recording format, including video data, recorded by the recording control unit 270. The processing unit 152 generates an image to be presented to the user 30, based on the video data included in the information in the second recording format received by the communication unit 158. The processing unit 152 functions as an image generating unit that generates an image to be presented to the user 30.

[0044] Furthermore, the communication unit 158 ​​receives operation information indicating the operation content from the server 200. The processing unit 152 controls the control target 155 based on the operation content received by the communication unit 158. The control target 155 includes a speaker, motors for driving each part of the robot 40 such as the limbs, a light emitting device, and the like. When information indicating the vocalization content is received from the server 200, the processing unit 152 outputs a sound from the speaker according to the received vocalization content. Furthermore, the processing unit 152 can control a part of the behavior of the robot 40 by controlling the drive motors of the limbs. Furthermore, the processing unit 152 can express a part of the emotion of the robot 40 by controlling these motors.

[0045] In the server 200, the communication unit 208 outputs the information received from the robot 40 to the processing unit 202. The initial value setting unit 210 stores the initial values ​​of the parameters indicating the initial state of the neural network received by the communication unit 208 in the parameter initial values ​​286 in the storage unit 280. Note that the initial values ​​of the parameters of the neural network may be predetermined in the server 200, or may be changeable by the user 30 via the communication network 90.

[0046] The external input data generating unit 230 processes at least a part of the sensor information received by the communication unit 208 to generate input information from outside the neural network, and outputs the generated information to the parameter processing unit 240. The parameter processing unit 240 performs calculations on the neural network based on the input information and current parameters 288 and definition information 284 of the neural network stored in the storage unit 280.

[0047] The artificial neurons of the neural network include a plurality of artificial neurons in which the situation of the robot 40 is defined, a plurality of emotion artificial neurons in which a plurality of emotions of the robot 40 itself is defined, and a plurality of endocrine artificial neurons in which the production state of the endocrine substance of the robot 40 itself is defined. The parameter processing unit 240 calculates parameters representing the internal states of the plurality of artificial neurons in the neural network based on the input information generated by the external input data generating unit 230. For example, the parameter processing unit 240 updates the parameters of the current internal states of the plurality of artificial neurons in which the situation of the robot 40 is defined, based on the input information generated by the external input data generating unit 230. In addition, the parameter processing unit 240 calculates parameters of the internal states of other artificial neurons in the neural network. As a result, for example, the parameters of the internal state of the emotion artificial neuron in which the emotion "happy" is defined are calculated. The parameters of the internal state of this emotion artificial neuron are an example of an index representing the strength of the emotion "happy". Therefore, the parameter processing unit 240 can determine the strength of the emotion in the control system based on the internal state of the emotion artificial neuron. In this way, the parameter processing unit 240 functions as an emotion determining unit that determines the intensity of an emotion using a neural network based on at least a part of the information detected by the sensor of the sensor unit 156.

[0048] The parameters of the neural network calculated by the parameter processing unit 240 are supplied to the switching control unit 260 and the operation determination unit 250. The switching control unit 260 determines the recording format of the information generated by the processing unit 152 of the robot 40 based on the parameters supplied from the parameter processing unit 240. When it is necessary to switch the recording format of the information generated by the processing unit 152, the switching control unit 260 causes the robot 40 to transmit an instruction to switch the recording format via the communication unit 208. In the robot 40, the processing unit 152 switches the recording format in accordance with the instruction received from the server 200.

[0049] For example, when the switching control unit 260 is causing the processing unit 152 to generate information in the first recording format, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format in response to an increase in the intensity of the emotion determined by the parameter processing unit 240. In this way, the switching control unit 260 switches the recording format of the information recorded by the recording control unit 270 from the first recording format to the second recording format in response to an increase in the intensity of the emotion determined by the parameter processing unit 240. This allows the information when the emotion of the robot 40 increases to be recorded in detail. In addition, in the robot 40, the processing unit 152 acquires the video data in the second recording format acquired from the server 200 and generates a video to be presented to the user 30. Therefore, the user 30 can enjoy the information when the emotion of the robot 40 increases as a video.

[0050] In addition, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format in response to an increase in the intensity of the emotion of the user 30. In this way, the switching control unit 260 switches the recording format of the information to be recorded by the recording control unit 270 from the first recording format to the second recording format in response to an increase in the intensity of the emotion determined by the parameter processing unit 240. This allows information when the emotion of the user 30 is increased to be recorded in detail. In addition, in the robot 40, the processing unit 152 acquires video data in the second recording format acquired from the server 200 and generates a video to be presented to the user 30. Therefore, the user 30 can enjoy information when his / her emotion is increased as a video. The emotion of the user can be detected, for example, based on a voice or facial expression detected from the user 30. In addition, the switching control unit 260 may recognize a change in the emotion of the user 30 based on the detection of an impact by a touch sensor (not shown) included in the sensor unit 156. For example, when an impact is detected by a touch sensor included in the sensor unit 156, the switching control unit 260 may recognize that the user 30 has become upset.

[0051] When the switching control unit 260 is causing the processing unit 152 to generate information in the second recording format, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the second recording format to the first recording format in response to a decrease in the intensity of the emotion of the robot 40 or the emotion of the user determined by the parameter processing unit 240. In this way, the switching control unit 260 switches the recording format of the information recorded by the recording control unit 270 from the second recording format to the first recording format in response to a decrease in the intensity of the emotion of the robot 40 or the emotion of the user determined by the parameter processing unit 240.

[0052] Furthermore, when the emotion of the user 30 is transmitted to the robot 40, or when the emotion of the robot 40 is transmitted to the user 30, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information generated by the processing unit 152 from the first recording format to the second recording format. This allows the information transmitted by the user 30 or the robot 40 to the other party to be recorded in detail.

[0053] The switching control unit 260 switches the trigger for switching from the first recording format to the second recording format depending on the situation. For example, the switching control unit 260 switches the detection trigger for switching from the first recording format to the second recording format depending on the situation.

[0054] The switching control unit 260 switches the detection trigger when it detects a conversation topic, a keyword or phrase included in a conversation, or a gesture of the user 30 or the robot 40 when the emotional value of the user 30 or the robot 40 exceeds a threshold based on data such as past conversations.

[0055] For example, the switching control unit 260 sets a detection trigger based on past data when the emotional values ​​of the user 30 or the robot 40, or both the user 30 and the robot 40, exceeded a threshold, and when the detection trigger is detected, switches from the first recording format to the second recording format.

[0056] In addition, in the robot 40, the processing unit 152 acquires the video data in the second recording format acquired from the server 200, and generates a video to be presented to the user 30. Therefore, the user 30 can enjoy information of a moment when the user or the robot 40 is emotional as a video.

[0057] When the switching control unit 260 is causing the processing unit 152 to generate information in the second recording format, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the second recording format to the first recording format in response to the change in the emotion of the robot 40 or the emotion of the user subsiding (the amount of change becomes equal to or less than the threshold) or the emotion of the robot 40 or the emotion of the user returning to the emotion before the change. In this way, the switching control unit 260 switches the recording format of the information recorded by the recording control unit 270 from the second recording format to the first recording format in response to the change in the emotion of the robot 40 or the emotion of the user subsiding.

[0058] Furthermore, when the recording control unit 270 switches the recording format of the information generated by the processing unit 152 from the first recording format to the second recording format, it transmits an instruction to the robot 40 to tag the information in the second recording format. This allows the user 30 to clearly recall the situation at the time from the tagged information when checking the recorded information. Furthermore, by tagging, the information can be easily found when searching for information on a specific situation from among the recorded information.

[0059] The action decision rule 282 defines an action to be performed by the robot 40 in association with the state of the robot 40. For example, the action decision rule 282 defines an action to be performed by the robot 40 in association with the internal state of an artificial neuron of a neural network. For example, the action decision rule 282 defines an action of vocalizing a word expressing happiness in association with the condition that an emotion artificial neuron defining the emotion of "happy" is high. The action decision rule 282 also defines an action to be taken when the robot becomes sleepy in association with the condition that an internal state of an endocrine artificial neuron defining an endocrine substance corresponding to sleepiness is high.

[0060] Incidentally, endocrine substances refer to substances such as neurotransmitters and hormones that are secreted in the body and transmit signals. Also, endocrine means that endocrine substances are secreted in the body. However, the endocrine substances of the robot 40 itself are one piece of information that affects the behavior of the robot 40, and do not mean that the robot 40 actually generates endocrine substances. Similarly, the emotions of the robot 40 itself are one piece of information that affects the behavior of the robot 40, and do not mean that the robot 40 actually has emotions.

[0061] The action determination unit 250 determines the action of the robot 40 based on the action defined in the action determination rules 282 in association with the activation state or internal state of each artificial neuron determined by the parameter processing unit 240. Action information indicating the action determined by the action determination unit 250 is transmitted from the communication unit 208 to the robot 40. In the robot 40, the processing unit 152 controls the control target 155 to cause the robot 40 to perform the action indicated by the information received from the server 200. This enables the robot 40 to perform an appropriate action according to the current emotion of the robot 40.

[0062] 3 is a schematic diagram of a neural network 300. The neural network 300 is an example neural network for explaining the operation of the parameter processing unit 240. The neural network 300 includes a plurality of artificial neurons including artificial neuron 1, artificial neuron 2, artificial neuron 3, artificial neuron 4, artificial neuron 5, artificial neuron 6, artificial neuron 7, artificial neuron 8, artificial neuron 9, artificial neuron a, artificial neuron b, and artificial neuron c. The neural network 300 includes a plurality of artificial synapses including artificial synapse 301, artificial synapse 302, artificial synapse 303, artificial synapse 304, artificial synapse 305, artificial synapse 306, artificial synapse 307, artificial synapse 308, artificial synapse 309, artificial synapse 310, artificial synapse 311, artificial synapse 312, artificial synapse 313, artificial synapse 314, artificial synapse 315, artificial synapse 316, artificial synapse 317, artificial synapse 318, and artificial synapse 319. The artificial neuron corresponds to a neuron in a living organism. The artificial synapse corresponds to a synapse in a living organism.

[0063] The artificial synapse 301 connects the artificial neuron 4 and the artificial neuron 1. The artificial synapse 301 is an artificial synapse that connects in one direction, as shown by the arrow of the artificial synapse 301. The artificial neuron 4 is an artificial neuron that is connected to the input of the artificial neuron 1. The artificial synapse 302 connects the artificial neuron 1 and the artificial neuron 2. The artificial synapse 302 is an artificial synapse that connects in two directions, as shown by the arrows on both ends of the artificial synapse 302. The artificial neuron 1 is an artificial neuron that is connected to the input of the artificial neuron 2. The artificial neuron 2 is an artificial neuron that is connected to the input of the artificial neuron 1.

[0064] In this embodiment, an artificial neuron may be represented by N, and an artificial synapse may be represented by S. When identifying each artificial neuron, a superscripted reference symbol may be used as an identification letter. When identifying an arbitrary artificial neuron, i or j may be used as an identification letter. For example, N i represents an arbitrary artificial neuron.

[0065] In addition, an artificial synapse may be identified by the identification numbers i and j of the two artificial neurons connected to the artificial synapse. For example, S 41 is N 1 and N 4 It represents an artificial synapse that connects ij is N i The output of N j represents an artificial synapse that inputs to S ji is N j The output of N i represents an artificial synapse input to

[0066] In FIG. 3, A to J indicate the definition of the state of the robot 40. The state of the robot 40 includes the emotion of the robot 40, the state of production of endocrine substances, the situation of the robot 40, etc. 4 , N 6 , and N 7 is a conceptual artificial neuron in which a concept that represents the situation of the robot 40 is defined. For example, N 4 is a conceptual artificial neuron that has been assigned the situation "the bell rang". 6 is a conceptual artificial neuron that has been assigned the state "charging has started". 7 is a conceptual artificial neuron to which the state “charge level is below the threshold” is assigned.

[0067] N 1 , N 3 , N b and N c is an emotion artificial neuron in which the emotion of the robot 40 is defined. 1 is an emotional artificial neuron to which the emotion "happy" is assigned.3 is an emotional artificial neuron to which the emotion "sad" is assigned. b is an emotional artificial neuron to which the emotion "fear" is assigned. c is an emotional artificial neuron to which the emotion “fun” is assigned.

[0068] N 2 , N 5 and N a is an endocrine artificial neuron with a defined endocrine state of the robot 40. 5 is an endocrine artificial neuron that is assigned the developmental state of dopamine. Dopamine is an example of an endocrine substance involved in the reward system. 5 is an example of an endocrine artificial neuron involved in the reward system. 2 is an endocrine artificial neuron that is assigned the generation state of serotonin. Serotonin is an example of an endocrine substance involved in the sleep system. 2 is an example of an endocrine artificial neuron involved in the sleep system. a is an endocrine artificial neuron that has been assigned the developmental state of noradrenaline. Noradrenaline is an example of an endocrine substance involved in the sympathetic nervous system. a is an endocrine artificial neuron involved in the sympathetic nervous system.

[0069] The definition information 284 in the storage unit 280 stores information defining the state of the robot 40 as described above for each of the multiple artificial neurons that make up the neural network. Thus, the neural network 300 includes conceptual artificial neurons, emotional artificial neurons, and endocrine artificial neurons. Conceptual artificial neurons, emotional artificial neurons, and endocrine artificial neurons are artificial neurons in which the meanings of concepts, emotions, endocrine, etc. are explicitly defined. In contrast, N 8 Or N 9 is an artificial neuron with no defined state for the robot 40. 8 Or N 9is an artificial neuron whose meanings, such as concepts, emotions, and endocrine functions, are not explicitly defined.

[0070] The parameters of the neural network 300 are i I, which is the input to t i and N from outside the neural network i E, which is the input to t i And, N i The parameters of and S i and parameters.

[0071] N i The parameters of N i S stands for status t i And, N i represents the internal state of the artificial neuron, V i m t And, N i T represents the threshold for firing i t And, N i The time when the last firing occurred is t f and the artificial neuron N at the final firing time i V represents the internal state of i m tf and the output increase / decrease parameter a t i , b t i , h t i The output increase / decrease parameter is an example of a parameter that determines the time evolution of the output when the artificial neuron fires. In this embodiment, the subscript t indicates that the parameter can be updated as time progresses. i m t is information corresponding to the membrane potential of the artificial neuron, and is an example of a parameter representing the internal state or output of the artificial neuron.

[0072] S ij The parameters of S ijBS represents the coupling coefficient of the artificial synapse t ij And, S. ij N connected i and N j The time when the two last fired simultaneously is t cf and BS, which represents the coupling coefficient at the time of the last simultaneous firing. ij tcf and a, which is the coupling coefficient increase / decrease parameter t ij , b t ij , h t ij The increase / decrease parameter of the coupling coefficient is an example of a parameter that determines the time evolution of the coupling coefficient after the two artificial neurons connected by the artificial synapse last fired simultaneously.

[0073] The parameter processing unit 240 updates the above-mentioned parameters based on the input from the external input data generating unit 230 and the neural network to determine the activation state of each artificial neuron. The operation determining unit 250 determines the operation of the robot 40 based on the internal state or activation state of at least some of the artificial neurons determined by the parameter values ​​of at least some of the artificial neurons among the multiple artificial neurons in the neural network, and the state defined for at least some of the artificial neurons by the definition information 284. Note that the activation state may be an activated state or an inactivated state. In this embodiment, activation may be called "firing", and inactivation may be called "not firing". Note that, as described later, the "firing" state is divided into an "upward phase" and a "downward phase" depending on whether the internal state is rising or not. "Not firing", "upward phase", and "downward phase" are represented by the status Sti.

[0074] Figure 4 shows the parameters of the neural network in a schematic table format. Each neuron N is assigned a threshold T t and the increase / decrease parameter h t , a t and b tEach artificial synapse has a coupling coefficient BS t and the increase / decrease parameter h t , a t and b t The parameters are as follows: i Every time, an artificial synapse is used, i In each row, the parameters of all the artificial neurons directly connected to the corresponding artificial synapse are shown, as well as the parameters of the corresponding artificial synapse.

[0075] 5 shows an outline of the operation flow of the server 200 when the robot 40 is started or reset. When the server 200 receives a notification that the robot 40 has been started or reset, the parameter processing unit 240 performs initial setting of the parameters of the neural network. For example, the parameter processing unit 240 acquires the initial values ​​of the parameters from the storage unit 280, and generates parameter data of the neural network in a predetermined data structure (S502). In addition, at time t 0 When the initial setting is completed, a loop for time t is started in S504.

[0076] In S510, the parameter processing unit 240 calculates the time step t n+1 Calculate the parameters corresponding to the changes due to the electrical influence of the artificial synapse in . t ij Calculate.

[0077] In S520, the parameter processing unit 240 calculates the time step t n+1 Calculate the parameters corresponding to the changes due to the chemical effects of endocrine substances in N i and S ij More specifically, we calculate the change in the parameters of n+1 Artificial neuron N affected by endocrine artificial neuron i The internal state of the Sij Calculate the coupling coefficient increase / decrease parameters and coupling coefficients.

[0078] In S530, the parameter processing unit 240 acquires an input from outside the neural network. Specifically, the parameter processing unit 240 acquires the output of the external input data generating unit 230.

[0079] In S540, the parameter processing unit 240 calculates the time step t n+1 In N i Calculate the internal state of V i m tn+1 and Status S tt i Then, in S550, at time t n+1 The values ​​of the parameters at time t are stored in the parameter 288 of the storage unit 280. n+1 The value of each parameter in is output to the operation determination section 250 and the switching control section 260.

[0080] In S560, the switching control unit 260 n+1 N in i At time step t n+1 N in i If the parameters satisfy the switching conditions for the recording format, the switching control unit 260 instructs the robot 40 to switch the recording format (S570). Next, when switching from the first recording format to the second recording format, the recording control unit 270 instructs tagging of information in the second recording format (S580) and proceeds to S506. On the other hand, in S560, n+1 N in i If the parameters do not satisfy the conditions for switching the recording format, the process proceeds to S506.

[0081] In S506, the parameter processing unit 240 judges whether or not to end the loop. For example, it judges to end the loop when the time represented by the time step reaches a predetermined time or when sensor information from the robot 40 is not received for a predetermined time. If the loop is not to be ended, the process returns to S510 and the calculation of the next time step is further performed. If the loop is to be ended, this flow is ended.

[0082] FIG. 6 is a diagram for explaining the calculation of the coupling coefficient of the artificial synapse. Here, the constant a is used as the initial value of the increase / decrease parameter. ij and b ij This section describes the cases where is defined.

[0083] time t n At the time step of S ij N on both ends of i and N j are firing, the parameter processing unit 240 n+1 in B.S. tn+1 ij ,BS tn+1 ij =BS tn ij +a tn ij ×(t n+1 -t n ) is calculated. On the other hand, at time t n At time steps S i and S j If none of them fires at time t n+1 Coupling coefficient BS tn+1 ij ,BS tn+1 ij =BS tn ij +b tn ij ×(t n+1 -t n ) is calculated. Also, BS tn+1 ij If is negative, BS tn+1 ij is set to 0. Note that BS ij is a positive value of Sij So, a t ij is a positive value, and b t ij is a negative value. ij S is negative ij So, a t ij is a positive value, and b t ij is a negative value.

[0084] As shown in FIG. 6, at time t 0 Since the artificial neurons at both ends fire simultaneously, BS t ij is a per unit time t0 ij Also, at time t 1 Since they did not fire simultaneously, BS t ij is |b per unit time t1 ij Also, at time t 4 Due to simultaneous fires at BS t ij is a per unit time t4 ij increases with.

[0085] Figure 7 shows the function h t ij The time evolution of the coupling coefficient when is defined is shown below. t ij t cf The time elapsed from Δt (= tt cf ) ≧0. t ij is a function of at least Δt and takes real values.

[0086] The function 700 shown in FIG. t ij The function 700 is an example of cf Coupling coefficient BS tcf ijand Δt. The function 700 monotonically increases when Δt is less than a predetermined value, and monotonically decreases and gradually decreases toward 0 when Δt is greater than the predetermined value. The function 700 has a value BS at Δt=0. tcf ij Take.

[0087] FIG. 7 shows a function 700 defined as a parameter for increasing or decreasing the coupling coefficient at time t 0 At both ends of N i and N j The parameter processor 240 calculates the coupling coefficient at time t based on the function 700 and Δt. 1 ~time t 6 BS at each time t ij Calculate the time t 1 ~time t 6 Within the time range of i and N j do not fire simultaneously. For example, at time t 2 Thereafter, the coupling coefficient decreases monotonically.

[0088] Figure 8 shows the time t 2 N i and N j The time evolution of the coupling coefficient when the neurons fire simultaneously is shown in Fig. The coupling coefficient is expressed as 0 From time t 2 The calculation is the same as in Fig. 7. 2 In N i and N j When these two fire simultaneously, the parameter processor 240 t ij (tt 2 ,B.S. t2 ij ) at time t 3 ~t 6The coupling coefficient is calculated for each time. In this way, the coupling coefficient increases each time simultaneous firing is repeated. This has the effect of strengthening the artificial synaptic connection, as in Hebb's law in living organisms. On the other hand, as shown in Figures 6 and 7, if the time without simultaneous firing becomes long, the artificial synaptic connection is weakened.

[0089] Fig. 9 shows a schematic diagram of effect definition information that defines a chemical effect given to a parameter. This effect definition information is used to calculate a change in the parameter in S520 of Fig. 5. The definition information includes a condition regarding the internal state of the endocrine artificial neuron, information specifying the artificial neuron or artificial synapse that exerts an effect, and an equation that determines the effect content.

[0090] In the example of FIG. 9, the endocrine artificial neuron N 2 is an endocrine artificial neuron to which the endocrine substance responsible for sleepiness is assigned. Endocrine artificial neuron N 2 For definition information about Vm tn 2 >T tn 2 "Conditions for Endocrine Artificial Neuron N 2 The artificial neuron N 1 and N 3 ", and "T tn+1 i =T tn i Parameter processing unit 240 determines Vm tn 2 T tn 2 If it exceeds t n+1 Emotional Artificial Neuron N 1 and N 3 The threshold is increased by 10%. This makes it difficult for the emotional artificial neuron to fire when drowsiness occurs. For example, the conceptual artificial neuron N 7 The output of the endocrine artificial neuron N 2By defining a neural network connected to the input of the above, it is possible to embody the phenomenon in which emotions become less likely to increase as the battery level becomes low.

[0091] In addition, the endocrine artificial neuron N 5 is an endocrine artificial neuron to which dopamine is assigned. Endocrine artificial neuron N 5 The first definition information for "Vm tn 5 >T tn 5 and Vm tn 4 >T tn 4 "Conditions for Endocrine Artificial Neuron N 5 As an artificial synapse that is influenced by 49 and S 95 ", and "a" is the formula that determines the effect. tn+1 ij =a tn ij The formula "Vm × 1.1" is defined. tn 5 T tn 5 Exceeds Vm tn 4 T tn 4 If it exceeds t n+1 Artificial Synapse S 49 and S 95 Increases the increase / decrease parameters by 10%.

[0092] This allows the concept of artificial neuron N to be defined as the situation in which the reward system's endocrine artificial neuron fires, resulting in the "bell ringing" condition. 4 is firing, the implicit artificial neuron N 9 Conceptual artificial neuron N via 4 and N 5 This allows the binding of the artificial neuron N to the reward system when the "bell" is rung. 5 becomes more likely to ignite.

[0093] In addition, the endocrine artificial neuron N5 The second definition of "Vm tn 5 >T tn 5 "Conditions for Endocrine Artificial Neuron N 5 The artificial neuron that is influenced by 1 ", and "T tn+1 i =T tn i The formula "Vm × 1.1" is defined. tn 5 T tn 5 If it exceeds t n+1 Artificial neuron N 1 This reduces the increase / decrease parameter of the reward system endocrine artificial neuron N 5 When this emotion is ignited, it becomes easier for the emotion of happiness to be ignited.

[0094] According to the definition of the effect of such a reward system on endocrine artificial neurons, it is possible to implement an implementation in which, if the action of charging the robot 40 while ringing a bell is repeated, the robot 40 will behave in a way that expresses happiness just by ringing the bell.

[0095] The influence definition information is not limited to the example of FIG. 9. For example, a condition may be defined that the internal state of the artificial neuron is equal to or lower than a threshold. A condition regarding the status of the artificial neuron, for example, a condition regarding an ascending phase, a descending phase, or non-firing, may be defined. The influence range may be defined by directly specifying an artificial neuron or an artificial synapse, or by defining "all artificial synapses connected to a specific artificial neuron." Regarding the influence formula, in the case where the target is an artificial neuron, in addition to multiplying the threshold by a constant, a formula may be defined in which a constant is added to the threshold or an increase / decrease parameter of the internal state is multiplied by a constant. In the case where the target is an artificial synapse, in addition to multiplying the increase / decrease parameter by a constant, an formula may be defined in which a coupling coefficient is multiplied by a constant.

[0096] The influence definition information is stored in the definition information 284 of the storage unit 280. Thus, the storage unit 280 stores influence definition information that defines the influence of at least one of the internal state and firing state of the endocrine artificial neuron on the parameters of at least one of the other artificial neurons and artificial synapses that are not directly connected to the endocrine artificial neuron via an artificial synapse. Then, the parameter processing unit 240 updates the parameters of at least one of the other artificial neurons and artificial synapses that are not directly connected to the endocrine artificial neuron via an artificial synapse based on at least one of the internal state and firing state of the endocrine artificial neuron and the influence definition information. Furthermore, the parameters of the other artificial neurons that are influenced by at least one of the internal state and firing state of the endocrine artificial neuron can include at least one of the parameters that define the threshold value, firing state, and time development of the output at the time of firing of the other artificial neuron. Furthermore, the parameters of the artificial synapse influenced by at least one of the internal state and the firing state of the endocrine artificial neuron can include at least one of the coupling coefficient of the artificial synapse and a parameter that determines the time evolution of the coupling coefficient after the two artificial neurons linked by the artificial synapse last fired simultaneously. Furthermore, the influence definition information includes information that determines the influence that the firing state of the endocrine artificial neuron associated with the reward system has on the threshold of the emotional artificial neuron, and the parameter processing unit 240 updates the threshold of the emotional artificial neuron in accordance with the influence definition information when the endocrine artificial neuron fires.

[0097] Figure 10 shows the V tn+1 i and S tn+1 i 5. The process of this flowchart can be applied to a part of the process in S540 in FIG. 5. In S1100, the parameter processing unit 240 calculates S tn i indicates non-firing.

[0098] S tn i If indicates non-firing, the parameter processing unit 240 i Input to Itn+1 i Specifically, when the input from outside the neural network is N i If not connected to I tn+1 i =Σ j B.S. tn+1 ji ×VM tn j ×f(S tn j ) is calculated by the following formula: i If connected to I tn+1 i =Σ j B.S. tn+1 ji × VM tn j ×f(S tn j )+E tn+1 i Here, E tn i is the time t from outside the neural network. n This is the input in

[0099] In addition, f(S) returns 0 when S is a value that indicates no firing, and returns 1 when S is a value that indicates an ascending or descending phase. This model corresponds to a model in which a synapse transmits an action potential only when a neuron fires. Note that f(S)=1 may also be returned. This corresponds to a model in which a membrane potential is transmitted regardless of the firing state of the neuron.

[0100] In S1112, the parameter processing unit 240 tn+1 i T tn+1 i Determine whether it exceeds I tn+1 i T tn+1 i If it exceeds Vm tn+1 i is calculated based on the increase / decrease parameters, and Vm tn+1 i Depending on S tn+1i is set to a value indicating the rising phase or the falling phase (S1114), and this flow ends.

[0101] In the S1100, tn i When Vm is in the rising phase or the falling phase, the parameter processing unit 240 tn+1 i Then, the parameter processing unit 240 calculates t n+1 By Vm t i When Vmin is reached, S tn+1 i Set to the unfired value and t n+1 By Vm t i If Vmin is not reached, S tn+1 i The parameter processing unit 240 sets t n+i By Vm t i When Vmax is reached, S tn+1 i Set the value of the falling phase to t n+1 By Vm t i If Vmax is not reached, S tn+1 i Set the value of the rising phase to

[0102] In this way, N i If is firing, even if the output falls below the threshold, N i The output of is independent of the input. Such a period corresponds to the absolute refractory period in biological neurons.

[0103] Figure 11 shows the N i V when does not ignite t i FIG. 13 is a diagram for explaining an example of calculation of

[0104] time t 0 At time steps of N i is not fired. 1 It1 i T t1 i If the time t 1 V in t1 i , V t1 i =I t1 i At time t 0 From 1 V for the period up to t i , V t i =I t0 i Similarly, the parameter processing unit 240 calculates the time step t n The calculated V t n maintains the value of until the next time step, and V tn+1 In I tn+1 Change to.

[0105] Figure 12 shows the N i V when ignition occurs i t FIG. 12 is a diagram for explaining an example of calculation of the constant a i and b i This is a calculation example when is defined.

[0106] time t 0 At time steps, N i is not fired. 1 I t1 i T t1 i If the time t 1 V in t1 i , V t1 i =I t1 i At time t 0 From 1 V for the period up to t i , V t i =It0 i Here, the calculation is performed at time t 1 I t1 i is less than or equal to Vmax. 1 I t1 i If exceeds Vmax, I t1 i =Vmax.

[0107] As shown in FIG. 12, the parameter processing unit 240 1 From then on, V t i Until the time when V reaches Vmax, t i per unit time t ij In addition, the parameter processing unit 240 increases N i Status S t i is determined to be the ascending phase.

[0108] Also, V t i When V reaches Vmax, V t i Until Vmin is reached, V t i per unit time |b t i In addition, the parameter processing unit 240 reduces N i The status of V is determined to be in the descending phase. t i When Vmin is reached, V at the next time t6 i , V t6 i =I t6 i Calculate by V t i After reaching Vmin, the status is determined as non-firing.

[0109] In addition, N i If the status of is in the declining phase, the calculated Vm ti T t i Even if it falls below Vm t i I t i The parameter processing unit 240 does not depend on Vm t i T t i Even if it falls below Vm t i Vm increases or decreases according to the parameter until it reaches Vmin. t i Calculate.

[0110] Figure 13 shows the N i As a parameter of the function h t i The time evolution of the coupling coefficient when is defined is shown below. In general, h t i is the firing time t f The time elapsed from Δt (= tt f ) ≧0. t i is at least a function of Δt. t i takes real values, and h t i The value range is from Vmin to Vmax.

[0111] The function 1300 shown in FIG. t i The function 1300 is an example of f In tf i and Δt. The function 1300 increases monotonically when Δt is less than a predetermined value, and decreases monotonically when Δt is greater than a predetermined value. The function 1300 has a value Vm tf i Take.

[0112] FIG. 13 shows a function 1400 defined as an internal state increase / decrease parameter, and the time t 1 In N iThe parameter processing unit 240 calculates the output when the function 1400, Δt and Vm f i Based on the time t 1 ~ Calculate Vmti at each time t5. Vm t i is time t 5 Since Vmin is reached at time t 6 So, Vm i =I t6 i It becomes.

[0113] FIG. 14 shows an example of a rule 1400 stored in the recording format switching rule 290 in the form of a table. The rule 1400 is 1 , N 3 , N b and N c When at least the first condition is satisfied that any one of the Vmti exceeds a threshold, the information recording format is "switched to a low compression format". In this way, when information is recorded in a high compression format, the switching control unit 260 determines to switch the information recording format to a low compression format when the first condition is satisfied after not being satisfied. Note that the thresholds are set to N j The threshold value may be higher than Tit.

[0114] Also, rule 1400 states that 5 and N a VM t i If the second condition is met, that is, the total value of the data exceeds the threshold, the data recording format is switched to a low compression format. " is set. In this way, when information is recorded in a high compression format, the switching control unit 260 determines to switch the information recording format to a low compression format when the second condition is satisfied from a state in which the second condition is not satisfied. j The total value of Vmax of each is multiplied by a constant 0.9. j Ti t may be higher than the sum of the above.

[0115] N 1 , N 3 , N b and N c are emotion artificial neurons in which emotions "happy", "sad", "fearful" and "fun" are defined, respectively. Therefore, the parameter processing unit 240 determines the intensity of the emotion based on the internal state of the emotion artificial neuron, and when the determined intensity of the emotion exceeds a predetermined threshold, the recording format can be switched to a low compression format.

[0116] N5 and Na are endocrine artificial neurons in which the endocrine substances "dopamine" and "noradrenaline" are defined, respectively. The sum of the parameters of the internal states of these endocrine artificial neurons is an example of an index representing the intensity of the emotion of "excitement." Therefore, the parameter processing unit 240 determines the intensity of the emotion based on the internal states of the endocrine artificial neurons, and when the determined intensity of the emotion exceeds a predetermined threshold, the recording format can be switched to a low compression format.

[0117] Also, rule 1400 states that 1 , N 3 , N b and N c Any VM t i is also equal to or smaller than the first threshold, and N 5 and N a VM t i When a third condition is satisfied, that is, the sum of the first and second thresholds is equal to or less than the second threshold, the operation of "switching the data recording format to a high compression format" is defined. Therefore, when information is recorded in a low compression format, the switching control unit 260 determines to switch the information recording format to a high compression format when the third condition is satisfied from a state in which the third condition is not satisfied. In this way, the recording format can be switched to a high compression format in response to the intensity of the emotion becoming equal to or less than the predetermined threshold.

[0118] The first threshold of the third condition is j The second threshold value of the third condition is the value obtained by multiplying the Vmax of each N j The value is obtained by multiplying the total value of Vmax of the three conditions by the constant 0.8. In this manner, the case is exemplified in which the first threshold value of the third condition is smaller than the threshold value of the first condition, and the second threshold value of the third condition is smaller than the threshold value of the second condition. However, the first threshold value may be the same as the threshold value of the first condition, and the second threshold value may be the same as the threshold value of the second condition. Also, the first threshold value of the third condition is set to a value obtained by multiplying the total value of Vmax of the three conditions by the constant 0.8. j T i t The second threshold of the third condition may be higher than each N j T i t In addition, the threshold value for each condition is not limited to these examples, and various values ​​can be applied.

[0119] According to the system 20, the robot 40 continuously transmits highly compressed information such as skeletal data to the server 200 during a period when the emotion of the robot 40 is not significantly high, and the server 200 records the information. The continuous information such as skeletal data recorded in the server 200 can be used when analyzing the memory of the robot 40. When the emotion of the robot 40 significantly increases, the robot 40 starts transmitting full HD video data and audio data, and during a period when the emotion is strong enough to exceed a certain value, the server 200 records low-compression information including the full HD video data and audio data in addition to the skeletal data. When the robot 40 is requested by the user 30 to provide a video of the robot 40's memories, the robot 40 requests the server 200 to transmit the full HD video data and audio data, and provides the user 30 with the video data and audio data received from the server 200.

[0120] In this way, according to the system 20, high-quality video data of scenes in which the robot 40 feels strong emotions can be stored in the server 200. On the other hand, when the robot 40 does not feel strong emotions, simplified information such as skeletal data can be stored in the server 200. In this way, the robot 40, like a human being, can retain clear memories of times when it felt strong emotions, while retaining simplified memories of times when it did not feel strong emotions.

[0121] In this embodiment, "happy", "sad", "fearful", "fun" and "excited" are described as emotions, but the emotions handled by the system 20 are not limited to these. In addition, in this embodiment, "dopamine", "serotonin" and "noradrenaline" are described as endocrine substances, but the endocrine substances handled by the system 20 are not limited to these.

[0122] The functions of the server 200 may be implemented by one or more computers. At least some of the functions of the server 200 may be implemented by a virtual machine. At least some of the functions of the server 200 may be implemented in a cloud. Among the functions of the server 200, the functions of the components other than the storage unit 280 can be realized by a CPU operating based on a program. For example, at least some of the processing described as the operation of the server 200 can be realized by a processor controlling each piece of hardware (e.g., a hard disk, a memory, etc.) possessed by the computer according to a program. In this way, at least some of the processing of the server 200 can be realized by the processor operating according to a program to control each piece of hardware, so that each piece of hardware including the processor, the hard disk, the memory, etc. and the program operate in cooperation with each other. That is, the program can make the computer function as each component of the server 200. Similarly, among the components of the robot 40, the functions of the components other than the control target 155 and the sensor unit 156 can be realized by a CPU operating based on a program. That is, the program can make the computer function as each component of the robot 40. The computer may load a program that controls the execution of the above-mentioned processing, and operate according to the loaded program to execute the processing. The computer can read the program from a computer-readable recording medium storing the program. Alternatively, the program may be supplied to the computer via a communication line, and the computer may read the program supplied via the communication line.

[0123] In the embodiment described above, the server 200 separate from the robot 40 is responsible for processing the neural network. Also, the server 200 separate from the robot 40 stores information such as video data. However, the robot 40 itself may perform the functions of the server 200, such as processing the neural network. Also, the robot 40 itself may store information such as video data. Also, the robot 40 is an example of a device to be controlled by the server 200. The device to be controlled is not limited to the robot 40, and various devices such as home appliances, vehicles, toys, etc. can be applied as the controlled devices.

[0124] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0125] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order. [Explanation of symbols]

[0126] 20 Systems 30 users 40 Robot 90 Communication Network 152 Processing section 155 Control Object 156 Sensor section 158 Communications Department 161 Mike 162 3D Depth Sensor 163 2D Camera 164 Distance Sensor 200 Servers 202 Processing section 208 Communications Department 210 Initial value setting section 230 External input data generation unit 240 Parameter Processing Section 250 Action Decision Unit 260 Switching control section 270 Recording control section 280 Storage Unit 282 Action Decision Rules 284 Definition information 286 Parameter Initial Values 288 Parameters 290 Recording format switching rules 292 Recorded Data 300 Neural Networks

Claims

1. A control system for controlling the behavior of a robot, comprising: a recording control unit that processes at least a part of the information continuously detected by the sensor, and generates and records information in a first recording format or information in a second recording format having a larger amount of information than the information in the first recording format; a switching control unit that switches a recording format by the recording control unit from the first recording format to the second recording format when an emotion of the robot or a user exceeds a threshold value while generating information in the first recording format; A control system comprising:

2. The switching control unit is When the emotion of the user exceeding the threshold is transmitted to the robot, the recording format is switched from the first recording format to the second recording format. The control system of claim 1 .

3. The switching control unit is When the emotion of the robot exceeding the threshold is transmitted to the user, the recording format is switched from the first recording format to the second recording format. The control system of claim 1 .

4. The recording control unit A trigger for switching from the first recording format to the second recording format is switched depending on the situation. The control system of claim 1 .

5. The recording control unit The information in the second recording format is tagged and recorded. The control system of claim 1 .

6. The tagged information is Includes at least one of the following: surrounding information, smell, temperature, humidity, location information, weather, number of people around, facial expressions, and emotions The control system of claim 5.

7. The video generation unit receives information in the second recording format including video data from a server, and generates a video to be presented to the user based on the video data included in the received information in the second recording format. The control system of claim 1 .

8. A control system for controlling the behavior of a robot, comprising: a recording control unit that processes at least a part of the information continuously detected by the sensor, and generates and records information in a first recording format or information in a second recording format having a larger amount of information than the information in the first recording format; a switching control unit that switches a recording format by the recording control unit from the first recording format to the second recording format when an emotion of the robot or a user exceeds a threshold while generating information in the first recording format; a server that receives and records information in the first recording format or the second recording format; A system comprising:

9. A program for controlling the behavior of a robot, a recording control step of processing at least a part of information continuously detected by the sensor to generate and record information in a first recording format or information in a second recording format having a larger amount of information than the information in the first recording format; a switching control procedure for switching a recording format according to the recording control procedure from the first recording format to the second recording format when an emotion of the robot or a user exceeds a threshold value while generating information in the first recording format; A program that causes a computer to execute the following.

Images