Control system, method for control, and control program

Abstract

To properly control behaviors of a robot when recording data according to emotions.SOLUTION: A control system according to an embodiment is a control system for controlling operations of a robot and includes a recording control unit and an operation determination unit. The recording control unit processes at least a part of information detected continuously by a sensor to generate information, and records the generated information. The operation determination unit determines an operation of the robot to control an emotion value on the basis of an emotion value expressing emotions of the robot or information detected by the sensor.SELECTED DRAWING: Figure 17

Description

[Technical field]

[0001] The present invention relates to a control system, a control method, and a control program. [Background technology]

[0002] A terminal is known that learns a conversation between a user and a call partner and stores the call partner's replies to questions from the user in a reply table (see, for example, Patent Document 1). Also, an emotion generation device is known that includes a neural network that inputs user information, device information, and its own current emotional state and outputs the next emotional state (see, for example, Patent Document 2). Also, a technology is known that stores a spatiotemporal pattern in an associative memory that includes multiple electronic neurons having a layer-neural network relationship with directional artificial synaptic connectivity (see, for example, Patent Document 3). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-253389 A [Patent Document 2] Japanese Patent Application Publication No. 10-254592 [Patent Document 3] Special Publication No. 2013-535067 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, there is a problem in that the behavior of the robot when recording data may not be appropriately controlled in accordance with the robot's emotions.

[0005] The present invention has been made in consideration of the above, and aims to provide a control system, a system, and a program that can appropriately control a robot's behavior when recording data in accordance with its emotions. [Means for solving the problem]

[0006] A control system according to one aspect of an embodiment is a control system for controlling the behavior of a robot, and includes a recording control unit that processes at least a portion of information continuously detected by a sensor to generate information and records the generated information, and an action determination unit that determines an action of the robot to control an emotion value based on an emotion value representing the emotion of the robot or the information detected by the sensor. Effect of the Invention

[0007] According to one aspect of the embodiment, the behavior of the robot when recording data can be appropriately controlled according to the emotion. [Brief description of the drawings]

[0008] [Figure 1A] FIG. 1A is a schematic diagram illustrating an example of a system 20 according to the present embodiment. [Figure 1B] FIG. 1B is a diagram showing an example of emotion value changes. [Diagram 2] FIG. 2 is a diagram showing a schematic block configuration of the server 200 and the robot 40. As shown in FIG. [Diagram 3] FIG. 3 is a schematic diagram of a neural network 300. [Figure 4] FIG. 4 is a diagram showing a schematic diagram of the parameters of the neural network in a table format. [Diagram 5] FIG. 5 is a diagram illustrating an outline of the operation flow of the server 200 when the robot 40 is started or reset. [Figure 6] FIG. 6 is a diagram for explaining the outline of the calculation of the connection weights of the artificial synapses. [Figure 7] FIG. 7 is a diagram showing an outline of the time evolution of the coupling coefficient when a function ht ij is defined as an increase / decrease parameter of the coupling coefficient. [Figure 8] FIG. 8 is a diagram showing a schematic diagram of the time evolution of the coupling coefficient when further simultaneous firing occurs at time t2. [Figure 9]FIG. 9 is a diagram illustrating an outline of effect definition information that defines a chemical effect applied to a parameter. [Figure 10] FIG. 10 is a flowchart showing the calculation of the internal state and status. [Figure 11] FIG. 11 is a diagram for explaining an example of calculation of the internal state when the artificial neuron does not fire. [Figure 12] FIG. 12 is a diagram for explaining an example of calculation of an output when an artificial neuron fires. [Figure 13] FIG. 13 is a diagram showing a schematic diagram of the time evolution of the coupling coefficient when a function is defined as an increase / decrease parameter of the artificial neuron. [Figure 14] FIG. 14 is a diagram showing an example of rules 1400 stored in the recording format switching rule 290 in table form. [Figure 15] FIG. 15 is a diagram illustrating an example of the condition table. [Figure 16] FIG. 16 is a diagram illustrating an example of the operation table. [Figure 17] FIG. 17 is a flowchart showing the flow of a process for determining an operation at the timing of recording information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0010] Fig. 1A shows an example of a system 20 according to the present embodiment. Fig. 1B shows an example of an emotional value change. The system 20 includes a server 200, and a robot 40a and a robot 40b. The robots 40a and 40b communicate with the server 200 via a communication network 90 to exchange information.

[0011] The user 30a is a user of the robot 40a. The user 30b is a user of the robot 40b. The robot 40b has substantially the same functions as the robot 40a. Therefore, the system 20 will be described with the robots 40a and 40b collectively referred to as the robot 40.

[0012] The robot 40 performs various actions according to the situation, such as moving its head or limbs, conversing with the user 30, and providing an image to the user 30. At this time, the robot 40 decides the action in cooperation with the server 200. For example, the robot 40 transmits to the server 200 detection information such as a facial image of the user 30 acquired by a camera function and a voice of the user 30 acquired by a microphone function. The server 200 analyzes the detection information received from the robot 40, decides the action to be taken by the robot 40, and transmits action information representing the decided action to the robot 40. The robot 40 operates according to the action information received from the server 200.

[0013] The robot 40 has an emotion value that represents its own emotion. For example, the robot 40 has an emotion value that represents the strength of each emotion, such as "happy," "fun," "sad," "fear," and "excitement." The emotion value of the robot 40 is determined by the server 200. The server 200 causes the robot 40 to perform an action according to the determined emotion. For example, when the robot 40 has a high emotion value of excitement and is talking to the user 30, the server 200 causes the robot 40 to speak quickly. In this way, the robot 40 can express its own emotion through actions, etc.

[0014] The server 200 updates the current state of the robot 40 using a neural network based on the detection information received from the robot 40. The state of the robot 40 includes the emotion of the robot 40. Therefore, the server 200 determines the emotion of the robot 40 using the neural network.

[0015] The robot 40 also records the video data of the user 30 acquired by the camera function in the server 200. The robot 40 acquires the video data from the server 200 as necessary and provides it to the user 30. The robot 40 generates video data with a larger amount of information as the strength of its own emotion increases and records it in the server 200. For example, when the robot 40 records information in a highly compressed format such as skeletal data, the robot 40 switches to recording information in a low compression format such as HD video when the emotion value of the robot 40 exceeds a threshold. According to the system 20, high-definition video data when the emotion of the robot 40 increases can be recorded. The robot 40 also generates video data with a larger amount of information as the strength of the emotion of the user 30 increases and records it in the server 200. Specifically, when the robot 40 records information in a highly compressed format such as skeletal data, the robot 40 switches to recording information in a low compression format such as HD video when the emotion value of the user 30 exceeds a threshold. According to the system 20, high-definition video data of a moment when the user 30 is emotionally excited can be left as a record.

[0016] In addition, even if the robot 40's or the user's emotion is not strong as in state 2 of FIG. 1B, when the robot 40's or the user's emotion changes suddenly, the robot 40 generates video data with a large amount of information and records it in the server 200. For example, when the robot 40 records information in a highly compressed format such as skeletal data, the robot 40 switches to recording information in a low compression format such as HD video in response to the change in the emotion value exceeding the threshold even if the emotion value of the excitement of the robot 40 or the user is equal to or less than the threshold. According to the system 20, high-definition video data when the emotion of the robot 40 or the user 30 changes drastically can be recorded. In addition, the robot 40 may change the amount of information of the video data according to the degree of emotion change. Specifically, the robot 40 lowers the compression rate of the video data as the degree of change in the emotion of the robot 40 or the user increases. As another example, the robot 40 may change the format of the information to be recorded in the order of decreasing degree of change in the emotion of the robot 40 or the user, such as text only → image only → voice and image → low-quality video → high-quality video. Furthermore, the robot 40 may change the length or amount of video data, not just the compression rate or the format of information. This allows higher definition video data to be recorded when the emotions of the robot 40 or the user 30 change more dramatically.

[0017] Also, the robot 40 may change the information to be stored according to the continuity of the change in emotion of the robot 40 or the user 30, as in state 1 of FIG. 1B. For example, when the cumulative value of the emotion value within a predetermined time exceeds a threshold, the robot 40 generates video data with a large amount of information and records it in the server 200. Also, the robot 40 may change the information to be stored according to the combination of the speed of the change (slope) of emotion within a predetermined time or the degree of the slope. For example, the robot 40 generates video data with a large amount of information and records it in the server 200 when the slope indicating the change in the emotion value is a combination of a sudden rise → a gradual rise for a certain period of time → a sudden fall within a predetermined time. Thereby, for example, in a tense situation due to pressure or the like, video data of a situation in which the absolute value of the emotion is not relatively large but the time feels long (an emotion value above a certain value continues for a certain period of time) can be recorded.

[0018] In addition, the robot 40 tags the information to be recorded when the emotional value of itself or the robot 40 exceeds a threshold value or when a change in the emotional value exceeds a threshold value. Specifically, the robot 40 tags the video data with information detected by a sensor provided in the robot 40 and information on the emotional value of the user or itself. The tagged information includes, for example, surrounding information (information on objects present in the vicinity, etc.), smell, temperature, humidity, location information, weather, the number of people in the vicinity, their expressions and emotions, etc. This allows the user 30 to clearly recall the situation at the time from the tagged information when checking the recorded information. Furthermore, by tagging, the information can be easily found when searching for information on a specific situation from the recorded information.

[0019] 2 is a schematic block diagram of the server 200 and the robot 40. The robot 40b has a sensor unit 156, a processing unit 152, a control target 155, a communication unit 158, and a display unit 157. The server 200 has a processing unit 202, a storage unit 280, and a communication unit 208. The processing unit 202 includes an initial value setting unit 210, an external input data generating unit 230, a parameter processing unit 240, an action determining unit 250, a switching control unit 260, and a recording control unit 270. The storage unit 280 stores action determining rules 282, definition information 284, parameter initial values ​​286, the latest parameters 288, a recording format switching rule 290, and recorded data 291. Store 2.

[0020] In the robot 40, the sensor unit 156 has sensors such as a microphone 161, a 2D camera 163, a 3D depth sensor 162, and a distance sensor 164. Each sensor of the sensor unit 156 continuously detects information. The sensor information detected by the sensor unit 156 is output to the processing unit 152. The 2D camera 163 is an example of an image sensor that continuously captures an object, and captures an object with visible light to generate video information. The 3D depth sensor 162 detects the contour of the object by continuously irradiating an infrared pattern and analyzing the infrared pattern from the infrared images continuously captured by the infrared camera. In addition, the sensor unit 156 may include various other sensors such as a clock, a gyro sensor, a touch sensor, a sensor for motor feedback, a sensor for detecting remaining battery capacity, an odor sensor, a temperature sensor, a humidity sensor, a GPS (Global Positioning System) sensor, etc.

[0021] The processing unit 152 is formed of a processor such as a CPU. The processing unit 152 transmits sensor information continuously detected by each sensor of the sensor unit 156 to the server 200 through the communication unit 158. The processing unit 152 also processes at least a part of the sensor information continuously detected by each sensor of the sensor unit 156 to generate information for recording. The processing unit 152 generates information in a first recording format or information in a second recording format having a larger amount of information than the information in the first recording format. The information in the first recording format is, for example, information in a high compression format, and the information in the second recording format is, for example, information in a low compression format. For example, the processing unit 152 generates shape data such as skeleton data of an object as information in the first recording format based on skeleton information continuously detected by the 3D depth sensor 162. The processing unit 152 also generates full HD video data and audio data from video information captured by the 2D camera 163 and audio information detected by the microphone 161. The full HD video data is an example of video data having more information than the shape data of an object.

[0022] The communication unit 158 ​​transmits the information in the first recording format or the information in the second recording format generated by the processing unit 152 to the server 200. In the server 200, the recording control unit 270 stores the information in the first recording format or the information in the second recording format received by the communication unit 208 from the robot 40 in the recording data 292. The recording control unit 270 stores the information received from each robot 40 in the recording data 292 in association with information identifying each robot 40.

[0023] Furthermore, in the robot 40, the communication unit 158 ​​acquires information stored in the recording data 292 from the server 200. The communication unit 158 ​​functions as a recorded information receiving unit that acquires information in the second recording format, including video data, recorded by the recording control unit 270. The processing unit 152 generates an image to be presented to the user 30, based on the video data included in the information in the second recording format received by the communication unit 158. The processing unit 152 functions as an image generating unit that generates an image to be presented to the user 30.

[0024] Furthermore, the communication unit 158 ​​receives operation information indicating the operation content from the server 200. The processing unit 152 controls the control target 155 based on the operation content received by the communication unit 158. The control target 155 includes a speaker, motors for driving each part of the robot 40 such as the limbs, a light emitting device, and the like. When information indicating the vocalization content is received from the server 200, the processing unit 152 outputs a sound from the speaker according to the received vocalization content. Furthermore, the processing unit 152 can control a part of the behavior of the robot 40 by controlling the drive motors of the limbs. Furthermore, the processing unit 152 can express a part of the emotion of the robot 40 by controlling these motors.

[0025] In the server 200, the communication unit 208 outputs the information received from the robot 40 to the processing unit 202. The initial value setting unit 210 stores the initial values ​​of the parameters indicating the initial state of the neural network received by the communication unit 208 in the parameter initial values ​​286 in the storage unit 280. Note that the initial values ​​of the parameters of the neural network may be predetermined in the server 200, or may be changeable by the user 30 via the communication network 90.

[0026] The external input data generating unit 230 processes at least a part of the sensor information received by the communication unit 208 to generate input information from outside the neural network, and outputs the generated information to the parameter processing unit 240. The parameter processing unit 240 performs calculations on the neural network based on the input information and current parameters 288 and definition information 284 of the neural network stored in the storage unit 280.

[0027] The artificial neurons of the neural network include a plurality of artificial neurons in which the situation of the robot 40 is defined, a plurality of emotion artificial neurons in which a plurality of emotions of the robot 40 itself is defined, and a plurality of endocrine artificial neurons in which the production state of the endocrine substance of the robot 40 itself is defined. The parameter processing unit 240 calculates parameters representing the internal states of the plurality of artificial neurons in the neural network based on the input information generated by the external input data generating unit 230. For example, the parameter processing unit 240 updates the parameters of the current internal states of the plurality of artificial neurons in which the situation of the robot 40 is defined, based on the input information generated by the external input data generating unit 230. In addition, the parameter processing unit 240 calculates parameters of the internal states of other artificial neurons in the neural network. As a result, for example, the parameters of the internal state of the emotion artificial neuron in which the emotion "happy" is defined are calculated. The parameters of the internal state of this emotion artificial neuron are an example of an index representing the strength of the emotion "happy". Therefore, the parameter processing unit 240 can determine the strength of the emotion in the control system based on the internal state of the emotion artificial neuron. In this way, the parameter processing unit 240 functions as an emotion determining unit that determines the intensity of an emotion using a neural network based on at least a part of the information detected by the sensor of the sensor unit 156.

[0028] The parameters of the neural network calculated by the parameter processing unit 240 are supplied to the switching control unit 260 and the operation determination unit 250. The switching control unit 260 determines the recording format of the information generated by the processing unit 152 of the robot 40 based on the parameters supplied from the parameter processing unit 240. When it is necessary to switch the recording format of the information generated by the processing unit 152, the switching control unit 260 causes the robot 40 to transmit an instruction to switch the recording format via the communication unit 208. In the robot 40, the processing unit 152 switches the recording format in accordance with the instruction received from the server 200.

[0029] For example, when the switching control unit 260 is causing the processing unit 152 to generate information in the first recording format, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format in response to an increase in the intensity of the emotion determined by the parameter processing unit 240. In this way, the switching control unit 260 switches the recording format of the information recorded by the recording control unit 270 from the first recording format to the second recording format in response to an increase in the intensity of the emotion determined by the parameter processing unit 240. This allows the information when the emotion of the robot 40 increases to be recorded in detail. In addition, in the robot 40, the processing unit 152 acquires the video data in the second recording format acquired from the server 200 and generates a video to be presented to the user 30. Therefore, the user 30 can enjoy the information when the emotion of the robot 40 increases as a video.

[0030] In addition, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format in response to an increase in the intensity of the emotion of the user 30. In this way, the switching control unit 260 switches the recording format of the information to be recorded by the recording control unit 270 from the first recording format to the second recording format in response to an increase in the intensity of the emotion determined by the parameter processing unit 240. This allows information when the emotion of the user 30 is increased to be recorded in detail. In addition, in the robot 40, the processing unit 152 acquires video data in the second recording format acquired from the server 200 and generates a video to be presented to the user 30. Therefore, the user 30 can enjoy information when his / her emotion is increased as a video. The emotion of the user can be detected, for example, based on a voice or facial expression detected from the user 30. In addition, the switching control unit 260 may recognize a change in the emotion of the user 30 based on the detection of an impact by a touch sensor (not shown) included in the sensor unit 156. For example, when an impact is detected by a touch sensor included in the sensor unit 156, the switching control unit 260 may recognize that the user 30 has become upset.

[0031] When the switching control unit 260 is causing the processing unit 152 to generate information in the second recording format, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the second recording format to the first recording format in response to a decrease in the intensity of the emotion of the robot 40 or the emotion of the user determined by the parameter processing unit 240. In this way, the switching control unit 260 switches the recording format of the information recorded by the recording control unit 270 from the second recording format to the first recording format in response to a decrease in the intensity of the emotion of the robot 40 or the emotion of the user determined by the parameter processing unit 240.

[0032] Furthermore, in response to an increase in the emotional change of the user 30 or the robot 40, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information generated by the processing unit 152 from the first recording format to the second recording format. This allows detailed information about when the emotion of the user 30 or the robot 40 increases to be recorded. Furthermore, in the robot 40, the processing unit 152 acquires the video data in the second recording format acquired from the server 200, and generates a video to be presented to the user 30. Thus, the user 30 can enjoy information about when the emotion of the user 30 or the robot 40 increases as a video.

[0033] When the switching control unit 260 is causing the processing unit 152 to generate information in the second recording format, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the second recording format to the first recording format in response to the change in the emotion of the robot 40 or the emotion of the user subsiding (the amount of change becomes equal to or less than the threshold) or the emotion of the robot 40 or the emotion of the user returning to the emotion before the change. In this way, the switching control unit 260 switches the recording format of the information recorded by the recording control unit 270 from the second recording format to the first recording format in response to the change in the emotion of the robot 40 or the emotion of the user subsiding.

[0034] The switching control unit 260 may also switch the recording format according to the continuity of changes in the emotions of the robot 40 or the user 30. For example, when the cumulative value of the emotion value within a predetermined time exceeds a threshold, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format. The switching control unit 260 may also switch the recording format according to a combination of the speed of the change (slope) of the emotion within a predetermined time or the degree of the slope. For example, when the slope indicating the change in the emotion value is a combination of a sudden rise → a gradual change for a certain period of time → a sudden drop within a predetermined time, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format. This makes it possible to record video data of a situation in which, for example, in a tense scene due to pressure or the like, the absolute value of the emotion is not relatively large but the time feels long (an emotion value above a certain value continues for a predetermined time).

[0035] Furthermore, when the recording control unit 270 switches the recording format of the information generated by the processing unit 152 from the first recording format to the second recording format, it transmits an instruction to the robot 40 to tag the information in the second recording format. This allows the user 30 to clearly recall the situation at the time from the tagged information when checking the recorded information. Furthermore, by tagging, the information can be easily found when searching for information on a specific situation from among the recorded information.

[0036] The action decision rule 282 defines an action to be performed by the robot 40 in association with the state of the robot 40. For example, the action decision rule 282 defines an action to be performed by the robot 40 in association with the internal state of an artificial neuron of a neural network. For example, the action decision rule 282 defines an action of vocalizing a word expressing happiness in association with the condition that an emotion artificial neuron defining the emotion of "happy" is high. The action decision rule 282 also defines an action to be taken when the robot becomes sleepy in association with the condition that an internal state of an endocrine artificial neuron defining an endocrine substance corresponding to sleepiness is high.

[0037] Incidentally, endocrine substances refer to substances such as neurotransmitters and hormones that are secreted in the body and transmit signals. Also, endocrine means that endocrine substances are secreted in the body. However, the endocrine substances of the robot 40 itself are one piece of information that affects the behavior of the robot 40, and do not mean that the robot 40 actually generates endocrine substances. Similarly, the emotions of the robot 40 itself are one piece of information that affects the behavior of the robot 40, and do not mean that the robot 40 actually has emotions.

[0038] The action determination unit 250 determines the action of the robot 40 based on the action defined in the action determination rules 282 in association with the activation state or internal state of each artificial neuron determined by the parameter processing unit 240. Action information indicating the action determined by the action determination unit 250 is transmitted from the communication unit 208 to the robot 40. In the robot 40, the processing unit 152 controls the control target 155 to cause the robot 40 to perform the action indicated by the information received from the server 200. This enables the robot 40 to perform an appropriate action according to the current emotion of the robot 40.

[0039] 3 is a schematic diagram of a neural network 300. The neural network 300 is an example neural network for explaining the operation of the parameter processing unit 240. The neural network 300 includes a plurality of artificial neurons including artificial neuron 1, artificial neuron 2, artificial neuron 3, artificial neuron 4, artificial neuron 5, artificial neuron 6, artificial neuron 7, artificial neuron 8, artificial neuron 9, artificial neuron a, artificial neuron b, and artificial neuron c. The neural network 300 includes a plurality of artificial synapses including artificial synapse 301, artificial synapse 302, artificial synapse 303, artificial synapse 304, artificial synapse 305, artificial synapse 306, artificial synapse 307, artificial synapse 308, artificial synapse 309, artificial synapse 310, artificial synapse 311, artificial synapse 312, artificial synapse 313, artificial synapse 314, artificial synapse 315, artificial synapse 316, artificial synapse 317, artificial synapse 318, and artificial synapse 319. The artificial neuron corresponds to a neuron in a living organism. The artificial synapse corresponds to a synapse in a living organism.

[0040] The artificial synapse 301 connects the artificial neuron 4 and the artificial neuron 1. The artificial synapse 301 is an artificial synapse that connects in one direction, as shown by the arrow of the artificial synapse 301. The artificial neuron 4 is an artificial neuron that is connected to the input of the artificial neuron 1. The artificial synapse 302 connects the artificial neuron 1 and the artificial neuron 2. The artificial synapse 302 is an artificial synapse that connects in two directions, as shown by the arrows on both ends of the artificial synapse 302. The artificial neuron 1 is an artificial neuron that is connected to the input of the artificial neuron 2. The artificial neuron 2 is an artificial neuron that is connected to the input of the artificial neuron 1.

[0041] In this embodiment, an artificial neuron may be represented by N, and an artificial synapse may be represented by S. When identifying each artificial neuron, a superscripted reference symbol may be used as an identification letter. When identifying an arbitrary artificial neuron, i or j may be used as an identification letter. For example, N i represents an arbitrary artificial neuron.

[0042] In addition, an artificial synapse may be identified by the identification numbers i and j of the two artificial neurons connected to the artificial synapse. For example, S 41 is N 1 and N 4 It represents an artificial synapse that connects ij is N i The output of N j represents an artificial synapse that inputs to S ji is N j The output of N i represents an artificial synapse input to

[0043] In FIG. 3, A to J indicate the definition of the state of the robot 40. The state of the robot 40 includes the emotion of the robot 40, the state of production of endocrine substances, the situation of the robot 40, etc. 4 , N 6 , and N 7 is a conceptual artificial neuron in which a concept that represents the situation of the robot 40 is defined. For example, N 4 is a conceptual artificial neuron that has been assigned the situation "the bell rang". 6 is a conceptual artificial neuron that has been assigned the state "charging has started". 7 is a conceptual artificial neuron to which the state “charge level is below the threshold” is assigned.

[0044] N 1 , N 3 , N b and N c is an emotion artificial neuron in which the emotion of the robot 40 is defined. 1 is an emotional artificial neuron to which the emotion "happy" is assigned.3 is an emotional artificial neuron to which the emotion "sad" is assigned. b is an emotional artificial neuron to which the emotion "fear" is assigned. c is an emotional artificial neuron to which the emotion “fun” is assigned.

[0045] N 2 , N 5 and N a is an endocrine artificial neuron with a defined endocrine state of the robot 40. 5 is an endocrine artificial neuron that is assigned the developmental state of dopamine. Dopamine is an example of an endocrine substance involved in the reward system. 5 is an example of an endocrine artificial neuron involved in the reward system. 2 is an endocrine artificial neuron that is assigned the generation state of serotonin. Serotonin is an example of an endocrine substance involved in the sleep system. 2 is an example of an endocrine artificial neuron involved in the sleep system. a is an endocrine artificial neuron that has been assigned the developmental state of noradrenaline. Noradrenaline is an example of an endocrine substance involved in the sympathetic nervous system. a is an endocrine artificial neuron involved in the sympathetic nervous system.

[0046] The definition information 284 in the storage unit 280 stores information defining the state of the robot 40 as described above for each of the multiple artificial neurons that make up the neural network. Thus, the neural network 300 includes conceptual artificial neurons, emotional artificial neurons, and endocrine artificial neurons. Conceptual artificial neurons, emotional artificial neurons, and endocrine artificial neurons are artificial neurons in which the meanings of concepts, emotions, endocrine, etc. are explicitly defined. In contrast, N 8 Or N 9 is an artificial neuron with no defined state for the robot 40. 8 Or N 9is an artificial neuron whose meanings, such as concepts, emotions, and endocrine functions, are not explicitly defined.

[0047] The parameters of the neural network 300 are i The input to I t i and N from outside the neural network i E, which is the input to t i And, N i The parameters of and S i and parameters.

[0048] N i The parameters of N i S stands for status t i And, N i represents the internal state of the artificial neuron, V i m t And, N i T represents the threshold for firing i t And, N i The time when the last firing occurred is t f and the artificial neuron N at the final firing time i V represents the internal state of i m tf and the output increase / decrease parameter a t i , b t i , h t i The output increase / decrease parameter is an example of a parameter that determines the time evolution of the output when the artificial neuron fires. In this embodiment, the subscript t indicates that the parameter can be updated as time progresses. i m t is information corresponding to the membrane potential of the artificial neuron, and is an example of a parameter representing the internal state or output of the artificial neuron.

[0049] S ij The parameters of S ijBS represents the coupling coefficient of the artificial synapse t ij And, S. ij N connected i and N j The time when the two last fired simultaneously is t cf and BS, which represents the coupling coefficient at the time of the last simultaneous firing. ij tcf and a, which is the coupling coefficient increase / decrease parameter t ij , b t ij , h t ij The increase / decrease parameter of the coupling coefficient is an example of a parameter that determines the time evolution of the coupling coefficient after the two artificial neurons connected by the artificial synapse last fired simultaneously.

[0050] The parameter processing unit 240 updates the above-mentioned parameters based on the input from the external input data generating unit 230 and the neural network to determine the activation state of each artificial neuron. The motion determining unit 250 determines the motion of the robot 40 based on the internal state or activation state of at least some of the artificial neurons determined by the parameter values ​​of at least some of the artificial neurons among the multiple artificial neurons in the neural network, and the state defined for at least some of the artificial neurons by the definition information 284. Note that the activation state may be an activated state or an inactivated state. In this embodiment, activation may be referred to as "firing", and inactivation may be referred to as "not firing". Note that, as will be described later, the "firing" state is divided into an "upward phase" and a "downward phase" depending on whether the internal state is rising or not. The "not firing" and the "upward phase" and "downward phase" are different from each other. " is represented by the status Sti.

[0051] Figure 4 shows the parameters of the neural network in a schematic table format. Each neuron N is assigned a threshold T t and the increase / decrease parameter h t , a t and b tEach artificial synapse has a coupling coefficient BS t and the increase / decrease parameter h t , a t and b t The parameters are as follows: i Every time, an artificial synapse is used, i In each row, the parameters of all the artificial neurons directly connected to the corresponding artificial synapse are shown, as well as the parameters of the corresponding artificial synapse.

[0052] 5 shows an outline of the operation flow of the server 200 when the robot 40 is started or reset. When the server 200 receives a notification that the robot 40 has been started or reset, the parameter processing unit 240 performs initial setting of the parameters of the neural network. For example, the parameter processing unit 240 acquires initial values ​​of the parameters from the storage unit 280 and generates parameter data of the neural network in a predetermined data structure (S502). In addition, the parameter processing unit 240 sets the values ​​of the parameters of the neural network at time t0. When the initial setting is completed, in S504, a loop for time t is started.

[0053] In S510, the parameter processing unit 240 calculates the time step t n+1 Calculate the parameters corresponding to the changes due to the electrical influence of the artificial synapse in . t ij Calculate.

[0054] In S520, the parameter processing unit 240 calculates the time step t n+1 Calculate the parameters corresponding to the changes due to the chemical effects of endocrine substances in N i and S ij More specifically, we calculate the change in the parameters of n+1 Artificial neuron N affected by endocrine artificial neuron i The internal state of the S ijCalculate the coupling coefficient increase / decrease parameters and coupling coefficients.

[0055] In S530, the parameter processing unit 240 acquires an input from outside the neural network. Specifically, the parameter processing unit 240 acquires the output of the external input data generating unit 230.

[0056] In S540, the parameter processing unit 240 calculates the time step t n+1 In N i Calculate the internal state of V i m tn+1 and Status S tt i Then, in S550, at time t n+1 The values ​​of the parameters at time t are stored in the parameter 288 of the storage unit 280. n+1 The value of each parameter in is output to the operation determination section 250 and the switching control section 260.

[0057] In S560, the switching control unit 260 n+1 N in i At time step t n+1 N in i If the parameters satisfy the switching conditions for the recording format, the switching control unit 260 instructs the robot 40 to switch the recording format (S570). Next, when switching from the first recording format to the second recording format, the recording control unit 270 instructs tagging of information in the second recording format (S580) and proceeds to S506. On the other hand, in S560, n+1 N in i If the parameters do not satisfy the conditions for switching the recording format, the process proceeds to S506.

[0058] In S506, the parameter processing unit 240 judges whether or not to end the loop. For example, it judges to end the loop when the time represented by the time step reaches a predetermined time or when sensor information from the robot 40 is not received for a predetermined time. If the loop is not to be ended, the process returns to S510 and the calculation of the next time step is further performed. If the loop is to be ended, this flow is ended.

[0059] FIG. 6 is a diagram for explaining the calculation of the coupling coefficient of the artificial synapse. Here, the constant a is used as the initial value of the increase / decrease parameter. ij and b ij This section describes the cases where is defined.

[0060] time t n At the time step of S ij N on both ends of i and N j are firing, the parameter processing unit 240 n+1 in B.S. tn+1 ij ,BS tn+1 ij =BS tn ij +a tn ij ×(t n+1 -t n ) is calculated. On the other hand, at time t n At time steps S i and S j If none of them fires at time t n+1 Coupling coefficient BS tn+1 ij ,BS tn+1 ij =BS tn ij +b tn ij ×(t n+1 -t n ) is calculated. Also, BS tn+1 ij If is negative, BS tn+1 ij is set to 0. Note that BS ij is a positive value of Sij So, a t ij is a positive value, and b t ij is a negative value. ij S is negative ij So, a t ij is a positive value, and b t ij is a negative value.

[0061] As shown in Figure 6, at time t0, the artificial neurons at both ends fire simultaneously. t ij is a per unit time t0 ij In addition, since there are no simultaneous firings at time t1, BS t ij is |b per unit time t1 ij In addition, due to simultaneous firing at time t4, BS t ij is a per unit time t4 ij increases with.

[0062] Figure 7 shows the function h t ij The time evolution of the coupling coefficient when is defined is shown below. t ij t cf The time elapsed from Δt (= tt cf ) ≧0. t ij is a function of at least Δt and takes real values.

[0063] The function 700 shown in FIG. t ij The function 700 is an example of cf Coupling coefficient BS tcf ijand Δt. The function 700 monotonically increases when Δt is less than a predetermined value, and monotonically decreases and gradually decreases toward 0 when Δt is greater than the predetermined value. The function 700 has a value BS tcf ij Take.

[0064] FIG. 7 shows that a function 700 is defined as a parameter for increasing or decreasing the coupling coefficient. i and N j The parameter processing unit 240 calculates the coupling coefficients of BSs at each time t1 to t6 based on the function 700 and Δt. t ij In the time range from time t1 to time t6, N i and N j do not fire simultaneously. Therefore, for example, after time t2, the coupling coefficient decreases monotonically.

[0065] Figure 8 shows N i and N j The time evolution of the coupling coefficient when the neurons fire simultaneously is shown in Fig. The coupling coefficient is calculated in the same manner as in FIG. 7 from time t0 to time t2. At time t2, N i and N j When these two fire simultaneously, the parameter processor 240 t ij (t-t2,BS t2 ij ) and calculates the coupling coefficient at each of times t3 to t6. In this way, the coupling coefficient increases each time simultaneous firing is repeated. This has the effect of strengthening the artificial synaptic connection, as in Hebb's law in living organisms. On the other hand, as shown in Figures 6 and 7, when the time without simultaneous firing becomes long, it has the effect of weakening the artificial synaptic connection.

[0066] Fig. 9 shows a schematic diagram of effect definition information that defines a chemical effect given to a parameter. This effect definition information is used to calculate a change in the parameter in S520 of Fig. 5. The definition information includes a condition regarding the internal state of the endocrine artificial neuron, information specifying the artificial neuron or artificial synapse that exerts an effect, and an equation that determines the effect content.

[0067] In the example of FIG. 9, the endocrine artificial neuron N 2 is an endocrine artificial neuron to which the endocrine substance responsible for sleepiness is assigned. Endocrine artificial neuron N 2 For definition information about VM tn 2 >T tn 2 "Conditions for Endocrine Artificial Neuron N 2 The artificial neuron N 1 and N 3 ", and "T tn+1 i =T tn i Parameter processing unit 240 determines Vm tn 2 T tn 2 If it exceeds t n+1 Emotional Artificial Neuron N 1 and N 3 The threshold is increased by 10%. This makes it difficult for the emotional artificial neuron to fire when drowsiness occurs. For example, the conceptual artificial neuron N 7 The output of the endocrine artificial neuron N 2 By defining a neural network connected to the input of the above, it is possible to embody the phenomenon in which emotions become less likely to increase as the battery level becomes low.

[0068] In addition, the endocrine artificial neuron N 5 is an endocrine artificial neuron to which dopamine is assigned. Endocrine artificial neuron N 5 The first definition information for "Vm tn5 >T tn 5 and Vm tn 4 >T tn 4 "Conditions for Endocrine Artificial Neuron N 5 As an artificial synapse that is influenced by 49 and S 95 ", and "a" is the formula that determines the effect. tn+1 ij =a tn ij The formula "Vm × 1.1" is defined. tn 5 T tn 5 Exceeds Vm tn 4 T tn 4 If it exceeds t n+1 Artificial Synapse S 49 and S 95 Increases the increase / decrease parameters by 10%.

[0069] This allows the concept of artificial neuron N to be defined as the situation in which the reward system's endocrine artificial neuron fires, resulting in the "bell ringing" condition. 4 is firing, the implicit artificial neuron N 9 Conceptual artificial neuron N via 4 and N 5 This allows the binding of the artificial neuron N to the reward system when the "bell" is rung. 5 becomes more likely to ignite.

[0070] In addition, the endocrine artificial neuron N 5 The second definition of "Vm tn 5 >T tn 5 "Conditions for Endocrine Artificial Neuron N 5 The artificial neuron that is influenced by 1 ", and "T tn+1 i =T tn iThe formula "Vm × 1.1" is defined. tn 5 T tn 5 If it exceeds t n+1 Artificial neuron N 1 This reduces the increase / decrease parameter of the reward system endocrine artificial neuron N 5 When this emotion is ignited, it becomes easier for the emotion of happiness to be ignited.

[0071] According to the definition of the effect of such a reward system on endocrine artificial neurons, it is possible to implement an implementation in which, if the action of charging the robot 40 while ringing a bell is repeated, the robot 40 will behave in a way that expresses happiness just by ringing the bell.

[0072] The influence definition information is not limited to the example of FIG. 9. For example, a condition may be defined that the internal state of the artificial neuron is equal to or lower than a threshold. A condition regarding the status of the artificial neuron, for example, a condition regarding an ascending phase, a descending phase, or non-firing, may be defined. The influence range may be defined by directly specifying an artificial neuron or an artificial synapse, or by defining "all artificial synapses connected to a specific artificial neuron." Regarding the influence formula, in the case where the target is an artificial neuron, in addition to multiplying the threshold by a constant, a formula may be defined in which a constant is added to the threshold or an increase / decrease parameter of the internal state is multiplied by a constant. In the case where the target is an artificial synapse, in addition to multiplying the increase / decrease parameter by a constant, an formula may be defined in which a coupling coefficient is multiplied by a constant.

[0073] The influence definition information is stored in the definition information 284 of the storage unit 280. Thus, the storage unit 280 stores influence definition information that defines the influence of at least one of the internal state and firing state of the endocrine artificial neuron on the parameters of at least one of the other artificial neurons and artificial synapses that are not directly connected to the endocrine artificial neuron via an artificial synapse. Then, the parameter processing unit 240 updates the parameters of at least one of the other artificial neurons and artificial synapses that are not directly connected to the endocrine artificial neuron via an artificial synapse based on at least one of the internal state and firing state of the endocrine artificial neuron and the influence definition information. Furthermore, the parameters of the other artificial neurons that are influenced by at least one of the internal state and firing state of the endocrine artificial neuron can include at least one of the parameters that define the threshold value, firing state, and time development of the output at the time of firing of the other artificial neuron. Furthermore, the parameters of the artificial synapse influenced by at least one of the internal state and the firing state of the endocrine artificial neuron can include at least one of the coupling coefficient of the artificial synapse and a parameter that determines the time evolution of the coupling coefficient after the two artificial neurons linked by the artificial synapse last fired simultaneously. Furthermore, the influence definition information includes information that determines the influence that the firing state of the endocrine artificial neuron associated with the reward system has on the threshold of the emotional artificial neuron, and the parameter processing unit 240 updates the threshold of the emotional artificial neuron in accordance with the influence definition information when the endocrine artificial neuron fires.

[0074] Figure 10 shows the V tn+1 i and S tn+1 i 5. The process of this flowchart can be applied to a part of the process in S540 in FIG. 5. In S1100, the parameter processing unit 240 calculates S tn i indicates non-firing.

[0075] S tn i If indicates non-firing, the parameter processing unit 240 i Input to Itn+1 i Specifically, when the input from outside the neural network is N i If not connected to I tn+1 i =Σ j B.S. tn+1 ji ×VM tn j ×f(S tn j ) is calculated by the following formula: i If connected to I tn+1 i =Σ j B.S. tn+1 ji × VM tn j ×f(S tn j )+E tn+1 i Here, E tn i is the time t from outside the neural network. n This is the input in

[0076] In addition, f(S) returns 0 when S is a value that indicates no firing, and returns 1 when S is a value that indicates an ascending or descending phase. This model corresponds to a model in which a synapse transmits an action potential only when a neuron fires. Note that f(S)=1 may also be returned. This corresponds to a model in which a membrane potential is transmitted regardless of the firing state of the neuron.

[0077] In S1112, the parameter processing unit 240 tn+1 i T tn+1 i Determine whether it exceeds I tn+1 i T tn+1 i If it exceeds Vm tn+1 i is calculated based on the increase / decrease parameters, and Vm tn+1 i Depending on S tn+1i is set to a value indicating the rising phase or the falling phase (S1114), and this flow ends.

[0078] In the S1100, tn i When Vm is in the rising phase or the falling phase, the parameter processing unit 240 tn+1 i Then, the parameter processing unit 240 calculates t n+1 By Vm t i When Vmin is reached, S tn+1 i Set to the unfired value and t n+1 By Vm t i If Vmin is not reached, S tn+1 i The parameter processing unit 240 sets t n+i By Vm t i When Vmax is reached, S tn+1 i Set the value of the falling phase to t n+1 By Vm t i If Vmax is not reached, S tn+1 i Set the value of the rising phase to

[0079] In this way, N i If is firing, even if the output falls below the threshold, N i The output of is independent of the input. Such a period corresponds to the absolute refractory period in biological neurons.

[0080] Figure 11 shows the N i V when does not ignite t i FIG. 13 is a diagram for explaining an example of calculation of

[0081] At time step t0, N i is not fired. I at time t1 t1 i Tt1 i If V at time t1 is equal to or less than V t1 i , V t1 i =I t1 i Calculate V from time t0 to t1. t i , V t i =I t0 i Similarly, the parameter processing unit 240 calculates the time step t n The calculated V t n maintains the value of until the next time step, and V tn+1 In I tn+1 Change to.

[0082] Figure 12 shows the N i V when ignition occurs i t FIG. 12 is a diagram for explaining an example of calculation of the constant a i and b i This is a calculation example when is defined.

[0083] At time step t0, N i is not fired. I at time t1 t1 i T t1 i If the parameter processing unit 240 exceeds V t1 i , V t1 i =I t1 i Calculate V from time t0 to t1. t i , V t i =I t0 i Here, the calculation is performed using I at time t1. t1 i is less than Vmax. I at time t1 t1 iIf exceeds Vmax, I t1 i =Vmax.

[0084] As shown in FIG. 12, after time t1, the parameter processing unit 240 t i Until the time when V reaches Vmax, t i per unit time t ij In addition, the parameter processing unit 240 increases N i Status S t i is determined to be the ascending phase.

[0085] Also, V t i When V reaches Vmax, V t i Until V reaches Vmin, V t i per unit time |b t i In addition, the parameter processing unit 240 reduces N i The status of V is determined to be in the descending phase. t i When Vmin is reached, V at the next time t6 i , V t6 i =I t6 i Calculate by V t i After reaching Vmin, the status is determined as non-firing.

[0086] In addition, N i If the status of is in the declining phase, the calculated Vm t i T t i Even if it falls below Vm t i I t i The parameter processing unit 240 does not depend on Vm t i Tt i Even if it falls below Vm t i Vm increases or decreases according to the parameter until it reaches Vmin. t i Calculate.

[0087] Figure 13 shows the N i As a parameter of the function h t i The time evolution of the coupling coefficient when is defined is shown below. In general, h t i is the firing time t f The time elapsed from Δt (= tt f ) ≧0. t i is at least a function of Δt. t i takes real values, and h t i The value range is from Vmin to Vmax.

[0088] The function 1300 shown in FIG. t i The function 1300 is an example of f In tf i and Δt. The function 1300 increases monotonically when Δt is less than a predetermined value, and decreases monotonically when Δt is greater than a predetermined value. The function 1300 has a value Vm tf i Take.

[0089] FIG. 13 shows a case where a function 1400 is defined as an increase / decrease parameter of the internal state, and N i The parameter processing unit 240 calculates the output when the function 1400, Δt and Vm f i Based on this, Vmti is calculated for each time from time t1 to time t5. t i Since Vmin is reached at time t5, Vm i =I t6i It becomes.

[0090] FIG. 14 shows an example of a rule 1400 stored in the recording format switching rule 290 in the form of a table. The rule 1400 is 1 , N 3 , N b and N c When at least the first condition is satisfied that any one of the Vmti exceeds a threshold, the information recording format is "switched to a low compression format". In this way, when information is recorded in a high compression format, the switching control unit 260 determines to switch the information recording format to a low compression format when the first condition is satisfied from a state in which the first condition is not satisfied. Note that, as the threshold, each of N j The threshold value may be higher than Tit.

[0091] Also, rule 1400 states that 5 and N a VM t i If the second condition is met, that is, the total value of the data exceeds the threshold, the data recording format is switched to a low compression format. " is set. In this way, when information is recorded in a high compression format, the switching control unit 260 determines to switch the information recording format to a low compression format when the second condition is satisfied from a state in which the second condition is not satisfied. Note that, as a threshold value, j The total value of Vmax of each is multiplied by a constant 0.9. j T i t may be higher than the sum of the above.

[0092] N 1 , N 3 , N b and N care emotion artificial neurons in which emotions "happy", "sad", "fearful" and "fun" are defined, respectively. Therefore, the parameter processing unit 240 determines the intensity of the emotion based on the internal state of the emotion artificial neuron, and when the determined intensity of the emotion exceeds a predetermined threshold, the recording format can be switched to a low compression format.

[0093] N5 and Na are endocrine artificial neurons in which the endocrine substances "dopamine" and "noradrenaline" are defined, respectively. The sum of the parameters of the internal states of these endocrine artificial neurons is an example of an index representing the intensity of the emotion of "excitement." Therefore, the parameter processing unit 240 determines the intensity of the emotion based on the internal states of the endocrine artificial neurons, and when the determined intensity of the emotion exceeds a predetermined threshold, the recording format can be switched to a low compression format.

[0094] Also, rule 1400 states that 1 , N 3 , N b and N c Any VM t i is also equal to or smaller than the first threshold, and N 5 and N a VM t i When a third condition is satisfied, that is, the sum of the first and second thresholds is equal to or less than the second threshold, the operation of "switching the data recording format to a high compression format" is defined. Therefore, when information is recorded in a low compression format, the switching control unit 260 determines to switch the information recording format to a high compression format when the third condition is satisfied from a state in which the third condition is not satisfied. In this way, the recording format can be switched to a high compression format in response to the intensity of the emotion becoming equal to or less than the predetermined threshold.

[0095] The first threshold of the third condition is j The second threshold value of the third condition is the value obtained by multiplying the Vmax of each N jThe value is obtained by multiplying the total value of Vmax of the three conditions by the constant 0.8. In this manner, the case is exemplified in which the first threshold value of the third condition is smaller than the threshold value of the first condition, and the second threshold value of the third condition is smaller than the threshold value of the second condition. However, the first threshold value may be the same as the threshold value of the first condition, and the second threshold value may be the same as the threshold value of the second condition. Also, the first threshold value of the third condition is set to a value obtained by multiplying the total value of Vmax of the three conditions by the constant 0.8. j T i t The second threshold of the third condition may be higher than each N j T i t In addition, the threshold value for each condition is not limited to these examples, and various values ​​can be applied.

[0096] According to the system 20, the robot 40 continuously transmits highly compressed information such as skeletal data to the server 200 during a period when the emotion of the robot 40 is not significantly high, and the server 200 records the information. The continuous information such as skeletal data recorded in the server 200 can be used when analyzing the memory of the robot 40. When the emotion of the robot 40 significantly increases, the robot 40 starts transmitting full HD video data and audio data, and during a period when the emotion is strong enough to exceed a certain value, the server 200 records low-compression information including the full HD video data and audio data in addition to the skeletal data. When the robot 40 is requested by the user 30 to provide a video of the robot 40's memories, the robot 40 requests the server 200 to transmit the full HD video data and audio data, and provides the user 30 with the video data and audio data received from the server 200.

[0097] In this way, according to the system 20, high-quality video data of scenes in which the robot 40 feels strong emotions can be stored in the server 200. On the other hand, when the robot 40 does not feel strong emotions, simplified information such as skeletal data can be stored in the server 200. In this way, the robot 40, like a human being, can retain clear memories of times when it felt strong emotions, while retaining simplified memories of times when it did not feel strong emotions.

[0098] In this embodiment, "happy", "sad", "fearful", "fun" and "excited" are described as emotions, but the emotions handled by the system 20 are not limited to these. In addition, in this embodiment, "dopamine", "serotonin" and "noradrenaline" are described as endocrine substances, but the endocrine substances handled by the system 20 are not limited to these.

[0099] [Acting according to emotions] The robot 40 may take an action to control the emotion value when recording tagged information. The robot 40 simulates a human being taking an action according to an emotion. For example, the robot 40 may take an action to avoid an event that causes a negative emotion (such as "anger" or "sadness"). This simulates a human being's action to avoid or relieve stress. Also, for example, the robot 40 may take an action to enhance a positive emotion (such as "fun" or "happiness"). This simulates a human being's action to share an event that causes a positive emotion with another person.

[0100] As described above, the recording control unit 270 processes at least a portion of the information continuously detected by the sensor to generate information, and records the generated information.

[0101] The movement determination unit 250 determines the movement of the robot 40 for controlling the emotion value based on the emotion value representing the emotion of the robot 40 or information detected by a sensor. The movement determination unit 250 can determine the movement of the robot 40 at the timing when the recording control unit 270 records information (for example, immediately before, during, or immediately after recording).

[0102] The action determination unit 250 performs action determination at the timing to record information in parallel with the other action determinations described above or by interrupting the other action determinations. The action determination unit 250 also determines the action at the timing to record information according to the condition table and action table included in the action determination rule 282.

[0103] The action determination unit 250 refers to the condition table and determines whether a condition corresponding to the type of emotion is satisfied. If the condition is satisfied, the action determination unit 250 further refers to the condition table and specifies whether to increase or decrease the emotion value of the corresponding emotion.

[0104] Fig. 15 is a diagram showing an example of a condition table. As shown in Fig. 15, the condition table stores types of emotions, conditions, and increases and decreases in association with each other.

[0105] Furthermore, the movement determination unit 250 refers to the movement table to obtain a movement for increasing or decreasing the emotion value of each emotion type. The movement determination unit 250 determines the obtained movement as the movement of the robot 40.

[0106] Fig. 16 is a diagram showing an action table. As shown in Fig. 16, the action table stores types of emotions, increases and decreases, and actions in association with each other.

[0107] For example, when the condition "emotion value is equal to or greater than a threshold" is satisfied for the emotion type "anger", the action determination unit 250 determines an action that will "reduce" the emotion value of "anger" based on the condition table. In this case, the action determination unit 250 refers to the action table and specifies that the action that will reduce the emotion value of "anger" is "turning off the power". In this case, the action determination unit 250 determines the action of "turning off the power" of the robot 40 in order to reduce the emotion value of "anger". Note that when the robot 40 is turned off, it is possible that the emotion value will not change, but here, the action that reduces the emotion value also includes the action of stopping an increase in the emotion value.

[0108] As shown in FIG. 16, actions that reduce the emotion value of "anger" include not only "turning off the power" but also "moving to another location." In this way, there may be a plurality of actions that control (increase or decrease) the emotion value of the same emotion in the same way. In this case, the action determination unit 250 may randomly determine one action from the plurality of actions, or may determine one action according to a predetermined rule. "Moving to another location" is an action for moving away from the cause of the increase in the emotion value of "anger" (for example, a human being who harms the robot 40).

[0109] The determination of the action by the action determining unit 250 is not limited to the example described here. The action determining unit 250 can determine the action as follows, according to the condition table in FIG. 15 and the action table in FIG.

[0110] For example, when the rate of increase (amount of increase per unit time) of the emotion value of "anger" is equal to or greater than a threshold value, the action determination section 250 determines an action for reducing the emotion value.

[0111] Also, for example, when the information to be recorded is the same as when the condition was satisfied in the past for the emotion of "anger", the action determination unit 250 determines an action to reduce the emotion value. For example, it is assumed that when the first video data was recorded in the past, the emotion value of "anger" became equal to or greater than a threshold value, and an action to reduce the emotion value of "anger" was performed. It is also assumed that the second video data to be recorded currently has the same content as the first video data. For example, the content of the video data is a user hitting the robot 40. In this case, the action determination unit 250 determines that stress needs to be avoided regardless of the emotion value, and determines an action to reduce the emotion value of "anger".

[0112] Also, for example, when the emotion value of "fun" is equal to or greater than a threshold, the action determination unit 250 determines an action for increasing the emotion value. Also, for example, when the rate of increase in the emotion value of "fun" is equal to or greater than a threshold, the action determination unit 250 determines an action for increasing the emotion value. An example of an action for increasing the emotion value of "fun" is "singing a song" (outputting audio with singing voice).

[0113] In this way, the action determination unit 250 can determine an action for increasing the emotion value of a positive emotion such as "fun," as opposed to determining an action for decreasing the emotion value of a negative emotion such as "anger."

[0114] Furthermore, the combinations of conditions and actions are not limited to those shown in Figures 15 and 16. For example, the action determination section 250 can determine the action as follows.

[0115] For example, when the emotional value of "anger" is equal to or greater than a threshold value and the emotional value of "sadness" is equal to or greater than a threshold value, or when the emotional value of "sadness" is approaching the emotional value of "anger", the action determination unit 250 determines an action for decreasing the emotional value of at least one of "anger" and "sadness". The emotional value of "sadness" approaching the emotional value of "anger" means, for example, that the emotional value of "sadness" is less than the emotional value of "anger" and the rate of increase in the emotional value of "sadness" is equal to or greater than a threshold value.

[0116] Also, for example, the robot 40 stores, as a blacklist, images of the faces of surrounding users when the emotion value of "anger" in the past exceeded a threshold value. Then, when the action determination unit 250 recognizes from the image of the robot 40's surroundings that a user included in the blacklist is approaching the robot 40, it determines an action to move away from the scene, thereby avoiding in advance a situation in which the emotion value of "anger" increases.

[0117] Also, for example, the robot 40 stores the surrounding weather (e.g., severe thunderstorm) when the emotion value of "fear" was equal to or greater than the threshold value in the past. Then, the operation determination unit 250 determines the behavior of turning off the power when the weather around the robot 40 is predicted to be the stored weather. The operation determination unit 250 may predict the weather from the surrounding temperature, humidity, cloud images, etc., or may predict the weather based on weather forecast information obtained from an external server, etc.

[0118] So far, it has been explained that the action determination unit 250 determines an action that reduces the emotion value of a negative emotion (such as "anger", "sadness", etc.) and determines an action that increases the emotion value of a positive emotion (such as "fun", "joy", etc.). On the other hand, the action determination unit 250 may determine an action that increases the emotion value of a negative emotion, or may determine an action that decreases the emotion value of a positive emotion.

[0119] For example, when the user's emotion is "sad" and the emotion value of "sad" of the robot 40 is less than a threshold value, the action determination section 250 determines an action that increases the emotion value of "sad".

[0120] Also, for example, when the user's emotion is "sad" and the emotion value of "fun" of the robot 40 is equal to or greater than the threshold, the action determination section 250 determines an action to reduce the emotion value of "fun".

[0121] This allows the user and the robot 40 to share emotions, improving the user's level of satisfaction.

[0122] Furthermore, the action determination unit 250 may determine an action for removing a phenomenon that increases the emotion value of a negative emotion as an action for reducing the emotion value of a negative emotion. For example, if the emotion of "anger" of the robot 40 is increased by the user hitting the robot 40, the action determination unit 250 determines an action of making a statement requesting the user to stop hitting.

[0123] Furthermore, instead of turning off the power of the robot 40, the action determination unit 250 may determine that the robot 40 temporarily goes to sleep. During sleep, the robot 40 does not perform any action other than sleep, and returns from sleep after a certain period of time has elapsed. During sleep, the emotion value of the robot 40 does not increase or decrease.

[0124] It can be said that the movement determination unit 250 determines the movement of the robot 40 by determining the control of one or more devices constituting the robot 40. The devices constituting the robot 40 include a control target 155, a sensor unit 156, a display unit 157, etc. In other words, the devices constituting the robot 40 are motors for moving the limbs, etc., various sensors including a camera, a speaker, a display for displaying images, a power source, etc.

[0125] The action determination unit 250 can determine the action of "turning off the power supply" by determining control of the power supply that supplies power to the robot 40. In addition, the action determination unit 250 can determine the action of "moving to another location" by determining control of a motor for moving a device (e.g., wheels) provided on the robot 40.

[0126] Furthermore, the movement determination unit 250 can determine the movement of "shaking the shoulders" by determining the control of the motor that moves the arm of the robot 40.

[0127] Furthermore, the operation determination unit 250 can determine an operation that changes the degree of interest in the surroundings of the robot 40 by determining control of the sensitivity of the sensor of the robot 40. For example, the operation determination unit 250 can determine an operation of "disinterest" by lowering the sensitivity of the sound sensor of the robot 40.

[0128] Furthermore, the movement determination unit 250 can determine the movement of "eyes wandering" by determining the control of the motors that move the eyeballs of the robot 40. The movement determination unit 250 can determine the movement of the hands and feet by determining the control of the motors that move the arms and feet.

[0129] FIG. 17 is a flowchart showing the flow of a process for determining an operation at the timing of recording information.

[0130] First, recording control section 270 starts recording processing (step S2101). Then, operation control section 250 determines whether or not a condition for increasing or decreasing a specific emotion value is satisfied (step S2102). For example, operation control section 250 refers to a condition table and determines whether or not a condition is satisfied.

[0131] If the condition is met in step S2103 (Yes in step S2103), the action determination unit 250 determines an action that increases or decreases the emotion value, and causes the robot 40 to execute the determined action (step S2104). Then, the recording control unit 270 records the information (step S2105), and ends the recording process (step S2106).

[0132] If the condition is not satisfied in step S2103 (No in step S2103), the recording control unit 270 records the information (step S2105), and ends the recording process (step S2106).

[0133] 17, the operation control unit 250 performs the process of determining the operation from when the recording control unit 270 starts the recording process to when the recording process ends. On the other hand, the timing at which the operation control unit 250 performs the process of determining the operation is not limited to that described in FIG.

[0134] The operation control unit 250 may perform the process of determining the operation before the recording control unit 270 starts the recording process. Also, the operation control unit 250 may perform the process of determining the operation after the recording control unit 270 ends the recording process.

[0135] The functions of the server 200 may be implemented by one or more computers. At least some of the functions of the server 200 may be implemented by a virtual machine. At least some of the functions of the server 200 may be implemented in a cloud. Among the functions of the server 200, the functions of the components other than the storage unit 280 can be realized by a CPU operating based on a program. For example, at least some of the processing described as the operation of the server 200 can be realized by a processor controlling each piece of hardware (e.g., a hard disk, a memory, etc.) possessed by the computer according to a program. In this way, at least some of the processing of the server 200 can be realized by the processor operating according to a program to control each piece of hardware, so that each piece of hardware including the processor, the hard disk, the memory, etc. and the program operate in cooperation with each other. That is, the program can make the computer function as each component of the server 200. Similarly, among the components of the robot 40, the functions of the components other than the control target 155 and the sensor unit 156 can be realized by a CPU operating based on a program. That is, the program can make the computer function as each component of the robot 40. The computer may load a program that controls the execution of the above-mentioned processing, and operate according to the loaded program to execute the processing. The computer can read the program from a computer-readable recording medium that stores the program. The program can also be supplied to the computer through a communication line, and the computer can read the program through the communication line. The program may be read from a source connected via a communication line.

[0136] In the embodiment described above, the server 200 separate from the robot 40 is responsible for processing the neural network. Also, the server 200 separate from the robot 40 stores information such as video data. However, the robot 40 itself may perform the functions of the server 200, such as processing the neural network. Also, the robot 40 itself may store information such as video data. Also, the robot 40 is an example of a device to be controlled by the server 200. The device to be controlled is not limited to the robot 40, and various devices such as home appliances, vehicles, toys, etc. can be applied as the controlled devices.

[0137] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiment 40. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0138] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order. [Explanation of symbols]

[0139] 20 Systems 30 users 40 Robot 90 Communication Network 152 Processing section 155 Control Object 156 Sensor section 158 Communications Department 161 Mike 162 3D Depth Sensor 163 2D Camera 164 Distance Sensor 200 Servers 202 Processing section 208 Communications Department 210 Initial value setting section 230 External input data generation unit 240 Parameter Processing Section 250 Action Decision Unit 260 Switching control section 270 Recording control section 280 Storage Unit 282 Action Decision Rules 284 Definition information 286 Parameter Initial Values 288 Parameters 290 Recording format switching rules 292 Recorded Data 300 Neural Networks

Claims

1. A control system for controlling the behavior of a robot, comprising: a recording control unit that processes at least a part of the information continuously detected by the sensor to generate information and records the generated information; an action determination unit that determines an action of the robot for controlling an emotion value representing an emotion of the robot or information detected by the sensor; A control system comprising:

2. The action determination unit determines an action for increasing or decreasing an emotion value representing an emotion of the robot or information detected by the sensor when the emotion value satisfies a condition. The control system of claim 1 .

3. The action determination unit determines an action of one or more devices constituting the robot when an emotion value indicating an emotion of the robot or information detected by the sensor satisfies a condition. The control system of claim 1 .

4. The operation determination unit determines to turn off the power supply of the robot when an emotion value indicating an emotion of the robot or information detected by the sensor satisfies a condition. The control system of claim 3.

5. The action determination unit determines to drive a device for movement provided in the robot and move the robot when an emotion value indicating an emotion of the robot or information detected by the sensor satisfies a condition. The control system of claim 3.

6. The action determination unit determines an action of the robot for controlling the emotion value when the emotion value is equal to or greater than a threshold value. The control system of claim 1 .

7. The action determination unit determines an action of the robot for controlling the emotion value when the increase rate of the emotion value is equal to or greater than a threshold value. The control system of claim 1 .

8. The recording control unit At least one of the following information is added as a tag to the generated information: surrounding information, smell, temperature, humidity, location information, weather, number of people in the vicinity, facial expression, and emotion information. The control system of claim 1 .

9. The control system that controls the robot's behavior Processing at least a portion of the information continuously sensed by the sensor to generate information and recording the generated information; Based on an emotion value representing an emotion of the robot or information detected by the sensor, a motion of the robot for controlling the emotion value is determined. A control method comprising: executing a process.

10. The computer that controls the robot's actions Processing at least a portion of the information continuously sensed by the sensor to generate information and recording the generated information; Based on an emotion value representing an emotion of the robot or information detected by the sensor, a motion of the robot for controlling the emotion value is determined. A control program that executes a process.

Images