Control system, system, and program

Abstract

To output information that a user can feel empathy easily.SOLUTION: A control system according to an embodiment is a control system for controlling operations of a robot and includes a recording control unit and an output control unit. The recording control unit processes at least a part of information detected continuously by a sensor to generate information of a first recording format or information of a second recording format, the information of the second recording format being massive than the information of the first recording format, tags the information of the second recording format, and records the tagged information of the second recording format. The output control unit performs control so that the information of the second recording format selected on the basis of the tagged information is output from the information of the second recording format recorded by the recording control unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a control system, a system, and a program. [Background technology]

[0002] A terminal is known that learns a conversation between a user and a call partner and stores the call partner's replies to questions from the user in a reply table (see, for example, Patent Document 1). Also, an emotion generation device is known that includes a neural network that inputs user information, device information, and its own current emotional state and outputs the next emotional state (see, for example, Patent Document 2). Also, a technology is known that stores a spatiotemporal pattern in an associative memory that includes multiple electronic neurons having a layer-neural network relationship with directional artificial synaptic connectivity (see, for example, Patent Document 3). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-253389 A [Patent Document 2] Japanese Patent Application Publication No. 10-254592 [Patent Document 3] Special Publication No. 2013-535067 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional technology has room for improvement in terms of outputting information from a robot that is likely to elicit sympathy from users.

[0005] The present invention has been made in consideration of the above, and has an object to provide a control system, a system, and a program that can output information that is likely to resonate with the user. [Means for solving the problem]

[0006] A control system according to one aspect of the embodiment is a control system for controlling the behavior of a robot, and includes a recording control unit and an output control unit. The recording control unit processes at least a part of information continuously detected by a sensor to generate information in a first recording format or information in a second recording format having a larger amount of information than the information in the first recording format, and tags and records the information in the second recording format. The output control unit controls to output information in the second recording format selected from the information in the second recording format recorded by the recording control unit based on the tagged information. Effect of the Invention

[0007] According to one aspect of the embodiment, information that is likely to resonate with the user can be output. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a system 20 according to the present embodiment. [Diagram 2] FIG. 2 is a diagram showing a schematic block configuration of the server 200 and the robot 40. As shown in FIG. [Diagram 3] FIG. 3 is a schematic diagram of a neural network 300. [Figure 4] FIG. 4 is a diagram showing a schematic diagram of the parameters of the neural network in a table format. [Diagram 5] FIG. 5 is a diagram illustrating an outline of the operation flow of the server 200 when the robot 40 is started or reset. [Figure 6] FIG. 6 is a diagram for explaining the outline of the calculation of the connection weights of the artificial synapses. [Figure 7] FIG. 7 is a diagram showing an outline of the time evolution of the coupling coefficient when a function ht ij is defined as an increase / decrease parameter of the coupling coefficient. [Figure 8] FIG. 8 is a diagram showing a schematic diagram of the time evolution of the coupling coefficient when further simultaneous firing occurs at time t2. [Figure 9]FIG. 9 is a diagram illustrating an outline of effect definition information that defines a chemical effect applied to a parameter. [Figure 10] FIG. 10 is a flowchart showing the calculation of the internal state and status. [Figure 11] FIG. 11 is a diagram for explaining an example of calculation of the internal state when the artificial neuron does not fire. [Figure 12] FIG. 12 is a diagram for explaining an example of calculation of an output when an artificial neuron fires. [Figure 13] FIG. 13 is a diagram showing a schematic diagram of the time evolution of the coupling coefficient when a function is defined as an increase / decrease parameter of the artificial neuron. [Figure 14] FIG. 14 is a diagram showing an example of rules 1400 stored in the recording format switching rule 290 in table form. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0010] 1 is a schematic diagram of an example of a system 20 according to the present embodiment. The system 20 includes a server 200, and a robot 40a and a robot 40b. The robots 40a and 40b communicate with the server 200 via a communication network 90 to exchange information.

[0011] The user 30a is a user of the robot 40a. The user 30b is a user of the robot 40b. The robot 40b has substantially the same functions as the robot 40a. Therefore, the system 20 will be described with the robots 40a and 40b collectively referred to as the robot 40.

[0012] The robot 40 performs various actions according to the situation, such as moving its head or limbs, conversing with the user 30, and providing an image to the user 30. At this time, the robot 40 decides the action in cooperation with the server 200. For example, the robot 40 transmits to the server 200 detection information such as a facial image of the user 30 acquired by a camera function and a voice of the user 30 acquired by a microphone function. The server 200 analyzes the detection information received from the robot 40, decides the action to be taken by the robot 40, and transmits action information representing the decided action to the robot 40. The robot 40 operates according to the action information received from the server 200.

[0013] The robot 40 has an emotion value that represents its own emotion. For example, the robot 40 has an emotion value that represents the strength of each emotion, such as "happy," "fun," "sad," "fear," and "excitement." The emotion value of the robot 40 is determined by the server 200. The server 200 causes the robot 40 to perform an action according to the determined emotion. For example, when the robot 40 has a high emotion value of excitement and is talking to the user 30, the server 200 causes the robot 40 to speak quickly. In this way, the robot 40 can express its own emotion through actions, etc.

[0014] The server 200 updates the current state of the robot 40 using a neural network based on the detection information received from the robot 40. The state of the robot 40 includes the emotion of the robot 40. Therefore, the server 200 determines the emotion of the robot 40 using the neural network.

[0015] The robot 40 also records the video data of the user 30 acquired by the camera function in the server 200. The robot 40 acquires the video data from the server 200 as necessary and provides it to the user 30. The robot 40 generates video data with a larger amount of information as the strength of its own emotion increases and records it in the server 200. For example, when the robot 40 records information in a highly compressed format such as skeletal data, the robot 40 switches to recording information in a low compression format such as HD video when the emotion value of the robot 40 exceeds a threshold. According to the system 20, high-definition video data when the emotion of the robot 40 increases can be recorded. The robot 40 also generates video data with a larger amount of information as the strength of the emotion of the user 30 increases and records it in the server 200. Specifically, when the robot 40 records information in a highly compressed format such as skeletal data, the robot 40 switches to recording information in a low compression format such as HD video when the emotion value of the user 30 exceeds a threshold. According to the system 20, high-definition video data of a moment when the user 30 is emotionally excited can be left as a record.

[0016] Furthermore, even if the robot 40's or the user's emotion is not strong, when the robot 40's or the user's emotion changes suddenly, the robot 40 generates video data with a large amount of information and records it in the server 200. For example, when the robot 40 records information in a highly compressed format such as skeletal data, the robot 40 switches to recording information in a low compression format such as HD video in response to the change in the emotion value exceeding a threshold even if the emotion value of the excitement of the robot 40 or the user is below the threshold. According to the system 20, high-definition video data when the emotion of the robot 40 or the user 30 changes drastically can be recorded. Furthermore, the robot 40 may change the amount of information of the video data according to the degree of emotion change. Specifically, the robot 40 lowers the compression rate of the video data as the degree of change in the emotion of the robot 40 or the user increases. As another example, the robot 40 may change the format of the information to be recorded in the order of decreasing degree of change in the emotion of the robot 40 or the user, such as text only → image only → voice and image → low-quality video → high-quality video. Furthermore, the robot 40 may change the length or amount of video data, not just the compression rate or the format of information. This allows higher definition video data to be recorded when the emotions of the robot 40 or the user 30 change more dramatically.

[0017] The robot 40 may also change the information to be stored according to the continuity of the change in emotion of the robot 40 or the user 30. For example, when the cumulative value of the emotion value within a predetermined time exceeds a threshold, the robot 40 generates video data with a large amount of information and records it in the server 200. The robot 40 may also change the information to be stored according to the combination of the speed of the change (slope) of emotion within a predetermined time or the degree of the slope. For example, the robot 40 generates video data with a large amount of information and records it in the server 200 when the slope indicating the change in the emotion value is a combination of a sudden rise → a gradual rise for a certain period of time → a sudden fall within a predetermined time. This allows the robot 40 to record video data of a situation in which, for example, in a tense situation due to pressure or the like, the absolute value of the emotion is not relatively large but the time feels long (an emotion value above a certain value continues for a certain period of time). Change

[0018] In addition, the robot 40 tags the information to be recorded when the emotional value of the user 30 or the robot 40 exceeds a threshold value or when a change in the emotional value exceeds a threshold value. Specifically, the robot 40 tags the video data with information detected by a sensor provided in the robot 40 and information on the emotional value of the user 30 or the robot itself. The tagged information includes, for example, surrounding information (information on objects present in the vicinity, etc.), smell, temperature, humidity, location information, weather, the number of people in the vicinity, their expressions and emotions, etc. This allows the user 30 to clearly recall the situation at the time from the tagged information when checking the recorded information. Furthermore, by tagging, the information can be easily found when searching for information on a specific situation from the recorded information.

[0019] Furthermore, when outputting the recorded information, the robot 40 selects information to be output based on the tagged information. Specifically, the robot 40 selects information related to the current situation of the robot 40 and the tagged information as the output target.

[0020] For example, the robot 40 selects and outputs information relating to the attributes of people around and the attributes of the tagged person. Specifically, the robot 40 selects and outputs video data in which the ages of people around and the ages of the tagged person match or are similar, and does not select (or output) video data in which the ages do not match or are similar.

[0021] For example, if the video data shows a person in his / her twenties, the robot 40 tags the video data with "twenties". If the robot 40 determines that the person to whom the video data is to be provided is in his / her twenties, it selects and outputs the video data tagged with "twenties". In addition, even if the video data does not show a person, the robot 40 may estimate the age from an object shown in the video data and tag the person. For example, if the video data shows a stuffed toy for infants, the robot 40 may tag the video data with "infant". In addition, if the video data shows multiple people of different ages, the robot 40 may tag the range of ages of the multiple people (e.g., 20s to 50s). Alternatively, the robot 40 may count the ages of the multiple people and tag the statistical value (mode, median, average, etc.) of the counted ages.

[0022] Alternatively, the robot 40 selects and outputs video data in which the affiliation of the people around it matches or is similar to that of the tagged person. For example, the robot 40 identifies the workplace, department, title, etc. of the person shown in the video data, and tags the identified information. Then, the robot 40 selects and outputs the video data when the workplace, department, title, etc. of the person to whom the video data is provided matches the tagged information. In addition, the robot 40 may select and provide the video data when there is a certain relationship even if it does not match the tagged information, such as when the workplace of the person to whom the video data is provided is an affiliated company of the tagged workplace. In this way, by selecting and discarding information to be output based on the tagged information, it is possible to narrow down and output information that is likely to gain sympathy, and to avoid outputting information that is likely to not gain sympathy.

[0023] Furthermore, even if there is information relating the attributes of people in the vicinity to the attributes of a tagged person, if the emotions of the people in the vicinity are negative (anxiety, sadness, anger, etc.), the robot 40 will not output the information (prohibit output). This makes it possible to refrain from outputting information in a situation where the user is not comfortable with the information to be output. Note that, if the video data is tagged as "for people with negative emotions," the robot 40 may select and output the information even if the emotions of the people in the vicinity are negative.

[0024] In addition, even if the attributes of the people in the vicinity do not match the attributes of the people tagged in the video data, the robot 40 may select and output the video data if the attributes of the people in the vicinity are specific attributes designated in advance by the user.

[0025] Furthermore, the robot 40 judges whether the output information has gained the user's sympathy, for example, based on the user's emotion or behavior. Specifically, the robot 40 judges that the user has sympathy for the output information when the result of the information output by the robot 40 is the same as (or a close value within a predetermined range) the user's emotion value (or type of emotion) estimated by the robot 40. Furthermore, the robot 40 judges that the user has sympathy for the output information when the result of the information output by the robot 40 is the same as (or similar to) the user's behavior estimated by the robot 40 (nodding, utterance of "I see" or the like).

[0026] When selecting information to be output, the robot 40 determines whether the information will elicit sympathy if output, and selects and outputs information determined to elicit sympathy. The robot 40 may select information by setting whether to prioritize sympathy based on the user's emotion or sympathy based on the user's movement. In other words, the robot 40 selects information to be output by prioritizing the target of sympathy (emotion or movement). The priority order may be changed as appropriate to prioritize the user's emotion, the movement, or any of a plurality of elements other than emotion or movement.

[0027] In addition, the multiple elements other than emotions and actions include, for example, actions that a user who wants to gain sympathy from others takes toward other users. For example, the robot 40 determines that the user has sympathized with the user when the robot 40 talks to the other users about the content of the information output to the user, or imitates the movements of people shown in the video data output to the user. In other words, when selecting information to be output, the robot 40 infers what kind of behavior the user will take toward other users by outputting the information, and selects information to be output based on the inference.

[0028] Furthermore, when selecting information to be output, the robot 40 may select information that is likely to gain sympathy and information that is unlikely to gain sympathy, and output the information that is likely to gain sympathy first, and then output the information that is likely to gain sympathy. This makes it easier for the information to be output to gain sympathy.

[0029] 2 is a schematic block diagram of the server 200 and the robot 40. The robot 40b has a sensor unit 156, a processing unit 152, a control target 155, a communication unit 158, and a display unit 157. The server 200 has a processing unit 202, a storage unit 280, and a communication unit 208. The processing unit 202 includes an initial value setting unit 210, an external input data generating unit 230, a parameter processing unit 240, an action determining unit 250, a switching control unit 260, a recording control unit 270, and an output control unit 271. The storage unit 280 stores an action determining rule 282, definition information 284, a parameter initial value 286, the latest parameter 288, a recording format switching rule 290, and recorded data 292.

[0030] In the robot 40, the sensor unit 156 has sensors such as a microphone 161, a 2D camera 163, a 3D depth sensor 162, and a distance sensor 164. Each sensor of the sensor unit 156 continuously detects information. The sensor information detected by the sensor unit 156 is output to the processing unit 152. The 2D camera 163 is an example of an image sensor that continuously captures an object, and captures an object with visible light to generate video information. The 3D depth sensor 162 detects the contour of the object by continuously irradiating an infrared pattern and analyzing the infrared pattern from the infrared images continuously captured by the infrared camera. In addition, the sensor unit 156 may include various other sensors such as a clock, a gyro sensor, a touch sensor, a sensor for motor feedback, a sensor for detecting remaining battery capacity, an odor sensor, a temperature sensor, a humidity sensor, a GPS (Global Positioning System) sensor, etc.

[0031] The processing unit 152 is formed of a processor such as a CPU. The processing unit 152 transmits sensor information continuously detected by each sensor of the sensor unit 156 to the server 200 through the communication unit 158. The processing unit 152 also processes at least a part of the sensor information continuously detected by each sensor of the sensor unit 156 to generate information for recording. The processing unit 152 generates information in a first recording format or information in a second recording format having a larger amount of information than the information in the first recording format. The information in the first recording format is, for example, information in a high compression format, and the information in the second recording format is, for example, information in a low compression format. For example, the processing unit 152 generates shape data such as skeleton data of an object as information in the first recording format based on skeleton information continuously detected by the 3D depth sensor 162. The processing unit 152 also generates full HD video data and audio data from video information captured by the 2D camera 163 and audio information detected by the microphone 161. The full HD video data is an example of video data having more information than the shape data of an object.

[0032] The communication unit 158 ​​transmits the information in the first recording format or the information in the second recording format generated by the processing unit 152 to the server 200. In the server 200, the recording control unit 270 stores the information in the first recording format or the information in the second recording format received by the communication unit 208 from the robot 40 in the recording data 292. The recording control unit 270 stores the information received from each robot 40 in the recording data 292 in association with information identifying each robot 40.

[0033] Furthermore, in the robot 40, the communication unit 158 ​​acquires information stored in the recording data 292 from the server 200. The communication unit 158 ​​functions as a recorded information receiving unit that acquires information in the second recording format, including video data, recorded by the recording control unit 270. The processing unit 152 generates an image to be presented to the user 30, based on the video data included in the information in the second recording format received by the communication unit 158. The processing unit 152 functions as an image generating unit that generates an image to be presented to the user 30.

[0034] Furthermore, the communication unit 158 ​​receives operation information indicating the operation content from the server 200. The processing unit 152 controls the control target 155 based on the operation content received by the communication unit 158. The control target 155 includes a speaker, motors for driving each part of the robot 40 such as the limbs, a light emitting device, and the like. When information indicating the vocalization content is received from the server 200, the processing unit 152 outputs a sound from the speaker according to the received vocalization content. Furthermore, the processing unit 152 can control a part of the behavior of the robot 40 by controlling the drive motors of the limbs. Furthermore, the processing unit 152 can express a part of the emotion of the robot 40 by controlling these motors.

[0035] In the server 200, the communication unit 208 outputs the information received from the robot 40 to the processing unit 202. The initial value setting unit 210 stores the initial values ​​of the parameters indicating the initial state of the neural network received by the communication unit 208 in the parameter initial values ​​286 in the storage unit 280. Note that the initial values ​​of the parameters of the neural network may be predetermined in the server 200, or may be changeable by the user 30 via the communication network 90.

[0036] The external input data generating unit 230 processes at least a part of the sensor information received by the communication unit 208 to generate input information from outside the neural network, and outputs the generated information to the parameter processing unit 240. The parameter processing unit 240 performs calculations on the neural network based on the input information and current parameters 288 and definition information 284 of the neural network stored in the storage unit 280.

[0037] The artificial neurons of the neural network include a plurality of artificial neurons in which the situation of the robot 40 is defined, a plurality of emotion artificial neurons in which a plurality of emotions of the robot 40 itself is defined, and a plurality of endocrine artificial neurons in which the production state of the endocrine substance of the robot 40 itself is defined. The parameter processing unit 240 calculates parameters representing the internal states of the plurality of artificial neurons in the neural network based on the input information generated by the external input data generating unit 230. For example, the parameter processing unit 240 updates the parameters of the current internal states of the plurality of artificial neurons in which the situation of the robot 40 is defined, based on the input information generated by the external input data generating unit 230. In addition, the parameter processing unit 240 calculates parameters of the internal states of other artificial neurons in the neural network. As a result, for example, the parameters of the internal state of the emotion artificial neuron in which the emotion "happy" is defined are calculated. The parameters of the internal state of this emotion artificial neuron are an example of an index representing the strength of the emotion "happy". Therefore, the parameter processing unit 240 can determine the strength of the emotion in the control system based on the internal state of the emotion artificial neuron. In this way, the parameter processing unit 240 functions as an emotion determining unit that determines the intensity of an emotion using a neural network based on at least a part of the information detected by the sensor of the sensor unit 156.

[0038] The parameters of the neural network calculated by the parameter processing unit 240 are supplied to the switching control unit 260 and the operation determination unit 250. The switching control unit 260 determines the recording format of the information generated by the processing unit 152 of the robot 40 based on the parameters supplied from the parameter processing unit 240. When it is necessary to switch the recording format of the information generated by the processing unit 152, the switching control unit 260 causes the robot 40 to transmit an instruction to switch the recording format via the communication unit 208. In the robot 40, the processing unit 152 switches the recording format in accordance with the instruction received from the server 200.

[0039] For example, when the switching control unit 260 is causing the processing unit 152 to generate information in the first recording format, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format in response to an increase in the intensity of the emotion determined by the parameter processing unit 240. In this way, the switching control unit 260 switches the recording format of the information recorded by the recording control unit 270 from the first recording format to the second recording format in response to an increase in the intensity of the emotion determined by the parameter processing unit 240. This allows the information when the emotion of the robot 40 increases to be recorded in detail. In addition, in the robot 40, the processing unit 152 acquires the video data in the second recording format acquired from the server 200 and generates a video to be presented to the user 30. Therefore, the user 30 can enjoy the information when the emotion of the robot 40 increases as a video.

[0040] In addition, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format in response to an increase in the intensity of the emotion of the user 30. In this way, the switching control unit 260 switches the recording format of the information to be recorded by the recording control unit 270 from the first recording format to the second recording format in response to an increase in the intensity of the emotion determined by the parameter processing unit 240. This allows information when the emotion of the user 30 is increased to be recorded in detail. In addition, in the robot 40, the processing unit 152 acquires video data in the second recording format acquired from the server 200 and generates a video to be presented to the user 30. Therefore, the user 30 can enjoy information when his / her emotion is increased as a video. The emotion of the user can be detected, for example, based on a voice or facial expression detected from the user 30. In addition, the switching control unit 260 may recognize a change in the emotion of the user 30 based on the detection of an impact by a touch sensor (not shown) included in the sensor unit 156. For example, when an impact is detected by a touch sensor included in the sensor unit 156, the switching control unit 260 may recognize that the user 30 has become upset.

[0041] When the switching control unit 260 is causing the processing unit 152 to generate information in the second recording format, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the second recording format to the first recording format in response to a decrease in the intensity of the emotion of the robot 40 or the emotion of the user determined by the parameter processing unit 240. In this way, the switching control unit 260 switches the recording format of the information recorded by the recording control unit 270 from the second recording format to the first recording format in response to a decrease in the intensity of the emotion of the robot 40 or the emotion of the user determined by the parameter processing unit 240.

[0042] Furthermore, in response to an increase in the emotional change of the user 30 or the robot 40, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information generated by the processing unit 152 from the first recording format to the second recording format. This allows detailed information about when the emotion of the user 30 or the robot 40 increases to be recorded. Furthermore, in the robot 40, the processing unit 152 acquires the video data in the second recording format acquired from the server 200, and generates a video to be presented to the user 30. Thus, the user 30 can enjoy information about when the emotion of the user 30 or the robot 40 increases as a video.

[0043] When the switching control unit 260 is causing the processing unit 152 to generate information in the second recording format, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the second recording format to the first recording format in response to the change in the emotion of the robot 40 or the emotion of the user subsiding (the amount of change becomes equal to or less than the threshold) or the emotion of the robot 40 or the emotion of the user returning to the emotion before the change. In this way, the switching control unit 260 switches the recording format of the information recorded by the recording control unit 270 from the second recording format to the first recording format in response to the change in the emotion of the robot 40 or the emotion of the user subsiding.

[0044] The switching control unit 260 may also switch the recording format according to the continuity of changes in the emotions of the robot 40 or the user 30. For example, when the cumulative value of the emotion value within a predetermined time exceeds a threshold, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format. The switching control unit 260 may also switch the recording format according to a combination of the speed of the change (slope) of the emotion within a predetermined time or the degree of the slope. For example, when the slope indicating the change in the emotion value is a combination of a sudden rise → a gradual change for a certain period of time → a sudden drop within a predetermined time, the switching control unit 260 transmits to the robot 40 an instruction to switch the recording format of the information to be generated by the processing unit 152 from the first recording format to the second recording format. This makes it possible to record video data of a situation in which, for example, in a tense scene due to pressure or the like, the absolute value of the emotion is not relatively large but the time feels long (an emotion value above a certain value continues for a predetermined time).

[0045] Furthermore, when the recording control unit 270 switches the recording format of the information generated by the processing unit 152 from the first recording format to the second recording format, it transmits an instruction to the robot 40 to tag the information in the second recording format. This allows the user 30 to clearly recall the situation at the time from the tagged information when checking the recorded information. Furthermore, by tagging, the information can be easily found when searching for information on a specific situation from among the recorded information.

[0046] When outputting the recorded information in the second recording format, the output control unit 271 selects the information in the second recording format to be output based on the information tagged to the information in the second recording format. Specifically, the output control unit 271 selects the information in the second recording format in which the current situation of the robot 40 and the tagged information are related to each other as the output target.

[0047] For example, the output control unit 271 selects and outputs information in the second recording format in which the attributes of the people around and the attributes of the tagged person are related. Specifically, the output control unit 271 selects and outputs video data in which the ages of the people around and the ages of the tagged person match or are similar, and does not select (output) video data in which the ages of the people around and the ages of the tagged person do not match or are similar. In addition, the output control unit 271 selects and outputs video data in which the affiliation of the people around and the affiliation of the tagged person match or are similar. In this way, by selecting and choosing information to be output based on tagged information, it is possible to narrow down and output information that is likely to gain sympathy, and to avoid outputting information that is unlikely to gain sympathy.

[0048] Furthermore, even if there is information in the second recording format in which the attributes of people around and the attributes of the tagged person are related, if the emotions of the people around are negative (anxiety, sadness, anger, etc.), the output control unit 271 does not output (prohibits output of) the information in the second recording format. This makes it possible to refrain from outputting information in a situation in which the user is not emotionally receptive to the information to be output.

[0049] The action decision rule 282 defines an action to be performed by the robot 40 in association with the state of the robot 40. For example, the action decision rule 282 defines an action to be performed by the robot 40 in association with the internal state of an artificial neuron of a neural network. For example, the action decision rule 282 defines an action of vocalizing a word expressing happiness in association with the condition that an emotion artificial neuron defining the emotion of "happy" is high. The action decision rule 282 also defines an action to be taken when the robot becomes sleepy in association with the condition that an internal state of an endocrine artificial neuron defining an endocrine substance corresponding to sleepiness is high.

[0050] Incidentally, endocrine substances refer to substances such as neurotransmitters and hormones that are secreted in the body and transmit signals. Also, endocrine means that endocrine substances are secreted in the body. However, the endocrine substances of the robot 40 itself are one piece of information that affects the behavior of the robot 40, and do not mean that the robot 40 actually generates endocrine substances. Similarly, the emotions of the robot 40 itself are one piece of information that affects the behavior of the robot 40, and do not mean that the robot 40 actually has emotions.

[0051] The action determination unit 250 determines the action of the robot 40 based on the action defined in the action determination rules 282 in association with the activation state or internal state of each artificial neuron determined by the parameter processing unit 240. Action information indicating the action determined by the action determination unit 250 is transmitted from the communication unit 208 to the robot 40. In the robot 40, the processing unit 152 controls the control target 155 to cause the robot 40 to perform the action indicated by the information received from the server 200. This enables the robot 40 to perform an appropriate action according to the current emotion of the robot 40.

[0052] 3 is a schematic diagram of a neural network 300. The neural network 300 is an example neural network for explaining the operation of the parameter processing unit 240. The neural network 300 includes a plurality of artificial neurons including artificial neuron 1, artificial neuron 2, artificial neuron 3, artificial neuron 4, artificial neuron 5, artificial neuron 6, artificial neuron 7, artificial neuron 8, artificial neuron 9, artificial neuron a, artificial neuron b, and artificial neuron c. The neural network 300 includes a plurality of artificial synapses including artificial synapse 301, artificial synapse 302, artificial synapse 303, artificial synapse 304, artificial synapse 305, artificial synapse 306, artificial synapse 307, artificial synapse 308, artificial synapse 309, artificial synapse 310, artificial synapse 311, artificial synapse 312, artificial synapse 313, artificial synapse 314, artificial synapse 315, artificial synapse 316, artificial synapse 317, artificial synapse 318, and artificial synapse 319. The artificial neuron corresponds to a neuron in a living organism. The artificial synapse corresponds to a synapse in a living organism.

[0053] The artificial synapse 301 connects the artificial neuron 4 and the artificial neuron 1. The artificial synapse 301 is an artificial synapse that connects in one direction, as shown by the arrow of the artificial synapse 301. The artificial neuron 4 is an artificial neuron that is connected to the input of the artificial neuron 1. The artificial synapse 302 connects the artificial neuron 1 and the artificial neuron 2. The artificial synapse 302 is an artificial synapse that connects in two directions, as shown by the arrows on both ends of the artificial synapse 302. The artificial neuron 1 is an artificial neuron that is connected to the input of the artificial neuron 2. The artificial neuron 2 is an artificial neuron that is connected to the input of the artificial neuron 1.

[0054] In this embodiment, an artificial neuron may be represented by N, and an artificial synapse may be represented by S. When identifying each artificial neuron, a superscripted reference symbol may be used as an identification letter. When identifying an arbitrary artificial neuron, i or j may be used as an identification letter. For example, N i represents an arbitrary artificial neuron.

[0055] In addition, an artificial synapse may be identified by the identification numbers i and j of the two artificial neurons connected to the artificial synapse. For example, S 41 is N 1 and N 4 It represents an artificial synapse that connects ij is N i The output of N j represents an artificial synapse that inputs to S ji is N j The output of N i represents an artificial synapse input to

[0056] In FIG. 3, A to J indicate the definition of the state of the robot 40. The state of the robot 40 includes the emotion of the robot 40, the state of production of endocrine substances, the situation of the robot 40, etc. 4 , N 6 , and N 7 is a conceptual artificial neuron in which a concept that represents the situation of the robot 40 is defined. For example, N 4 is a conceptual artificial neuron that has been assigned the situation "the bell rang". 6 is a conceptual artificial neuron that has been assigned the state "charging has started". 7 is a conceptual artificial neuron to which the state “charge level is below the threshold” is assigned.

[0057] N 1 , N 3 , N b and N c is an emotion artificial neuron in which the emotion of the robot 40 is defined. 1 is an emotional artificial neuron to which the emotion "happy" is assigned.3 is an emotional artificial neuron to which the emotion "sad" is assigned. b is an emotional artificial neuron to which the emotion "fear" is assigned. c is an emotional artificial neuron to which the emotion “fun” is assigned.

[0058] N 2 , N 5 and N a is an endocrine artificial neuron with a defined endocrine state of the robot 40. 5 is an endocrine artificial neuron that is assigned the developmental state of dopamine. Dopamine is an example of an endocrine substance involved in the reward system. 5 is an example of an endocrine artificial neuron involved in the reward system. 2 is an endocrine artificial neuron that is assigned the generation state of serotonin. Serotonin is an example of an endocrine substance involved in the sleep system. 2 is an example of an endocrine artificial neuron involved in the sleep system. a is an endocrine artificial neuron that has been assigned the developmental state of noradrenaline. Noradrenaline is an example of an endocrine substance involved in the sympathetic nervous system. a is an endocrine artificial neuron involved in the sympathetic nervous system.

[0059] The definition information 284 in the storage unit 280 stores information defining the state of the robot 40 as described above for each of the multiple artificial neurons that make up the neural network. Thus, the neural network 300 includes conceptual artificial neurons, emotional artificial neurons, and endocrine artificial neurons. Conceptual artificial neurons, emotional artificial neurons, and endocrine artificial neurons are artificial neurons in which the meanings of concepts, emotions, endocrine, etc. are explicitly defined. In contrast, N 8 Or N 9 is an artificial neuron with no defined state for the robot 40. 8 Or N 9is an artificial neuron whose meanings, such as concepts, emotions, and endocrine functions, are not explicitly defined.

[0060] The parameters of the neural network 300 are i I, which is the input to t i and N from outside the neural network i E, which is the input to t i And, N i The parameters of and S i and parameters.

[0061] N i The parameters of N i S stands for status t i And, N i represents the internal state of the artificial neuron, V i m t And, N i T represents the threshold for firing i t And, N i The time when the last firing occurred is t f and the artificial neuron N at the final firing time i V represents the internal state of i m tf and the output increase / decrease parameter a t i , b t i , h t i The output increase / decrease parameter is an example of a parameter that determines the time evolution of the output when the artificial neuron fires. In this embodiment, the subscript t indicates that the parameter can be updated as time progresses. i m t is information corresponding to the membrane potential of the artificial neuron, and is an example of a parameter representing the internal state or output of the artificial neuron.

[0062] S ij The parameters of S ijBS represents the coupling coefficient of the artificial synapse t ij And, S. ij N connected i and N j The time when the two last fired simultaneously is t cf and BS, which represents the coupling coefficient at the time of the last simultaneous firing. ij tcf and a, which is the coupling coefficient increase / decrease parameter t ij , b t ij , h t ij The increase / decrease parameter of the coupling coefficient is an example of a parameter that determines the time evolution of the coupling coefficient after the two artificial neurons connected by the artificial synapse last fired simultaneously.

[0063] The parameter processing unit 240 updates the above-mentioned parameters based on the input from the external input data generating unit 230 and the neural network to determine the activation state of each artificial neuron. The motion determining unit 250 determines the motion of the robot 40 based on the internal state or activation state of at least some of the artificial neurons determined by the parameter values ​​of at least some of the artificial neurons among the multiple artificial neurons in the neural network, and the state defined for at least some of the artificial neurons by the definition information 284. Note that the activation state may be an activated state or an inactivated state. In this embodiment, activation may be referred to as "firing", and inactivation may be referred to as "not firing". Note that, as will be described later, the "firing" state is divided into an "upward phase" and a "downward phase" depending on whether the internal state is rising or not. The "not firing" and the "upward phase" and "downward phase" are different from each other. " is represented by the status Sti.

[0064] Figure 4 shows the parameters of the neural network in a schematic table format. Each neuron N is assigned a threshold T t and the increase / decrease parameter h t , a t and b tEach artificial synapse has a coupling coefficient BS t and the increase / decrease parameter h t , a t and b t The parameters are as follows: i Every time, an artificial synapse is used, i In each row, the parameters of all the artificial neurons directly connected to the corresponding artificial synapse are shown, as well as the parameters of the corresponding artificial synapse.

[0065] 5 shows an outline of the operation flow of the server 200 when the robot 40 is started or reset. When the server 200 receives a notification that the robot 40 has been started or reset, the parameter processing unit 240 performs initial setting of the parameters of the neural network. For example, the parameter processing unit 240 acquires initial values ​​of the parameters from the storage unit 280 and generates parameter data of the neural network in a predetermined data structure (S502). In addition, the parameter processing unit 240 sets the values ​​of the parameters of the neural network at time t0. When the initial setting is completed, in S504, a loop for time t is started.

[0066] In S510, the parameter processing unit 240 calculates the time step t n+1 Calculate the parameters corresponding to the changes due to the electrical influence of the artificial synapse in . t ij Calculate.

[0067] In S520, the parameter processing unit 240 calculates the time step t n+1 Calculate the parameters corresponding to the changes due to the chemical effects of endocrine substances in N i and S ij More specifically, we calculate the change in the parameters of n+1 Artificial neuron N affected by endocrine artificial neuron i The internal state of the S ijCalculate the coupling coefficient increase / decrease parameters and coupling coefficients.

[0068] In S530, the parameter processing unit 240 acquires an input from outside the neural network. Specifically, the parameter processing unit 240 acquires the output of the external input data generating unit 230.

[0069] In S540, the parameter processing unit 240 calculates the time step t n+1 In N i Calculate the internal state of V i m tn+1 and Status S tt i Then, in S550, at time t n+1 The values ​​of the parameters at time t are stored in the parameter 288 of the storage unit 280. n+1 The value of each parameter in is output to the operation determination section 250 and the switching control section 260.

[0070] In S560, the switching control unit 260 n+1 N in i At time step t n+1 N in i If the parameters satisfy the switching conditions for the recording format, the switching control unit 260 instructs the robot 40 to switch the recording format (S570). Next, when switching from the first recording format to the second recording format, the recording control unit 270 instructs tagging of information in the second recording format (S580) and proceeds to S506. On the other hand, in S560, n+1 N in i If the parameters do not satisfy the conditions for switching the recording format, the process proceeds to S506.

[0071] In S506, the parameter processing unit 240 judges whether or not to end the loop. For example, it judges to end the loop when the time represented by the time step reaches a predetermined time or when sensor information from the robot 40 is not received for a predetermined time. If the loop is not to be ended, the process returns to S510 and the calculation of the next time step is further performed. If the loop is to be ended, this flow is ended.

[0072] FIG. 6 is a diagram for explaining the calculation of the coupling coefficient of the artificial synapse. Here, the constant a is used as the initial value of the increase / decrease parameter. ij and b ij This section describes the cases where is defined.

[0073] time t n At the time step of S ij N on both ends of i and N j are firing, the parameter processing unit 240 n+1 in B.S. tn+1 ij ,BS tn+1 ij =BS tn ij +a tn ij ×(t n+1 -t n ) is calculated. On the other hand, at time t n At time steps S i and S j If none of them fires at time t n+1 Coupling coefficient BS tn+1 ij ,BS tn+1 ij =BS tn ij +b tn ij ×(t n+1 -t n ) is calculated. Also, BS tn+1 ij If is negative, BS tn+1 ij is set to 0. Note that BS ij is a positive value of Sij So, a t ij is a positive value, and b t ij is a negative value. ij S is negative ij So, a t ij is a positive value, and b t ij is a negative value.

[0074] As shown in Figure 6, at time t0, the artificial neurons at both ends fire simultaneously. t ij is a per unit time t0 ij In addition, since there are no simultaneous firings at time t1, BS t ij is |b per unit time t1 ij In addition, due to simultaneous firing at time t4, BS t ij is a per unit time t4 ij increases with.

[0075] Figure 7 shows the function h t ij The time evolution of the coupling coefficient when is defined is shown below. t ij t cf The time elapsed from Δt (= tt cf ) ≧0. t ij is a function of at least Δt and takes real values.

[0076] The function 700 shown in FIG. t ij The function 700 is an example of cf Coupling coefficient BS tcf ijand Δt. The function 700 monotonically increases when Δt is less than a predetermined value, and monotonically decreases and gradually decreases toward 0 when Δt is greater than the predetermined value. The function 700 has a value BS tcf ij Take.

[0077] FIG. 7 shows that a function 700 is defined as a parameter for increasing or decreasing the coupling coefficient. i and N j The parameter processing unit 240 calculates the coupling coefficients of BSs at each time t1 to t6 based on the function 700 and Δt. t ij In the time range from time t1 to time t6, N i and N j do not fire simultaneously. Therefore, for example, after time t2, the coupling coefficient decreases monotonically.

[0078] Figure 8 shows N i and N j The time evolution of the coupling coefficient when the neurons fire simultaneously is shown in Fig. The coupling coefficient is calculated in the same manner as in FIG. 7 from time t0 to time t2. At time t2, N i and N j When these two fire simultaneously, the parameter processor 240 t ij (t-t2,BS t2 ij ) and calculates the coupling coefficient at each of times t3 to t6. In this way, the coupling coefficient increases each time simultaneous firing is repeated. This has the effect of strengthening the artificial synaptic connection, as in Hebb's law in living organisms. On the other hand, as shown in Figures 6 and 7, when the time without simultaneous firing becomes long, it has the effect of weakening the artificial synaptic connection.

[0079] Fig. 9 shows a schematic diagram of effect definition information that defines a chemical effect given to a parameter. This effect definition information is used to calculate a change in the parameter in S520 of Fig. 5. The definition information includes a condition regarding the internal state of the endocrine artificial neuron, information specifying the artificial neuron or artificial synapse that exerts an effect, and an equation that determines the effect content.

[0080] In the example of FIG. 9, the endocrine artificial neuron N 2 is an endocrine artificial neuron to which the endocrine substance responsible for sleepiness is assigned. Endocrine artificial neuron N 2 For definition information about VM tn 2 >T tn 2 "Conditions for Endocrine Artificial Neuron N 2 The artificial neuron N 1 and N 3 ", and "T tn+1 i =T tn i Parameter processing unit 240 determines Vm tn 2 T tn 2 If it exceeds t n+1 Emotional Artificial Neuron N 1 and N 3 The threshold is increased by 10%. This makes it difficult for the emotional artificial neuron to fire when drowsiness occurs. For example, the conceptual artificial neuron N is defined as "the amount of stored power is below the threshold". 7 The output of the endocrine artificial neuron N 2 By defining a neural network connected to the input of the above, it is possible to embody the phenomenon in which emotions become less likely to increase as the battery level becomes low.

[0081] In addition, the endocrine artificial neuron N 5 is an endocrine artificial neuron to which dopamine is assigned. Endocrine artificial neuron N 5 The first definition information for "Vm tn5 >T tn 5 and Vm tn 4 >T tn 4 "Conditions for Endocrine Artificial Neuron N 5 As an artificial synapse that is influenced by 49 and S 95 ", and "a" is the formula that determines the effect. tn+1 ij =a tn ij The formula "Vm × 1.1" is defined. tn 5 T tn 5 Exceeds Vm tn 4 T tn 4 If it exceeds t n+1 Artificial Synapse S 49 and S 95 Increases the increase / decrease parameters by 10%.

[0082] This allows the concept of artificial neuron N to be defined as the situation in which the reward system's endocrine artificial neuron fires, resulting in the "bell ringing" condition. 4 is firing, the implicit artificial neuron N 9 Conceptual artificial neuron N via 4 and N 5 This allows the binding of the artificial neuron N to the reward system when the "bell" is rung. 5 becomes more likely to ignite.

[0083] In addition, the endocrine artificial neuron N 5 The second definition of "Vm tn 5 >T tn 5 "Conditions for Endocrine Artificial Neuron N 5 The artificial neuron that is influenced by 1 ", and "T tn+1 i =T tn iThe formula "Vm × 1.1" is defined. tn 5 T tn 5 If it exceeds t n+1 Artificial neuron N 1 This reduces the increase / decrease parameter of the reward system endocrine artificial neuron N 5 When this emotion is ignited, it becomes easier for the emotion of happiness to be ignited.

[0084] According to the definition of the effect of such a reward system on endocrine artificial neurons, it is possible to implement an implementation in which, if the action of charging the robot 40 while ringing a bell is repeated, the robot 40 will behave in a way that expresses happiness just by ringing the bell.

[0085] The influence definition information is not limited to the example of FIG. 9. For example, a condition may be defined that the internal state of the artificial neuron is equal to or lower than a threshold. A condition regarding the status of the artificial neuron, for example, a condition regarding an ascending phase, a descending phase, or non-firing, may be defined. The influence range may be defined by directly specifying an artificial neuron or an artificial synapse, or by defining "all artificial synapses connected to a specific artificial neuron." Regarding the influence formula, in the case where the target is an artificial neuron, in addition to multiplying the threshold by a constant, a formula may be defined in which a constant is added to the threshold or an increase / decrease parameter of the internal state is multiplied by a constant. In the case where the target is an artificial synapse, in addition to multiplying the increase / decrease parameter by a constant, an formula may be defined in which a coupling coefficient is multiplied by a constant.

[0086] The influence definition information is stored in the definition information 284 of the storage unit 280. Thus, the storage unit 280 stores influence definition information that defines the influence of at least one of the internal state and firing state of the endocrine artificial neuron on the parameters of at least one of the other artificial neurons and artificial synapses that are not directly connected to the endocrine artificial neuron via an artificial synapse. Then, the parameter processing unit 240 updates the parameters of at least one of the other artificial neurons and artificial synapses that are not directly connected to the endocrine artificial neuron via an artificial synapse based on at least one of the internal state and firing state of the endocrine artificial neuron and the influence definition information. Furthermore, the parameters of the other artificial neurons that are influenced by at least one of the internal state and firing state of the endocrine artificial neuron can include at least one of the parameters that define the threshold value, firing state, and time development of the output at the time of firing of the other artificial neuron. Furthermore, the parameters of the artificial synapse influenced by at least one of the internal state and the firing state of the endocrine artificial neuron can include at least one of the coupling coefficient of the artificial synapse and a parameter that determines the time evolution of the coupling coefficient after the two artificial neurons linked by the artificial synapse last fired simultaneously. Furthermore, the influence definition information includes information that determines the influence that the firing state of the endocrine artificial neuron associated with the reward system has on the threshold of the emotional artificial neuron, and the parameter processing unit 240 updates the threshold of the emotional artificial neuron in accordance with the influence definition information when the endocrine artificial neuron fires.

[0087] Figure 10 shows the V tn+1 i and S tn+1 i 5. The process of this flowchart can be applied to a part of the process in S540 in FIG. 5. In S1100, the parameter processing unit 240 calculates S tn i indicates non-firing.

[0088] S tn i If indicates non-firing, the parameter processing unit 240 i Input to Itn+1 i Specifically, when the input from outside the neural network is N i If not connected to I tn+1 i =Σ j B.S. tn+1 ji ×VM tn j ×f(S tn j ) is calculated by the following formula: i If connected to I tn+1 i =Σ j B.S. tn+1 ji × VM tn j ×f(S tn j )+E tn+1 i Here, E tn i is the time t from outside the neural network. n This is the input in

[0089] In addition, f(S) returns 0 when S is a value that indicates no firing, and returns 1 when S is a value that indicates an ascending or descending phase. This model corresponds to a model in which a synapse transmits an action potential only when a neuron fires. Note that f(S)=1 may also be returned. This corresponds to a model in which a membrane potential is transmitted regardless of the firing state of the neuron.

[0090] In S1112, the parameter processing unit 240 tn+1 i T tn+1 i Determine whether it exceeds I tn+1 i T tn+1 i If it exceeds Vm tn+1 i is calculated based on the increase / decrease parameters, and Vm tn+1 i Depending on S tn+1i is set to a value indicating the rising phase or the falling phase (S1114), and this flow ends.

[0091] In the S1100, tn i When Vm is in the rising phase or the falling phase, the parameter processing unit 240 tn+1 i Then, the parameter processing unit 240 calculates t n+1 By Vm t i When Vmin is reached, S tn+1 i Set to the unfired value and t n+1 By Vm t i If Vmin is not reached, S tn+1 i The parameter processing unit 240 sets t n+i By Vm t i When Vmax is reached, S tn+1 i Set the value of the falling phase to t n+1 By Vm t i If Vmax is not reached, S tn+1 i Set the value of the rising phase to

[0092] In this way, N i If is firing, even if the output falls below the threshold, N i The output of is independent of the input. Such a period corresponds to the absolute refractory period in biological neurons.

[0093] Figure 11 shows the N i V when does not ignite t i FIG. 13 is a diagram for explaining an example of calculation of

[0094] At time step t0, N i is not fired. I at time t1 t1 i Tt1 i If V at time t1 is equal to or less than V t1 i , V t1 i =I t1 i Calculate V from time t0 to t1. t i , V t i =I t0 i Similarly, the parameter processing unit 240 calculates the time step t n The calculated V t n maintains the value of until the next time step, and V tn+1 In I tn+1 Change to.

[0095] Figure 12 shows the N i V when ignition occurs i t FIG. 12 is a diagram for explaining an example of calculation of the constant a i and b i This is a calculation example when is defined.

[0096] At time step t0, N i is not fired. I at time t1 t1 i T t1 i If the parameter processing unit 240 exceeds V t1 i , V t1 i =I t1 i Calculate V from time t0 to t1. t i , V t i =I t0 i Here, the calculation is performed using I at time t1. t1 i is less than Vmax. I at time t1 t1 iIf exceeds Vmax, I t1 i =Vmax.

[0097] As shown in FIG. 12, after time t1, the parameter processing unit 240 t i Until the time when V reaches Vmax, t i per unit time t ij In addition, the parameter processing unit 240 increases N i Status S t i is determined to be the ascending phase.

[0098] Also, V t i When V reaches Vmax, V t i Until Vmin is reached, V t i per unit time |b t i In addition, the parameter processing unit 240 reduces N i The status of V is determined to be in the descending phase. t i When Vmin is reached, V at the next time t6 i , V t6 i =I t6 i Calculate by V t i After reaching Vmin, the status is determined as non-firing.

[0099] In addition, N i If the status of is in the declining phase, the calculated Vm t i T t i Even if it falls below Vm t i I t i The parameter processing unit 240 does not depend on Vm t i Tt i Even if it falls below Vm t i Vm increases or decreases according to the parameter until it reaches Vmin. t i Calculate.

[0100] Figure 13 shows the N i As a parameter of the function h t i The time evolution of the coupling coefficient when is defined is shown below. In general, h t i is the firing time t f The time elapsed from Δt (= tt f ) ≧0. t i is at least a function of Δt. t i takes real values, and h t i The value range is from Vmin to Vmax.

[0101] The function 1300 shown in FIG. t i The function 1300 is an example of f In tf i and Δt. The function 1300 increases monotonically when Δt is less than a predetermined value, and decreases monotonically when Δt is greater than a predetermined value. The function 1300 has a value Vm tf i Take.

[0102] FIG. 13 shows a case where a function 1400 is defined as an increase / decrease parameter of the internal state, and N i The parameter processing unit 240 calculates the output when the function 1400, Δt and Vm f i Based on this, Vmti is calculated for each time from time t1 to time t5. t i Since Vmin is reached at time t5, Vm i =I t6i It becomes.

[0103] FIG. 14 shows an example of a rule 1400 stored in the recording format switching rule 290 in the form of a table. The rule 1400 is 1 , N 3 , N b and N c When at least the first condition is satisfied that any one of the Vmti exceeds a threshold, the information recording format is "switched to a low compression format". In this way, when information is recorded in a high compression format, the switching control unit 260 determines to switch the information recording format to a low compression format when the first condition is satisfied from a state in which the first condition is not satisfied. Note that, as the threshold, each of N j The threshold value may be higher than Tit.

[0104] Also, rule 1400 states that 5 and N a VM t i If the second condition is met, that is, the total value of the data exceeds the threshold, the data recording format is switched to a low compression format. " is set. In this way, when information is recorded in a high compression format, the switching control unit 260 determines to switch the information recording format to a low compression format when the second condition is satisfied from a state in which the second condition is not satisfied. j The total value of Vmax of each is multiplied by a constant 0.9. j T i t may be higher than the sum of the above.

[0105] N 1 , N 3 , N b and N care emotion artificial neurons in which emotions "happy", "sad", "fearful" and "fun" are defined, respectively. Therefore, the parameter processing unit 240 determines the intensity of the emotion based on the internal state of the emotion artificial neuron, and when the determined intensity of the emotion exceeds a predetermined threshold, the recording format can be switched to a low compression format.

[0106] N5 and Na are endocrine artificial neurons in which the endocrine substances "dopamine" and "noradrenaline" are defined, respectively. The sum of the parameters of the internal states of these endocrine artificial neurons is an example of an index representing the intensity of the emotion of "excitement." Therefore, the parameter processing unit 240 determines the intensity of the emotion based on the internal states of the endocrine artificial neurons, and when the determined intensity of the emotion exceeds a predetermined threshold, the recording format can be switched to a low compression format.

[0107] Also, rule 1400 states that 1 , N 3 , N b and N c Any VM t i is also equal to or smaller than the first threshold, and N 5 and N a VM t i When a third condition is satisfied, that is, the sum of the first and second thresholds is equal to or less than the second threshold, the operation of "switching the data recording format to a high compression format" is defined. Therefore, when information is recorded in a low compression format, the switching control unit 260 determines to switch the information recording format to a high compression format when the third condition is satisfied from a state in which the third condition is not satisfied. In this way, the recording format can be switched to a high compression format in response to the intensity of the emotion becoming equal to or less than the predetermined threshold.

[0108] The first threshold of the third condition is j The second threshold value of the third condition is the value obtained by multiplying the Vmax of each N jThe value is obtained by multiplying the total value of Vmax of the three conditions by the constant 0.8. In this manner, the case is exemplified in which the first threshold value of the third condition is smaller than the threshold value of the first condition, and the second threshold value of the third condition is smaller than the threshold value of the second condition. However, the first threshold value may be the same as the threshold value of the first condition, and the second threshold value may be the same as the threshold value of the second condition. Also, the first threshold value of the third condition is set to a value obtained by multiplying the total value of Vmax of the three conditions by the constant 0.8. j T i t The second threshold of the third condition may be higher than each N j T i t In addition, the threshold value for each condition is not limited to these examples, and various values ​​can be applied.

[0109] According to the system 20, the robot 40 continuously transmits highly compressed information such as skeletal data to the server 200 during a period when the emotion of the robot 40 is not significantly high, and the server 200 records the information. The continuous information such as skeletal data recorded in the server 200 can be used when analyzing the memory of the robot 40. When the emotion of the robot 40 significantly increases, the robot 40 starts transmitting full HD video data and audio data, and during a period when the emotion is strong enough to exceed a certain value, the server 200 records low-compression information including the full HD video data and audio data in addition to the skeletal data. When the robot 40 is requested by the user 30 to provide a video of the robot 40's memories, the robot 40 requests the server 200 to transmit the full HD video data and audio data, and provides the user 30 with the video data and audio data received from the server 200.

[0110] In this way, according to the system 20, high-quality video data of scenes in which the robot 40 feels strong emotions can be stored in the server 200. On the other hand, when the robot 40 does not feel strong emotions, simplified information such as skeletal data can be stored in the server 200. In this way, the robot 40, like a human being, can retain clear memories of times when it felt strong emotions, while retaining simplified memories of times when it did not feel strong emotions.

[0111] In this embodiment, "happy", "sad", "fearful", "fun" and "excited" are described as emotions, but the emotions handled by the system 20 are not limited to these. In addition, in this embodiment, "dopamine", "serotonin" and "noradrenaline" are described as endocrine substances, but the endocrine substances handled by the system 20 are not limited to these.

[0112] The functions of the server 200 may be implemented by one or more computers. At least some of the functions of the server 200 may be implemented by a virtual machine. At least some of the functions of the server 200 may be implemented in a cloud. Among the functions of the server 200, the functions of the components other than the storage unit 280 can be realized by a CPU operating based on a program. For example, at least some of the processing described as the operation of the server 200 can be realized by a processor controlling each piece of hardware (e.g., a hard disk, a memory, etc.) possessed by the computer according to a program. In this way, at least some of the processing of the server 200 can be realized by the processor operating according to a program to control each piece of hardware, so that each piece of hardware including the processor, the hard disk, the memory, etc. and the program operate in cooperation with each other. That is, the program can make the computer function as each component of the server 200. Similarly, among the components of the robot 40, the functions of the components other than the control target 155 and the sensor unit 156 can be realized by a CPU operating based on a program. That is, the program can make the computer function as each component of the robot 40. The computer may load a program that controls the execution of the above-mentioned processing, and operate according to the loaded program to execute the processing. The computer can read the program from a computer-readable recording medium that stores the program. The program can also be supplied to the computer through a communication line, and the computer can read the program through the communication line. The program may be read from a source connected via a communication line.

[0113] In the embodiment described above, the server 200 separate from the robot 40 is responsible for processing the neural network. Also, the server 200 separate from the robot 40 stores information such as video data. However, the robot 40 itself may perform the functions of the server 200, such as processing the neural network. Also, the robot 40 itself may store information such as video data. Also, the robot 40 is an example of a device to be controlled by the server 200. The device to be controlled is not limited to the robot 40, and various devices such as home appliances, vehicles, toys, etc. can be applied as the controlled devices.

[0114] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0115] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order. [Explanation of symbols]

[0116] 20 Systems 30 users 40 Robot 90 Communication Network 152 Processing section 155 Control Object 156 Sensor section 158 Communications Department 161 Mike 162 3D Depth Sensor 163 2D Camera 164 Distance Sensor 200 Servers 202 Processing section 208 Communications Department 210 Initial value setting section 230 External input data generation unit 240 Parameter Processing Section 250 Action Decision Unit 260 Switching control section 270 Recording control section 271 Output control section 280 Storage Unit 282 Action Decision Rules 284 Definition information 286 Parameter Initial Values 288 Parameters 290 Recording format switching rules 292 Recorded Data 300 Neural Networks

Claims

1. A control system for controlling the behavior of a robot, comprising: a recording control unit that processes at least a part of the information continuously detected by the sensor to generate information in a first recording format or information in a second recording format having a larger amount of information than the information in the first recording format, and records the information in the second recording format by tagging it; an output control unit that controls to output information in the second recording format selected from the information in the second recording format recorded by the recording control unit based on tagged information; A control system comprising:

2. the tagged information includes a status of the robot; The output control unit is and controlling the selection and output of information in the second recording format that is related to the current status of the robot and the status of the robot included in the tagged information. The control system of claim 1 .

3. The status of the robot is as follows: including attributes of people around the robot, The output control unit is and controlling the robot to select and output information in the second recording format in which attributes of people currently around the robot are related to attributes of the people included in the tagged information. The control system of claim 2.

4. The attributes include age; The output control unit is and controlling the robot to select and output information in the second recording format in which the age of a person currently around the robot matches or is similar to the age of the person included in the tagged information. The control system of claim 3.

5. The attributes include affiliation; The output control unit is and controlling the selection and output of information in the second recording format in which the affiliation of a person currently around the robot matches or is similar to the affiliation of the person included in the tagged information. The control system of claim 3.

6. The output control unit is When there is information in the second recording format in which an attribute of a person currently around the robot is related to an attribute of the person included in the tagged information, and when the emotion of the person currently around the robot is a negative emotion, control is performed so as to prohibit output of the information in the second recording format. The control system of claim 3.

7. The tagged information is Includes at least one of the following: surrounding information, smell, temperature, humidity, location information, weather, number of people around, facial expressions, and emotions The control system of claim 1 .

8. A control system for controlling the behavior of a robot, comprising: a recording control unit that processes at least a part of the information continuously detected by the sensor to generate information in a first recording format or information in a second recording format having a larger amount of information than the information in the first recording format, and records the information in the second recording format by tagging it; an output control unit that controls to output information in the second recording format selected from the information in the second recording format recorded by the recording control unit based on tagged information; and a robot that outputs information in the second recording format selected by the output control unit of the server; A system comprising:

9. A program executed by a control system that controls the behavior of a robot, a recording control step of processing at least a part of the information continuously detected by the sensor to generate information in a first recording format or information in a second recording format having a larger amount of information than the information in the first recording format, and tagging and recording the information in the second recording format; an output control step of controlling to output information in the second recording format selected based on tagged information from the information in the second recording format recorded by the recording control step; A program that causes a computer to execute the following.

Images