NN growth device, information processing device, neural network information production method, and program

Abstract

[Problem] It has not beem possible to simulate growth of the brain of an infant. [Solution] The growth of the brain of an infant can be simulated by an NN growth device 1 comprising: a start point storage unit 111 that stores one or more pieces of firing start point information including an information identifier for feature information of image information and one or more node identifiers; a goal storage unit 112 that stores goal information which identifies a goal corresponding to each of two or more states; a state determination unit 131 that determines one state using received sound information; a feature acquisition unit 132 that acquires feature information from received image information; a firing node determination unit 133 that determines, from the start point storage unit 111, a node identifier corresponding to each piece of feature information, and determines the node identifier of a node that fires and is linked to nodes identified by the node identifiers; and a growth unit 134 that acquires goal information forming a pair with the one state, and performs processing for growing edge information or node information corresponding to the one or more node identifiers determined by the firing node determination unit 133.

Description

[Technical Field]

[0001] The present invention relates to a neural network growth device, which is a device that virtually realizes the mechanism of brain growth. [Background technology]

[0002] BACKGROUND ART Conventionally, there have been electroencephalogram signal processing devices that acquire and process electroencephalogram signals that represent the electroencephalograms of a subject (see, for example, Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-47239 [Patent Document 2] Japanese Patent Application Laid-Open No. 2016-52430 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional techniques have not been able to simulate the development of an infant's brain. [Means for solving the problem]

[0005] The NN growing device of the first invention includes an NN storage unit that stores neural network information having two or more pieces of node information each having a node identifier and one or more pieces of edge information each having an edge identifier and specifying a connection between the nodes; a start point storage unit that stores one or more pieces of firing start point information each having an information identifier that identifies feature information of image information and one or more node identifiers that identify a node that will fire first when the feature information is received; a goal storage unit that stores goal information that specifies a goal corresponding to each of two or more states including positive and negative; an information receiving unit that receives image information and sound information; a state determination unit that determines one state from two or more states using the sound information received by the information receiving unit; a feature acquisition unit that acquires one or more feature information for the image information using the image information received by the feature acquisition unit; an ignition node determination unit that determines, from the starting point storage unit, a node identifier of a node to be ignited, which is a node identifier corresponding to each of the one or more feature information acquired by the feature acquisition unit, and determines the node identifier of a node to be ignited, which is a node connected by an edge to the nodes identified by the one or more node identifiers and to which the feature information is passed; and a growth unit that acquires goal information that pairs with one state determined by the state determination unit, and performs processing to grow, using the goal information, node information or edge information corresponding to each of the one or more node identifiers determined by the ignition node determination unit.

[0006] With this configuration, it is possible to realize a model of infant brain growth for simulating the growth of an infant brain.

[0007] Furthermore, compared to the first invention, the NN growth device of the second invention further includes a window determination unit that sequentially determines sliding windows, which are partial areas of a still image, from the still image contained in the image information received by the information receiving unit, and the feature acquisition unit is an NN growth device in which the window determination unit sequentially uses the partial images corresponding to the sliding windows determined to acquire one or more pieces of feature information for the partial images.

[0008] This configuration makes it possible to realize a model of infant brain growth that more accurately simulates the growth of an infant's brain.

[0009] Furthermore, the NN growth device of the third invention is an NN growth device in which, compared to the first or second invention, the feature information has an information identifier that identifies the information and an information amount that indicates the size of the information, and the firing node determination unit determines whether one or more pieces of feature information passed from one or more other nodes connected by edges satisfy firing conditions for one or more pieces of feature information, and determines the node identifier of the node that is determined to satisfy the firing condition.

[0010] With this configuration, it is possible to realize a model of infant brain growth for simulating the growth of an infant brain.

[0011] Furthermore, the NN growth device of the fourth invention is an NN growth device in which, compared to any of the first to third inventions, the node information has node position information that identifies the position of the node, the goal information has goal position information that identifies the position of the goal or goal direction information that indicates the direction of the goal, and the growth unit performs an edge generation process to generate and accumulate edge information for edges extending from nodes identified by one or more node identifiers out of one or more node identifiers determined by the firing node determination unit in the direction indicated by the goal information that pairs with a state determined by the state determination unit.

[0012] With this configuration, it is possible to realize a model of infant brain growth for simulating the growth of an infant brain.

[0013] Furthermore, the NN growth device of the fifth invention is an NN growth device in which, compared to the fourth invention, the firing node determination unit stores frequency information relating to the number of times of the determined node identifier in association with the node identifier, and the growth unit performs edge generation processing on the node identified by the node identifier corresponding to the frequency information that meets the edge generation condition.

[0014] With this configuration, it is possible to realize a model of infant brain growth for simulating the growth of an infant brain.

[0015] Furthermore, the NN growth device of the sixth invention is an NN growth device in which, compared to any of the first to third inventions, the node information has node position information that identifies the position of the node, the goal information has goal position information that identifies the position of the goal or goal direction information that indicates the direction of the goal, and the growth unit performs an edge growth process to acquire and store edge information obtained by growing edges extending from nodes identified by one or more node identifiers out of one or more node identifiers determined by the firing node determination unit in the direction indicated by the goal information that pairs with a state determined by the state determination unit.

[0016] With this configuration, it is possible to realize a model of infant brain growth for simulating the growth of an infant brain.

[0017] Furthermore, the NN growth device of the seventh invention is an NN growth device in which, compared to the sixth invention, the firing node determination unit stores frequency information relating to the number of times of the determined node identifier in correspondence with the node identifier, and the growth unit performs edge growth processing on the node identified by the node identifier corresponding to the frequency information that meets the edge generation condition.

[0018] With this configuration, it is possible to realize a model of infant brain growth for simulating the growth of an infant brain.

[0019] Furthermore, the NN growth device of the eighth invention is an NN growth device in which, compared to any one of the first to seventh inventions, the nodes are somas, the edges have AXONs and Dendrites, and the edge information has AXON information having an AXON identifier and AXON position information indicating the position of the AXON, and Dendrites information having a Dendrites identifier and Dendrites position information indicating the position of the Dendrites.

[0020] With this configuration, it is possible to realize a model of infant brain growth for simulating the growth of an infant brain.

[0021] a feature acquisition unit that acquires one or more feature information corresponding to the received information received by the information receiving unit; an information transmission unit that determines a node identifier of a node to be fired, the node identifier corresponding to each of the one or more feature information acquired by the feature acquisition unit, from an origin storage unit that stores one or more firing origin information having an information identifier that identifies the feature information of the received information and one or more node identifiers that identify a node that will be fired first when the feature information is received; and determines a node identifier of a node to be fired, the node identifier being a node that is connected by an edge to each of the nodes identified by the one or more node identifiers and that receives the feature information; a firing pattern acquisition unit that acquires a firing pattern using the one or more node identifiers determined by the information transmission unit; an output information acquisition unit that acquires output information corresponding to the firing pattern acquired by the firing pattern acquisition unit; and an information output unit that outputs the output information.

[0022] This configuration allows the operation of a baby's developing brain to be simulated.

[0023] Furthermore, the information processing device of the tenth invention is an information processing device that, compared to the ninth invention, further includes a temperature receiving unit that receives temperature information, and the information transmission unit performs an information transmission process that is a process of passing feature information corresponding to the fired node from the fired node to the next node that will fire, and changes the processing time for performing the information transmission process depending on the temperature information received by the temperature receiving unit.

[0024] This configuration allows the operation of a baby's developing brain to be simulated. [Effects of the Invention]

[0025] The neural network growing device according to the present invention can simulate the growth of an infant's brain. [Brief explanation of the drawings]

[0026] [Figure 1] Block diagram of NN growth apparatus 1 according to embodiment 1 [Figure 2] A flowchart illustrating an example of the operation of the NN growth device 1. [Figure 3] Flowchart illustrating an example of the growth process [Figure 4] A flowchart illustrating an example of the single-node process. [Figure 5] Flowchart illustrating an example of the node generation process [Figure 6] Flowchart illustrating an example of node information generation processing [Figure 7] A flowchart illustrating an example of the edge generation process [Figure 8] A flowchart illustrating an example of the edge information generation process [Figure 9] A flowchart illustrating an example of the edge growth process. [Figure 10] 10 is a flowchart illustrating an example of the edge extension process. [Figure 11] A flowchart illustrating an example of the firing transmission process. [Figure 12] A flowchart illustrating an example of the firing determination process [Figure 13] Second block diagram of the NN growth apparatus 1 [Figure 14] 10 is a flowchart illustrating a second operation example of the NN growth apparatus 1. [Figure 15] A flowchart illustrating an example of the window determination process. [Figure 16] Block diagram of information processing device 2 according to embodiment 2. [Figure 17] A flowchart illustrating an example of the operation of the information processing device 2 [Figure 18]A flowchart illustrating an example of the information transmission process [Figure 19] Flowchart illustrating an example of homogeneous transfer processing [Figure 20] A flowchart illustrating an example of the firing determination process [Figure 21] Block diagram of the information processing device 3 [Figure 22] Overview of the computer system in the above embodiment [Figure 23] Block diagram of the computer system DETAILED DESCRIPTION OF THE INVENTION

[0027] Hereinafter, embodiments of the NN growth apparatus and the like will be described with reference to the drawings. Note that, in the embodiments, components with the same reference numerals perform similar operations, and therefore, repeated description may be omitted.

[0028] (Embodiment 1) In this embodiment, we will explain an NN growing device that receives image information and sound information, determines a state such as positive or negative using the sound information, detects a node that will fire based on one or more pieces of feature information acquired from the image information in accordance with the determination result, and performs growth processing corresponding to the node. Note that the growth processing is, for example, edge generation processing, edge growth processing, and node generation processing.

[0029] The edge generation process is the process of generating edges that make up a neural network (hereinafter referred to as "NN" where appropriate). The edge growth process is the process of growing the edges that make up the NN. The node generation process is the process of generating nodes that make up the NN. Note that the neural network here is preferably a spiking neural network. However, the neural network may be other types of neural networks, such as a deep neural network. In other words, the type of neural network is not important.

[0030] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.

[0031] 1 is a block diagram of an NN growth device 1 according to this embodiment. The NN growth device 1 includes a storage unit 11, a reception unit 12, a processing unit 13, and an output unit 14. The storage unit 11 includes a starting point storage unit 111, a goal storage unit 112, and an NN storage unit 113. The reception unit 12 includes an information reception unit 121. The processing unit 13 includes a state determination unit 131, a feature acquisition unit 132, a firing node determination unit 133, and a growth unit 134.

[0032] Various types of information are stored in the storage unit 11 constituting the NN growth device 1. The various types of information include, for example, firing start information (to be described later), goal information (to be described later), neural networks (NNs), one or more pieces of glial cell information (to be described later), one or more pieces of connection information (to be described later), one or more pieces of firing information (to be described later), and state determination information (to be described later).

[0033] One or more pieces of firing start point information are stored in the start point storage unit 111. The start point storage unit 111 normally stores two or more pieces of firing start point information.

[0034] The firing start point information is information that specifies a node that fires in the first stage when image information is received. A node that fires in the first stage is a node that fires without passing through other nodes. The firing start point information has an information identifier and one or more node identifiers. The firing start point information may also have an initial firing condition and one or more node identifiers.

[0035] An information identifier is information that identifies the feature information of image information. An information identifier is information that specifies the type of feature of an image. Examples of information identifiers are "R", "G", and "B". "R" is information that indicates the color red, "G" is information that indicates the color green, and "B" is information that indicates the color blue.

[0036] The initial firing condition is a condition under which a node fires in the first stage. The initial firing condition is a condition related to the information identifier. The initial firing condition usually has an information identifier. The initial firing condition is, for example, a condition related to the information identifier and the amount of information. For example, the initial firing condition is "<information identifier>R <condition> amount of information>=150". For example, the initial firing condition is "information amount of 'R' >= 150", "(information amount of 'R' >= 150) & (information amount of 'G' >= 80)", or "(information amount of 'R' >= 150) & (information amount of 'G' >= 80) & (information amount of 'B' >= 120)".

[0037] A node identifier is information that identifies a node that makes up an NN. A node identifier is, for example, a node ID or node name. A node may also be called a soma. A node identifier may also be called a soma identifier.

[0038] Here, the feature information is the feature amount of the image information. The feature information is, for example, an information identifier, or an information identifier and an information amount. The information identifier is information that identifies the feature information. The information amount is information that indicates the size of the information identified by the paired information identifier. The feature information is, for example, "<information identifier>R <information amount>150".

[0039] One or more pieces of goal information are stored in the goal storage unit 112. The goal storage unit 112 normally stores two or more pieces of goal information.

[0040] Goal information is information that specifies the goal to which the nodes or edges that make up the NN will grow. Goal information is information that specifies a goal corresponding to one of two or more states. A state is, for example, an emotion or an internal state of the brain. A state is, for example, positive or negative. It is preferable that the type of state is either positive or negative. However, there may be three or more types of states. When there are three or more types of states, each state is, for example, information indicating one of two or more degrees of positivity, or information indicating one of two or more degrees of negativity.

[0041] It can be said that goal information is information that specifies the position at which a node or edge grows. The goal information has goal position information or goal direction information. The goal information corresponds to a state identifier. The goal position information is information that specifies the position of the goal. The goal direction information indicates the direction of the goal. A position is a position in a virtual space of two or more dimensions. The goal information is, for example, position information. The position information is, for example, three-dimensional coordinate values ​​(x, y, z) or two-dimensional coordinate values ​​(x, y) or a four-dimensional quaternion (x, y, x, w). Furthermore, the fact that goal information corresponds to a state identifier means that the goal information corresponding to the determined state identifier is used for growing the NN.

[0042] The NN storage unit 113 stores neural network information (hereinafter referred to as "NN information" where appropriate). The NN information can be said to be information that mimics the brain. The NN information has two or more pieces of node information and one or more pieces of edge information. The NN information may also be referred to as NN.

[0043] Node information is information about the nodes that make up a NN. The node information includes a node identifier. The node information includes, for example, node position information, firing conditions, firing probability information, and count information. It is also preferable that the node information includes required energy amount information that indicates the amount of energy required for firing.

[0044] Node position information is the position information of a node. As described above, the position information is, for example, three-dimensional coordinate values ​​(x, y, z), two-dimensional coordinate values ​​(x, y), or four-dimensional quaternion (x, y, x, w).

[0045] An ignition condition is a condition under which a node is ignited. An ignition condition usually has one or more pieces of characteristic information. The characteristic information may be information that has an information identifier that identifies the information and an information amount that indicates the size of the information, or it may be information that only has an information amount that indicates the size of the information. The information amount is, for example, a numerical value greater than 0. Examples of ignition conditions are "R>=180", "R>=150 & G>=100", "G<=50 & B<=30", "R>=250 & G>=250 & B>=250", etc.

[0046] Firing probability information is information about the probability of firing. The firing probability information may be the firing probability itself, or a value obtained by converting the firing probability using a function or the like. It is preferable that the firing probability information is referenced, and the node may or may not fire at the probability indicated by the firing probability information, even if the feature information is the same.

[0047] The number of times information is information based on the number of times that a trigger has fired, such as the number of times that a trigger has fired or the firing frequency (firing rate).

[0048] Edge information is information about the edges that make up a NN. Edge information is information that identifies the connections between nodes. Edge information usually has an edge identifier. Edge information, for example, has the node identifier of each of the two nodes that an edge connects. Edge information, for example, has the node identifier of one connecting node. When edge information has only one node identifier, the edge is an edge in the growth process before connecting the two nodes. Edge information, for example, has edge position information. Edge position information is information that identifies the position of the endpoint of the edge. It is also preferable that edge information have retained energy amount information that indicates the amount of energy retained by the edge.

[0049] The edge information includes, for example, Dendrites information and AXON information. In such a case, the edge includes Dendrites and AXON. Note that an edge may be considered to be a single line or a line branched into two or more branches. An edge may also be called a synapse.

[0050] An edge identifier is information for identifying an edge, such as an edge ID or edge name.

[0051] Dendrite information is information about DENDRITES. Dendrites are also called dendrites and are parts of neurons. They are multiple projections that branch out from the cell body like tree branches in order for neurons to receive external stimuli and information sent from the axons of other neurons. DENDRITES are the elements that make up edges in this case. DENDRITES information has a DENDRITES identifier and DENDRITES position information.

[0052] The DENDRITES identifier is information that identifies DENDRITES. The DENDRITES identifier is, for example, the DENDRITES ID or the DENDRITES name.

[0053] DENDRITES position information is position information that indicates the position of DENDRITES. DENDRITES position information is information that specifies the position of DENDRITES, and is, for example, one or more three-dimensional coordinate values ​​(x, y, z) or one or more two-dimensional coordinate values ​​(x, y). When the DENDRITES position information has two or more coordinate values, DENDRITES is a line that connects each point of the two or more coordinate values.

[0054] It is also preferable that the Dendrites information includes information on the amount of energy held by the Dendrites, which indicates the amount of energy held by the Dendrites.It is also preferable that the Dendrites information includes information on the amount of energy required to transmit information using the Dendrites.

[0055] AXON information is information about AXONs. AXONs, also known as axons, are protruding structures extending from the cell body and responsible for signal output in neurons. AXONs are elements that make up edges. AXON information includes an AXON identifier and AXON position information.

[0056] The AXON identifier is information that identifies the AXON. For example, the AXON identifier is the AXON ID or the AXON name.

[0057] AXON position information is position information that indicates the position of the AXON. AXON position information is information that specifies the position of the AXON, and is, for example, one or more three-dimensional coordinate values ​​(x, y, z) or one or more two-dimensional coordinate values ​​(x, y). When the AXON position information has two or more coordinate values, the AXON is a line that connects each point of the two or more coordinate values.

[0058] It is also preferable that the AXON information includes information on the amount of energy held by the AXON, which indicates the amount of energy held by the AXON.It is also preferable that the AXON information includes information on the amount of energy required to transmit information using the AXON.

[0059] Note that dendrites and axons may be branched. When dendrites and axons are branched, the position information of each may be expressed using three or more coordinate values. However, there is no restriction on the method of expressing dendrite position information and axon position information.

[0060] The glial cell information stored in the storage unit 11 is information relating to glial cells. Note that the glial cell information does not necessarily have to be present in the storage unit 11.

[0061] Glial cells, also known as neuroglial cells, are a general term for non-neuronal cells that make up the nervous system. Glial cells are a glue- or cement-like substance that fills the spaces between neurons.

[0062] The glial cell information preferably includes a glial cell identifier for identifying the glial cell. The glial cell information may include, for example, a node identifier for identifying a node that assists in the connection, or an edge identifier for identifying an edge that assists in the connection. The glial cell information may include, for example, an axon identifier for an axon whose connection the glial cell assists, or a dendrite identifier for a dendrite whose connection the glial cell assists. The glial cell information may also include a glial cell type identifier for identifying the type of the glial cell. The type of glial cell may be, for example, oligodendrocites (hereinafter referred to as "oligo") or astrocites. Note that oligos are cells that can connect to axons. Astrocites are cells that can connect to somas or dendrites. The glial cell information preferably includes glial cell position information. The glial cell position information is position information that specifies the position of the glial cell. In particular, it is preferable that the glial cell information of oligos includes glial cell position information. The glial cell information may also include hand length information indicating the length of one or more hands. The glial cell information may also include hand number information indicating the number of hands emerging from the glial cell. Typically, it is preferable that when the total length of the glial cell calculated from the hand length information of each hand reaches a threshold, the glial cell will not grow any further.

[0063] The connection information stored in the storage unit 11 is information that specifies a connection between two or more nodes. The connection information may be information that specifies a connection between an AXON of one node and a Dendrites of another node. Such information is also information that specifies a connection between nodes. The connection information may be information that specifies a connection between a synapse and a spine. Such information is also information that specifies a connection between nodes. The connection information has, for example, identifiers of two nodes to be connected. The connection information also has, for example, an AXON identifier of an AXON and a Dendrites identifier of a Dendrite connected to the AXON. The connection information also has, for example, a synapse identifier of a synapse and a spine identifier of a spine that can transmit information to the synapse. The connection information may also have information transmission probability information. The information transmission probability information is information about the probability of information transmission between one node and another node. The information transmission probability information may be information about the probability of information transmission between an AXON and a Dendrites. In such a case, the information transmission probability information is information about the probability of information transmission between one node and another node. The information transmission probability information may also be information about the probability of information transmission between a synapse and a spine. In such a case, the information transmission probability information is information about the probability of information transmission between one node and another node. Note that the connection direction between nodes is usually unidirectional.

[0064] The binding information may be information indicating a binding between a node and an AXON. In such a case, the binding information includes a node identifier and an AXON identifier. The binding information may also be information indicating a binding between a node and a Dendrites. In such a case, the binding information includes a node identifier and a Dendrites identifier.

[0065] The binding information may be information specifying a binding between a glial cell and an AXON or a Dendritic cell. In such a case, the binding information may include, for example, a glial cell identifier that identifies the glial cell information and an AXON identifier. The binding information may include, for example, a glial cell identifier and a Dendritic cell identifier.

[0066] Note that connection information specifying the connections between elements (nodes, edges, axons, dendrites, glial cells, synapses, or spines) that make up the NN may be stored in the NN storage unit 113. In other words, the connection information specifying the connections between elements that make up the NN may be included in the information of each element.

[0067] Firing information is information about the result of firing. The firing information has a node identifier that identifies the node that fired. The firing information may usually have timer information that indicates the time of firing. The timer information may be information that indicates relative time or time information that indicates absolute time. Note that the firing information may be automatically deleted by the processing unit 13 after a certain time has passed since it was accumulated.

[0068] State determination information is information for determining one state using sound information. The state determination information is, for example, a set of two or more sound conditions, which are conditions related to sound information, and state identifiers. The state determination information may be, for example, "<sound condition> amount of information of information identifier "frequency=X" >= 50 <state identifier> state P" or "<sound condition> amount of information of information identifier "frequency=X" >= 80 <state identifier> state N." "<sound condition> amount of information of information identifier "frequency=Y" >= 50 & amount of information of information identifier "frequency=Z" >= 90 <state identifier> state P".

[0069] The receiving unit 12 receives various types of information, such as image information and sound information.

[0070] Here, reception is a concept that includes reception of information input from input devices such as a camera, microphone, keyboard, mouse, or touch panel, reception of information transmitted via a wired or wireless communication line, and reception of information read from recording media such as an optical disk, magnetic disk, or semiconductor memory.

[0071] The information receiving unit 121, for example, acquires image information captured by a camera. The information receiving unit 121, for example, acquires sound information captured by a microphone. In other words, "reception" refers to the reception of information captured by a device such as a microphone or a camera, but may also be a concept that includes the reception of information transmitted via a wired or wireless communication line, or the reception of information read from a recording medium such as an optical disk, a magnetic disk, or a semiconductor memory.

[0072] The information receiving unit 121 receives image information and sound information. For example, the information receiving unit 121 receives the image information and sound information at the same time. However, for example, the information receiving unit 121 may receive the image information and the sound information with some delay. The image information is a still image or a moving image. The sound information is, for example, audio data or music data, but any type of sound information will do as long as it is sound information.

[0073] The processing unit 13 performs various types of processing. The various types of processing include processing performed by a state determination unit 131, a feature acquisition unit 132, an ignition node determination unit 133, and a growth unit 134, for example.

[0074] The state determination unit 131 determines one state from two or more states using the sound information received by the information receiving unit 121. The state is, for example, either "positive (appropriately referred to as state P)" or "negative (appropriately referred to as state N)." Determining the state means, for example, obtaining a state identifier. The state identifier is information that identifies the state. The state identifier is, for example, "state P" or "state N."

[0075] Here, the feature information includes, for example, an information identifier and an information amount. The state determination unit 131 analyzes, for example, sound information, which is received audio, and acquires one or more feature information of the audio. The feature acquisition unit 132 performs, for example, audio analysis of the received audio information, and acquires "information identifier (frequency=X), information amount=50", "information identifier (frequency=Y), information amount=120", etc. Note that such information indicates the level at each frequency.

[0076] Determining a state means, for example, obtaining a state identifier. A state identifier is information that identifies a state. A state identifier is, for example, "state P" or "state N." Note that determining a state usually means obtaining a state identifier.

[0077] The state determination unit 131 determines, for example, a sound condition, which is a condition related to the sound information received by the information receiving unit 121, from the sound conditions included in two or more pieces of state determination information, and acquires a state identifier paired with the sound condition.

[0078] The state determination unit 131 acquires, for example, one or more pieces of feature information for the sound information by using the sound information accepted by the information accepting unit 121. Next, the state determination unit 131 acquires, for example, one state identifier paired with a sound condition that matches the one or more pieces of feature information for the sound information.

[0079] The state determination unit 131 determines one state from the sound information by using, for example, the state determination information. The state determination unit 131 acquires, for example, a state identifier paired with a sound condition that matches one or more pieces of feature information for the sound information.

[0080] The process of acquiring one or more pieces of feature information from the sound information may be performed by the feature acquisition unit 132.

[0081] The feature acquisition unit 132 uses the image information received by the information reception unit 121 to acquire one or more pieces of feature information for the image information.

[0082] Here, the feature information includes, for example, an information identifier and an information amount. The feature acquisition unit 132, for example, analyzes the received image information and acquires one or more pieces of feature information for the image information. For example, the feature acquisition unit 132 acquires the information amount of information identifier "R," the information amount of information identifier "G," and the information amount of information identifier "B" from the received image information.

[0083] The information amount of the information identifier "R" is information relating to the amount of information of "R" in the region of interest of the image. The information amount of the information identifier "R" is, for example, a representative value of the R value of one or more pixels in the region of interest of the image. The information amount of the information identifier "R" is, for example, the number or percentage of pixels whose R value is equal to or greater than a threshold value among one or more pixels in the region of interest of the image.

[0084] The information amount of the information identifier "G" is information relating to the amount of "G" information in the region of interest of the image. The information amount of the information identifier "G" is, for example, a representative value of the G value of one or more pixels in the region of interest of the image. The information amount of the information identifier "G" is, for example, the number or percentage of pixels whose G value is equal to or greater than a threshold value among one or more pixels in the region of interest of the image.

[0085] The information amount of information identifier "B" is information relating to the amount of information of "B" in the region of interest of the image. The information amount of information identifier "B" is, for example, a representative value of the B value of one or more pixels in the region of interest of the image. The information amount of information identifier "B" is, for example, the number or percentage of pixels whose B value is equal to or greater than a threshold value among one or more pixels in the region of interest of the image.

[0086] The region of interest in the image is a window determined by the window determination unit 130, which will be described later, but may also be the entire image information.

[0087] Furthermore, the representative value is, for example, the average value, but may also be the median value or the like.

[0088] The feature acquisition unit 132 may acquire one or more pieces of feature information for one piece of image information accepted by the information acceptance unit 121, or may acquire one or more pieces of feature information for two or more pieces of image information (videos) that are consecutive in time.

[0089] The feature acquisition unit 132 acquires the amount of movement between two pieces of image information, for example, by using one piece of image information and the image information temporally preceding the one piece of image information. The amount of movement is an example of feature information of the moving image acquired by the feature acquisition unit 132.

[0090] The feature acquisition unit 132 may acquire, for example, a first movement amount which is the movement amount between a first image information and a second image information immediately preceding the first image information, a second movement amount which is the movement amount between the first image information and a second image information immediately preceding the first image information, and an Nth movement amount which is the movement amount between the first image information and an Nth image information which is N images preceding the first image information, and form a movement amount vector (first movement amount, second movement amount, . . . , Nth movement amount). Such a movement amount vector is also an example of feature information of a moving image acquired by the feature acquisition unit 132.

[0091] The amount of movement between images is information that specifies the amount of movement between two images, and is, for example, a motion vector or optical flow.

[0092] The firing node determination unit 133 determines the node identifier of the firing node, which is the node that will fire and corresponds to one or more pieces of feature information acquired by the feature acquisition unit 132, from the start point storage unit 111. Next, the firing node determination unit 133 determines the node identifier of the firing node, which is connected to each of the one or more firing nodes by an edge and to which feature information is passed from the firing node. Note that the node identifier of the firing node will be referred to as the firing node identifier as appropriate.

[0093] The firing node determination unit 133 determines whether or not one or more pieces of characteristic information passed from one or more other nodes connected by edges satisfy a firing condition, and determines the node identifier of the node determined to satisfy the firing condition. Note that the firing condition is a condition related to one or more pieces of characteristic information.

[0094] It is preferable that the firing node determination unit 133 stores frequency information relating to the number of times of the determined node identifier in association with the node identifier. The frequency information is information based on the number of times firing has occurred, such as the number of firings or the firing frequency (firing rate).

[0095] The growth unit 134 performs a growth process. Here, the growth unit 134 performs a growth process of an NN that imitates the brain of an infant. The growth unit 134 performs a growth process of the NN using the received image information and sound information. The growth unit 134 performs a process of growing nodes or edges, or nodes and edges, that constitute the NN, using one or more pieces of feature information acquired from the received image information and one or more pieces of feature information acquired from the sound information.

[0096] In more detail, the growth unit 134 acquires goal information paired with one state determined by the state determination unit 131, and performs processing to grow node information or edge information corresponding to each of one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 using the goal information.

[0097] The growth unit 134, for example, acquires goal information paired with the state identifier of one state determined by the state determination unit 131, and performs processing to grow, using the acquired goal information, node information or edge information corresponding to one or more firing node identifiers that satisfy a growth condition among the one or more node identifiers determined by the firing node determination unit 133. Note that a node identifier that satisfies a growth condition is a node identifier that identifies node information that satisfies the growth condition.

[0098] The growing unit 134 performs a process of growing, for example, the node information or edge information corresponding to all of the node identifiers among the one or more node identifiers determined by the firing node determining unit 133, using the goal information.

[0099] Furthermore, the process of growing using the acquired goal information is, for example, a process of extending an edge connected to a node identified by an ignition node identifier in the direction of the position indicated by the goal position information contained in the acquired goal information. The process of growing using the acquired goal information is, for example, a process of extending an edge connected to a node identified by an ignition node identifier in the direction indicated by the goal direction information contained in the acquired goal information. Note that the process of extending an edge typically involves moving the position of the edge position information contained in the edge information away from the connected node. The process of extending an edge typically involves moving the position of the edge position information contained in the edge information closer to the position of the goal position information contained in the goal information.

[0100] The growth condition is a condition for growth. The growth condition is, for example, a condition based on frequency information. The growth condition is, for example, "the number of firings is equal to or greater than a threshold," "the number of firings is greater than a threshold," "the firing frequency is equal to or greater than a threshold," or "the firing frequency is greater than a threshold."

[0101] The growth process includes, for example, an edge generation process, an edge growth process, and a node generation process, which will be described later. Each of the growth processes will be described in detail below. (1) Edge generation processing

[0102] The growing unit 134 performs, for example, an edge generation process. The edge generation process can be said to be a process of generating a new edge. The edge generation process is a process of generating new edge information and storing it in the NN storage unit 113. In other words, the growing unit 134 generates and stores edge information of edges extending from nodes identified by one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133, at positions in the direction indicated by the goal information paired with one state determined by the state determination unit 131.

[0103] It is preferable that the one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 are one or more node identifiers that match the edge generation condition among the one or more node identifiers determined by the firing node determination unit 133. However, the one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 may be all of the node identifiers determined by the firing node determination unit 133.

[0104] The process of generating edge information is a process of generating edge information of an edge connected to a node identified by a target node identifier. The process of generating edge information is, for example, a process of acquiring a unique edge identifier, and generating edge information having the edge identifier, which is information on an edge connecting the node identified by the target node identifier to another node in the direction indicated by the goal information from the node. Such edge information includes, for example, the edge identifier and the node identifiers of the two nodes to be connected. The process of generating edge information is, for example, a process of acquiring a unique edge identifier, acquiring edge position information that extends from the node identified by the target node identifier and identifies the position of its end point from the node in the direction indicated by the goal information, and generating edge information having the edge position information. Such edge information includes, for example, the edge identifier, the node identifier of the target (connecting) node, and the edge position information that identifies the end point of the edge.

[0105] Moreover, the edge generation condition is a condition for generating an edge. The edge generation condition is, for example, a condition based on count information. The edge generation condition is, for example, "the number of firings is equal to or greater than a threshold," "the number of firings is greater than a threshold," "the firing frequency is equal to or greater than a threshold," or "the firing frequency is greater than a threshold." The edge generation condition may be the same as the growth condition, or may be different. The edge generation condition may be common to all nodes of interest, may be different for each node, or may be different for each edge. When the edge generation condition is different for each node, for example, the node information includes the edge generation condition. When the edge generation condition is different for each edge, for example, the edge information includes the edge generation condition.

[0106] It is preferable that the growing unit 134 performs edge generation processing on a node identified by a node identifier corresponding to the number information that matches the edge generation condition.

[0107] The edge generation process may be one or more of a Dendrites generation process and an AXON generation process, which will be described later. The edge generation process may also include a glial cell generation process, which will be described later. (1-1) Dendrites generation process

[0108] The growing unit 134 performs, for example, a Dendrites generation process. The Dendrites generation process is a process of generating new Dendrites. The edge generation process may include a process of generating new Dendrites information and storing it in the NN storage unit 113. That is, the growing unit 134 generates and stores Dendrites information for Dendrites extending from nodes identified by one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 in a direction indicated by goal information paired with one state determined by the state determination unit 131.

[0109] It is preferable that the one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 are one or more node identifiers that match the Dendrites generation condition among the one or more node identifiers determined by the firing node determination unit 133. However, the one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 may be all of the node identifiers determined by the firing node determination unit 133.

[0110] The process of generating Dendrites information is a process of generating Dendrites information of Dendrites connected to a node identified by a target node identifier. The process of generating Dendrites information is, for example, a process of acquiring a unique Dendrites identifier, acquiring Dendrites position information of the direction indicated by the goal information from the node identified by the target node identifier (the node identifier of the node to which the Dendrites is connected), and configuring and storing Dendrites information having the Dendrites identifier and the Dendrites position information.

[0111] Furthermore, the dendrites generation conditions are conditions for generating dendrites. The dendrites generation conditions are, for example, conditions based on frequency information. The edge generation conditions are, for example, "the number of firings is equal to or greater than a threshold," "the number of firings is greater than a threshold," "the firing frequency is equal to or greater than a threshold," or "the firing frequency is greater than a threshold." The dendrites generation conditions may be the same as or different from the growth conditions. (1-2) AXON generation process

[0112] The growth unit 134 performs, for example, an AXON generation process. The AXON generation process is a process of generating a new AXON. The edge generation process may include a process of generating new AXON information and storing it in the NN storage unit 113. That is, the growth unit 134 generates and stores AXON information for an AXON extending from a node identified by one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133, in a direction indicated by goal information paired with one state determined by the state determination unit 131, for example.

[0113] It is preferable that the one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 are one or more node identifiers that meet the AXON generation conditions among the one or more node identifiers determined by the firing node determination unit 133. However, the one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 may be all of the node identifiers determined by the firing node determination unit 133.

[0114] The process of generating AXON information is a process of generating AXON information for an AXON connected to a node identified by a target node identifier (the node identifier of the node to which the AXON is connected). The process of generating AXON information is, for example, a process of acquiring a unique AXON identifier, acquiring AXON position information in the direction indicated by the goal information from the node identified by the target node identifier, and constructing and storing AXON information having the AXON identifier and the AXON position information.

[0115] Furthermore, AXON generation conditions are conditions for generating AXONs. AXON generation conditions are, for example, conditions based on frequency information. Edge generation conditions are, for example, "number of firings is equal to or greater than a threshold," "number of firings is greater than a threshold," "firing frequency is equal to or greater than a threshold," or "firing frequency is greater than a threshold." AXON generation conditions may be the same as growth conditions, or may be different. (2) Edge growth processing

[0116] The growing unit 134 performs, for example, an edge growing process. The edge growing process is a process of growing an edge. The process of growing an edge is usually a process of increasing the length of an edge. The process of growing an edge may also be a process of connecting the edge from the node from which the edge originates to another node. The growing unit 134 performs, for example, an edge growing process of acquiring and accumulating edge information obtained by growing an edge extending from a node identified by one or more node identifiers among the one or more node identifiers determined by the firing node determining unit 133 in a direction indicated by goal information paired with a state determined by the state determining unit 131.

[0117] It is preferable that each of the one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 is the node identifier of one or more nodes that meets the edge growth condition among the one or more node identifiers determined by the firing node determination unit 133. However, each of the one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 may be all of the node identifiers determined by the firing node determination unit 133.

[0118] In addition, obtaining edge information by growing an edge extending from a node means changing the edge position information contained in the edge information to position information in which the end point of the edge indicated by the edge position information is located farther away from the node.

[0119] The edge growth condition is a condition for performing the edge growth process. The edge growth condition is, for example, a condition based on count information. The edge growth condition is, for example, "the number of firings is equal to or greater than a threshold," "the number of firings is greater than a threshold," "the firing frequency is equal to or greater than a threshold," or "the firing frequency is greater than a threshold." The edge growth condition may be the same as the growth condition, or may be different. Furthermore, the edge growth condition may be common to all nodes of interest, may be different for each node, or may be different for each edge. When the edge growth condition is different for each node, for example, the node information includes the edge growth condition. When the edge growth condition is different for each edge, for example, the edge information includes the edge growth condition.

[0120] In addition, the process of growing an edge extending from a node is a process of obtaining position information that specifies the position in the direction indicated by the goal information from the edge position information contained in the edge information having a node identifier that identifies the node, and using this position information as edge position information.

[0121] The edge growth process may be one or more of a Dendrites growth process, which will be described later, and an AXON growth process, which will be described later. (2-1) Dendrites growth treatment

[0122] The growth unit 134 performs, for example, a dendrites growth process. The dendrites growth process is a process of growing dendrites. The dendrites growth process may be included in the edge growth process. The dendrites growth process is usually a process of increasing the length of the dendrites. The dendrites growth process is a process of acquiring new dendrites position information for the dendrites position information contained in the dendrites information of the dendrites in question, which sets the position of the endpoint indicated by the dendrites position information to a position away from the position of the connected node, and storing the new dendrites position information.

[0123] The growth unit 134 performs a Dendrites growth process, for example, by acquiring and accumulating Dendrites information obtained by growing Dendrites extending from nodes identified by one or more node identifiers among one or more node identifiers determined by the firing node determination unit 133 in a direction indicated by goal information paired with one state determined by the state determination unit 131.

[0124] It is preferable that each of the one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 is a node identifier of one or more nodes that meets the Dendrites growth condition among the one or more node identifiers determined by the firing node determination unit 133. However, the one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 may be all of the node identifiers determined by the firing node determination unit 133.

[0125] In addition, obtaining Dendrites information by growing Dendrites extending from a node means changing the Dendrites position information contained in the Dendrites information to position information in which the end point of the Dendrites indicated by the Dendrites position information is located farther away from the node.

[0126] The dendrites growth conditions are conditions for performing the dendrites growth process. The dendrites growth conditions are, for example, conditions based on frequency information. The dendrites growth conditions are, for example, "the number of firings is equal to or greater than a threshold," "the number of firings is greater than a threshold," "the firing frequency is equal to or greater than a threshold," or "the firing frequency is greater than a threshold." The dendrites growth conditions may be the same as the growth conditions or may be different.

[0127] In addition, the process of growing Dendrites extending from a node is a process of obtaining position information in the direction indicated by the goal information from the Dendrites position information contained in the Dendrites information that pairs with the node identifier that identifies the node, and setting this position information as the Dendrites position information. (2-2) AXON growth treatment

[0128] The growth unit 134 performs, for example, an axon growth process. The axon growth process is a process of growing an axon. The axon growth process may be included in the edge growth process. The axon growth process is usually a process of increasing the length of an axon. The axon growth process is a process of acquiring new axon position information for the axon position information contained in the axon information of the axon, which is a position that is further away from the position of the end point indicated by the axon position information than the position of the connected node, and storing the new axon position information.

[0129] The growth unit 134 performs an AXON growth process, for example, by acquiring and storing AXON information obtained by growing an AXON extending from a node identified by one or more node identifiers among one or more node identifiers determined by the firing node determination unit 133 in a direction indicated by goal information paired with one state determined by the state determination unit 131.

[0130] It is preferable that each of the one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 is a node identifier of one or more nodes that meets the AXON growth condition among the one or more node identifiers determined by the firing node determination unit 133. However, the one or more node identifiers among the one or more node identifiers determined by the firing node determination unit 133 may be all of the node identifiers determined by the firing node determination unit 133.

[0131] In addition, obtaining AXON information that has grown an AXON extending from a node means changing the AXON position information contained in the AXON information to position information in which the end point of the AXON indicated by the AXON position information is located farther away from the node.

[0132] The AXON growth conditions are conditions for performing the AXON growth process. The AXON growth conditions are, for example, conditions based on frequency information. The AXON growth conditions are, for example, "the number of firings is equal to or greater than a threshold," "the number of firings is greater than a threshold," "the firing frequency is equal to or greater than a threshold," or "the firing frequency is greater than a threshold." The AXON growth conditions may be the same as the growth conditions, or may be different.

[0133] In addition, the process of growing an AXON extending from a node is a process of obtaining position information in the direction indicated by the goal information from the AXON position information contained in the AXON information that pairs with the node identifier that identifies the node, and setting this position information as the AXON position information. (3) Node creation process

[0134] The growth unit 134 performs, for example, a node generation process. The node generation process is a process of generating new node information. That is, the growth unit 134 performs a node generation process to generate and store node information of a new node at a position in the vicinity of a position indicated by node position information of a node identified by one or more node identifiers among the one or more node identifiers acquired by the firing node determination unit 133, which is a position in a direction indicated by goal information paired with one state determined by the state determination unit 131.

[0135] The growth unit 134 acquires, for example, a new node identifier. The growth unit 134 also acquires new node position information, for example, of a position in the direction indicated by the goal information paired with one state determined by the state determination unit 131, which is a predetermined distance away from the position indicated by the node position information of the target node (firing node). The growth unit 134 also acquires, for example, information (for example, firing condition or firing probability information) included in the node information of the target node. The growth unit 134 then constructs node information including, for example, one or more pieces of information selected from the new node identifier, the new node position information, and the firing condition or firing probability information, and stores the node information in the NN storage unit 113. The predetermined distance may be a predetermined distance or may change dynamically.

[0136] It is preferable that the one or more node identifiers among the one or more node identifiers are, for example, one or more node identifiers included in the node information that matches the node generation condition among the one or more node identifiers. However, the one or more node identifiers among the one or more node identifiers may be, for example, all of the one or more node identifiers.

[0137] The node generation condition is a condition for generating a node. The node generation condition may be the same as the growth condition, or may be different. The node generation condition may be common to all nodes of interest, or may be different for each node. When the node generation condition is different for each node, for example, the node information includes the node generation condition. (4) Glial cell generation treatment

[0138] It is preferable that the growth unit 134 perform the following glial cell generation process. That is, for example, when the amount of energy possessed by an element that is a node or edge becomes so small relative to the amount of required energy that a predetermined condition is satisfied, the growth unit 134 generates glial cell information connected to the element. Note that the element may be an AXON or a Dendrite. That is, when the amount of energy possessed by an element that is an AXON or a Dendrite becomes so small relative to the amount of required energy that a predetermined condition is satisfied, the growth unit 134 generates glial cell information connected to the element.

[0139] The predetermined condition is, for example, "reserved energy amount < required energy amount", or "reserved energy amount <= required energy amount", or "reserved energy amount - required energy amount <= threshold", or "reserved energy amount - required energy amount < threshold".

[0140] More specifically, the growth unit 134, for example, determines whether the amount of retained energy indicated by the retained energy amount information contained in the information of each element (node ​​information, edge information, AXON information, or Dendrites information) is small enough to satisfy predetermined conditions by comparing it with the amount of required energy indicated by the required energy amount information contained in the information of each element, and if it determines that the amount is small, generates glial cell information having an identifier that identifies the element (node ​​identifier, edge identifier, AXON identifier, or Dendrites identifier) ​​and stores it in the storage unit 11.

[0141] The output unit 14 outputs various types of information, such as the identifier of the fired node, the NN information in the NN storage unit 113, and the state identifier acquired by the state determination unit 131.

[0142] The various types of information are, for example, information that graphically represents the NN information. In this case, the processing unit 13 constructs a diagram (e.g., a sphere) of the nodes that make up the NN from the node information contained in the NN information, and constructs a diagram (e.g., a line) of the edges that make up the NN from the edge information. The processing unit 13 also arranges the node diagram (e.g., a sphere) at the position in virtual space indicated by the node position information contained in each node information, arranges a diagram (e.g., a line) of the edge whose end point is the position in virtual space indicated by the edge position information contained in each edge information, and constructs a diagram that clearly shows that the node to which the edge is connected and the diagram (e.g., a line) of the edge are connected.

[0143] Here, output is a concept that includes displaying on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and delivery of processing results to other processing devices or other programs.

[0144] The storage unit 11, the starting point storage unit 111, the goal storage unit 112, and the NN storage unit 113 are preferably non-volatile recording media, but may also be realized as volatile recording media.

[0145] There is no restriction on the process by which information is stored in the storage unit 11 etc. For example, information may be stored in the storage unit 11 etc. via a recording medium, information transmitted via a communication line etc. may be stored in the storage unit 11 etc., or information input via an input device may be stored in the storage unit 11 etc.

[0146] The reception unit 12 and the information reception unit 121 may be realized by devices such as a microphone or a camera. The reception unit 12 and the like may also be realized by wireless or wired communication means. The reception unit 12 and the information reception unit 121 may also be realized by a processor, a memory, or the like.

[0147] The processing unit 13, state determination unit 131, feature acquisition unit 132, firing node determination unit 133, and growth unit 134 can usually be realized by a processor, memory, etc. The processing procedures of the processing unit 13, etc. are usually realized by software, and the software is recorded on a recording medium such as a ROM. However, they may also be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0148] The output unit 14 may or may not include an output device such as a display, a speaker, etc. The output unit 14 may be realized by driver software for an output device, or by a combination of driver software for an output device and the output device, etc.

[0149] Next, an example of the operation of the NN growth device 1 will be described with reference to the flowchart in Fig. 2. The first example of the operation of the NN growth device 1 is a case where the NN growth device 1 does not have a node determination unit 130, which will be described later.

[0150] (Step S201) Information receiving unit 121 determines whether sound information and image information have been received. If sound information and image information have been received, the process proceeds to step S202, and if not, the process returns to step S201.

[0151] (Step S202) The state determining unit 131 acquires the sound information received in step S201.

[0152] (Step S203) The state determining unit 131 acquires one or more pieces of feature information from the sound information acquired in step S202.

[0153] (Step S204) The state determination unit 131 acquires state determination information from the storage unit 11. The state determination unit 131 acquires a state identifier that identifies the state to which the sound information corresponds, using the state determination information and the one or more pieces of feature information acquired in step S203.

[0154] (Step S205) The feature acquisition unit 132 acquires the image information accepted in step S201.

[0155] (Step S206) The feature acquisition unit 132 acquires one or more pieces of feature information using the image information acquired in step S205.

[0156] (Step S207) The growing unit 134 and the like perform the growing process. Return to step S201. An example of the growing process will be described with reference to the flowchart in FIG. 3. The growing process is a process of constructing the neural network information stored in the NN storage unit 113.

[0157] 2, the feature acquisition unit 132 may acquire a partial image for a window determined by the determination unit 130, which will be described later, and may acquire one or more pieces of feature information from the partial image in step S206. In this case, it is preferable that the node determination unit 130 sequentially acquires partial images for different windows for the image information received in step S201, and the processing from step S205 to step S207 is repeated (looped) until the information receiving unit 121 receives the next image information and sound information in step S201.

[0158] In the flowchart of FIG. 2, the process ends when the power is turned off or an interrupt occurs to end the process.

[0159] Next, an example of the growth process in step S207 will be described with reference to the flowchart in FIG.

[0160] (Step S301) The ignition node determination unit 133 assigns 1 to a counter i.

[0161] (Step S302) The ignition node determination unit 133 determines whether or not the i-th feature information exists among the feature information acquired in step S206. If the i-th feature information exists, the process proceeds to step S303; if not, the process returns to the upper level process.

[0162] (Step S303) The firing node determination unit 133 acquires node identifiers contained in one or more pieces of firing start point information that are satisfied by the i-th feature information from the start point storage unit 111. The firing node determination unit 133 composes firing information including the node identifiers and stores it in the storage unit 11. It is preferable that the firing node determination unit 133 acquires timer information indicating the time of firing from a clock (not shown), composes firing information including the timer information and the node identifier, and stores it in the storage unit 11. The one or more node identifiers are identifiers of the node that will fire first. Also, the node identifiers are firing node identifiers.

[0163] (Step S304) The ignition node determination unit 133 assigns 1 to a counter j.

[0164] (Step S305) The ignition node determination unit 133 determines whether or not the j-th ignition node identifier exists among the ignition node identifiers acquired in step S303. If the j-th ignition node identifier exists, the process proceeds to step S306; if not, the process proceeds to step S309.

[0165] (Step S306) The firing node determination unit 133 updates the number of times information included in the node information corresponding to the j-th firing node identifier to increase the number of times information included in the node information corresponding to the j-th firing node identifier. For example, the firing node determination unit 133 reads the number of times information included in the node information corresponding to the j-th firing node identifier, and overwrites the number of times information with the number of times information added by 1.

[0166] (Step S307) The growing unit 134 performs a growing process corresponding to one node (referred to as a "node of interest") identified by the j-th firing node identifier (referred to as a "node of interest identifier"). Such a growing process is called a single-node process. An example of the single-node process will be described with reference to the flowchart in FIG. 4.

[0167] (Step S308) The ignition node determination unit 133 increments the counter j by 1. The process returns to step S305.

[0168] (Step S309) The ignition node determination unit 133 increments the counter i by 1. The process returns to step S302.

[0169] Next, an example of the single-node processing in step S307 will be described with reference to the flowchart in FIG.

[0170] (Step S401) The growing unit 134 performs a process of generating a new node for a node of interest identified by a node of interest identifier. An example of such a node generation process will be described with reference to the flowchart of FIG.

[0171] (Step S402) The growing unit 134 performs processing to generate an edge connected to the node of interest. An example of such edge generation processing will be described with reference to the flowchart of FIG.

[0172] (Step S403) The growing unit 134 performs a process of growing an edge connected to the node of interest. An example of such an edge growing process will be described with reference to the flowchart of FIG.

[0173] (Step S404) The ignition node determination unit 133 performs ignition transmission processing. An example of the ignition transmission processing will be described with reference to the flowchart in Fig. 11. Note that the ignition transmission processing is a processing in which feature information is transmitted from a node of interest and a node to be ignited is determined.

[0174] (Step S405) The ignition node determination unit 133 assigns 1 to a counter j.

[0175] (Step S406) The ignition node determination unit 133 determines whether or not the j-th ignition node identifier exists among the ignition node identifiers of the nodes determined to be ignited in step S404. If the j-th ignition node identifier exists, the process proceeds to step S407; if not, the process returns to the upper processing.

[0176] (Step S407) The ignition node determination unit 133 increments the number of times information paired with the j-th ignition node identifier.

[0177] (Step S408) The growing unit 134 etc. performs single-node processing with the ignition node identified by the j-th ignition node identifier as the node of interest. An example of the single-node processing will be described with reference to the flowchart of FIG.

[0178] (Step S409) The ignition node determination unit 133 increments the counter j by 1. The process returns to step S406.

[0179] Next, an example of the node generation process in step S401 will be described with reference to the flowchart in FIG.

[0180] (Step S501) The growing unit 134 acquires node information corresponding to a node identifier of interest from the NN storage unit 113.

[0181] (Step S502) The growing unit 134 acquires a node generation condition.

[0182] (Step S503) The growth unit 134 determines whether the node information acquired in step S501 satisfies the node generation condition acquired in step S502. If the node generation condition is satisfied, the process proceeds to step S504, and if the node generation condition is not satisfied, the process returns to the upper level process.

[0183] (Step S504) The growing unit 134 performs node information generation processing. An example of the node information generation processing will be described with reference to the flowchart in FIG.

[0184] (Step S505) The growing unit 134 accumulates the node information configured in step S504 in the NN storage unit 113. The process returns to the upper level processing.

[0185] Next, an example of the node information generation process in step S504 will be described with reference to the flowchart in FIG.

[0186] (Step S601) The growing unit 134 acquires node position information included in the node of interest information identified by the node of interest identifier.

[0187] (Step S602) The growth unit 134 acquires, from the goal storage unit 112, the goal information corresponding to the state identifier acquired in step S203.

[0188] (Step S603) The growth unit 134 uses the node position information acquired in step S601 and the goal information acquired in step S602 to acquire node position information indicating a position in a direction specified by the goal information relative to the position indicated by the node position information acquired in step S601. This node position information is the position information of the new node.

[0189] For example, the growth unit 134 acquires node position information indicating a position that is a predetermined distance away from the position indicated by the node position information acquired in step S601 in the direction specified by the goal information. For example, if another node exists in the direction specified by the goal information from the position indicated by the node position information acquired in step S601, the growth unit 134 acquires node position information indicating a position that is between the position indicated by the node position information acquired in step S601 and the node position information of the other node in the direction specified by the goal information and that is a distance that is within a predetermined distance. In other words, when acquiring node position information of a new node, the growth unit 134 only needs to acquire node position information in the direction specified by the goal information, and the node position information does not matter.

[0190] (Step S604) The growing unit 134 acquires a node identifier of the new node. The growing unit 134 generates a new node identifier. However, the growing unit 134 may acquire an unused node identifier from the set of node identifiers.

[0191] (Step S605) The growth unit 134 acquires information to be used for the node information of the new node, which is included in the node information acquired in step S501. Note that such information includes, for example, ignition conditions, ignition probability information, and possessed energy amount information.

[0192] (Step S606) The growing unit 134 generates node information having the node identifier acquired in step S604, the node position information acquired in step S603, and the information acquired in step S605, and then returns to the upper processing.

[0193] Next, an example of the edge generation process in step S402 will be described with reference to the flowchart in FIG.

[0194] (Step S701) The growing unit 134 acquires node information identified by the node identifier of interest from the NN storage unit 113.

[0195] (Step S702) The growing unit 134 acquires edge generation conditions.

[0196] (Step S703) The growing unit 134 determines whether or not the node information acquired in step S701 satisfies the edge generation condition. If the edge generation condition is satisfied, the process proceeds to step S704, and if not, the process returns to the upper level process.

[0197] (Step S704) The growing unit 134 performs edge information generation processing. An example of the edge information generation processing will be described with reference to the flowchart of FIG.

[0198] (Step S705) The growing unit 134 stores the edge information constructed in step S704 in the NN storage unit 113. The process returns to the upper level process.

[0199] Next, an example of the edge information generation process in step S704 will be described with reference to the flowchart in FIG.

[0200] (Step S801) The growth unit 134 acquires, from the goal storage unit 112, goal information corresponding to the state identifier acquired in step S203.

[0201] (Step S802) The growing unit 134 acquires edge position information of a new edge using the node position information acquired in step S601 and the goal information acquired in step S801. The growing unit 134 acquires edge position information of an edge that extends in the direction of the goal information, starting from the node position information. Note that, for example, the distance between the position indicated by the edge position information and the position indicated by the node position information may or may not be determined in advance.

[0202] The growing unit 134 acquires edge position information indicating a position that is a predetermined distance away from the position indicated by the node position information acquired in step S601 in the direction specified by the goal information, for example. If another node exists in the direction specified by the goal information from the position indicated by the node position information acquired in step S601, for example, the growing unit 134 acquires the node position information of the other node as edge position information. That is, here, the generated edge becomes an edge that connects the node corresponding to the node position information acquired in step S601 with the other node. For example, when another node exists in the direction specified by the goal information from the position indicated by the node position information acquired in step S601, and the distance between the position indicated by the node position information acquired in step S601 and the position indicated by the node position information of the other node is equal to or greater than a threshold, the growth unit 134 acquires node position information indicating a position that is a predetermined distance away from the position indicated by the node position information acquired in step S601, and when the distance between the position indicated by the node position information acquired in step S601 and the position indicated by the node position information of the other node is equal to or less than a threshold that is smaller than the threshold, the growth unit 134 acquires the node position information of the other node as edge position information. In other words, the growth unit 134 only needs to acquire edge position information of an edge that extends in the direction of the goal information from the node position information as a starting point, and the edge position information is not important.

[0203] (Step S803) The growing unit 134 acquires an edge identifier of a new edge. For example, the growing unit 134 generates a new edge identifier. For example, the growing unit 134 acquires an unused edge identifier from the set of edge identifiers.

[0204] (Step S804) The growing unit 134 acquires the node identifier (target node identifier) ​​of the node to which the edge is connected. Here, the growing unit 134 acquires the node identifier that pairs with the node position information acquired in step S601. The growing unit 134 may also acquire the node identifier of a newly connected node.

[0205] (Step S805) The growing unit 134 generates edge information having the edge identifier acquired in step S803, the edge position information acquired in step S802, and one or two node identifiers acquired in step S804, and then returns to the upper-level processing.

[0206] In the flowchart of FIG. 8, the edge corresponding to the generated edge information may be in a state where there is no node connected ahead, or may be in a state where there is a node connected ahead.

[0207] If there is a node connected to the edge corresponding to the generated edge information, the growing unit 134 acquires the node identifier paired with the node position information of the position in the direction of the goal information as the node identifier of the connected node. Note that when the growing unit 134 always configures and stores edge information so that a generated edge connects two nodes, edge growing processing is not usually performed.

[0208] Next, an example of the edge growing process in step S403 will be described with reference to the flowchart in FIG.

[0209] (Step S901) The growing unit 134 acquires node information identified by the node identifier of interest from the NN storage unit 113.

[0210] (Step S902) The growing unit 134 assigns 1 to the counter i.

[0211] (Step S903) The growing unit 134 determines whether or not the i-th edge information exists in the NN storage unit 113. If the i-th edge information exists, the process proceeds to step S904, and if not, the process returns to the upper level process.

[0212] (Step S904) The growing unit 134 acquires the i-th edge information from the NN storage unit 113.

[0213] (Step S905) The growing unit 134 determines whether or not a node is connected to the end of the edge corresponding to the i-th edge information. More specifically, the growing unit 134 determines whether or not the node identifier included in the i-th edge information is only the target node identifier. If it is only the target node identifier, the process proceeds to step S906, and if it is not only the target node identifier (if two node identifiers exist), the process proceeds to step S909. Note that edge information including only the target node identifier is edge information of an edge that is connected to the target node and can grow.

[0214] (Step S906) The growing unit 134 acquires edge growing conditions.

[0215] (Step S907) The growing unit 134 determines whether or not the edge information acquired in step S904 satisfies the edge growing condition. If the edge growing condition is satisfied, the process proceeds to step S908, and if not, the process proceeds to step S909.

[0216] Here, the growing unit 134 may determine whether or not the node information of the node identified by the node identifier included in the edge information acquired in step S904 satisfies the edge growth condition.

[0217] (Step S908) The growing unit 134 performs edge extension processing. An example of the edge extension processing will be described with reference to the flowchart of FIG.

[0218] The edge extension process is a process for extending the length of an edge, and is usually a process for changing edge position information or a process for adding the node identifier of the destination node to the edge information.

[0219] (Step S909) The growing unit 134 increments the counter i by 1. The process returns to step S903.

[0220] In the flowchart of FIG. 9, the edge growth process may be replaced with a dendrites growth process for dendrites that form an edge, or an axon growth process for axons.

[0221] Next, an example of the edge extension process in step S908 will be described with reference to the flowchart in FIG.

[0222] (Step S1001) The growing unit 134 acquires edge position information included in the acquired edge information.

[0223] (Step S1002) The growth unit 134 acquires goal information.

[0224] (Step S1003) The growth unit 134 acquires new edge position information and updates the edge position information using the edge position information acquired in step S1001 and the goal information acquired in step S1002. Note that the growth unit 134 acquires position information that identifies the position in the direction indicated by the goal information from the edge position information.

[0225] For example, the growing unit 134 acquires edge position information indicating a position that is a predetermined distance away from the position indicated by the edge position information acquired in step S1001 in the direction specified by the goal information. For example, if another node exists in the direction specified by the goal information from the position indicated by the edge position information acquired in step S1001, the growing unit 134 acquires edge position information indicating a position that is within a predetermined distance from the position indicated by the edge position information and the node position information of the other node in the direction specified by the goal information. For example, if another node exists in the direction specified by the goal information from the position indicated by the edge position information acquired in step S1001, the growing unit 134 acquires the node position information of the other node as edge position information. In other words, when acquiring new edge position information, it is sufficient for the growing unit 134 to acquire edge position information in the direction specified by the goal information from the current edge position information, and the edge position information does not matter. Note that the node position information of another node is acquired as edge position information when the edge is connected to another node by edge extension processing, as described below. In this case, the growing unit 134 may obtain the node identifier of the other node.

[0226] (Step S1004) The growing unit 134 assigns 1 to the counter i.

[0227] (Step S1005) The growing unit 134 determines whether or not the i-th node information exists in the NN storage unit 113. If the i-th node information exists, the process proceeds to step S1006, and if not, the process returns to the upper level process.

[0228] (Step S1006) The growing unit 134 acquires the node position information included in the i-th node information.

[0229] (Step S1007) The growing unit 134 determines whether or not the node position information acquired in step S1006 satisfies the connection condition. If the connection condition is satisfied, the process proceeds to step S1008; if not, the process proceeds to step S1011. The connection condition is a condition for an edge to be connected to a node. The connection condition is, for example, that the distance between the position indicated by the node position information acquired in step S1006 and the position indicated by the new edge position information acquired in step S1003 is within a threshold or is smaller than the threshold.

[0230] (Step S1008) The growing unit 134 acquires the node identifier included in the i-th node information.

[0231] (Step S1009) The growing unit 134 changes the edge position information updated in step S1003 to the node position information included in the i-th node information.

[0232] (Step S1010) The growing unit 134 adds the node identifier acquired in step S1008 to the acquired edge information, and returns to the upper level processing.

[0233] (Step S1011) The growing unit 134 increments the counter i by 1. The process returns to step S1005.

[0234] In the flowchart of FIG. 10, the edge extension process may be replaced with a Dendrites extension process of the Dendrites that make up the edge, or an AXON extension process of the AXON.

[0235] The dendrites extension process is a process for extending dendrites, and is a process in which edge information is replaced with dendrites information in the process description using Fig. 10. The axon extension process is a process for extending axons, and is a process in which edge information is replaced with axon information in the process description using Fig. 10.

[0236] 10, the process may proceed to step S1011 after the process of step S1010. In such a case, one edge may branch and be connected to two or more nodes.

[0237] In addition, in step S1007 of the flowchart in Figure 10, the distance between the new edge position information acquired in step S1003 and the node position information of each node may be calculated, and it may be determined whether the node position information of the node with the smallest distance satisfies the connection condition.

[0238] Next, an example of the firing transmission process in step S404 will be described with reference to the flowchart of FIG.

[0239] (Step S1101) The firing node determination unit 133 acquires all edge information including firing node identifiers from the NN storage unit 113.

[0240] (Step S1102) The ignition node determination unit 133 assigns 1 to a counter i.

[0241] (Step S1103) The ignition node determination unit 133 determines whether or not the i-th edge information exists among the edge information acquired in step S1101. If the i-th edge information exists, the process proceeds to step S1104, and if not, the process returns to the upper process.

[0242] (Step S1104) The ignition node determination unit 133 determines whether or not the node identifier of another node exists in the i-th edge information. If the node identifier of another node exists, the process proceeds to step S1105; if not, the process proceeds to step S1112. Note that the node identifier of another node in the edge information is the node identifier of the node to which the edge is connected.

[0243] (Step S1105) The ignition node determination unit 133 acquires the node identifier of another node in the i-th edge information. Next, the ignition node determination unit 133 acquires the node information of the node identified by the node identifier from the NN storage unit 113.

[0244] (Step S1106) The ignition node determination unit 133 uses the node information acquired in step S1105 to determine whether or not the node corresponding to the node information will ignite. An example of such ignition determination processing will be described with reference to the flowchart in FIG.

[0245] (Step S1107) If the determination result in step S1106 is "fire", the ignition node determination unit 133 proceeds to step S1108, and if the determination result is "not fire", the ignition node determination unit 133 proceeds to step S1112.

[0246] (Step S1108) The ignition node determination unit 133 acquires ignition information having the node identifier included in the node information acquired in step S1105, and accumulates the ignition information in the storage unit 11.

[0247] (Step S1109) The firing node determination unit 133 changes the firing probability information included in the node information acquired in step S1105. Here, the firing node determination unit 133 changes the firing probability information so that the firing probability specified by the firing probability information increases.

[0248] (Step S1110) The firing node determination unit 133 determines whether or not to terminate the transmission of firing (which can also be called the transmission of information) between nodes. If the transmission is to be terminated, the process proceeds to step S1112, and if the transmission is not to be terminated, the process proceeds to step S1111. Note that the transmission is to be terminated, for example, when the node in question is the terminal node in the NN.

[0249] (Step S1111) The ignition node determination unit 133 performs ignition transmission processing with the node in question as the node of interest. An example of the ignition transmission processing is shown in FIG.

[0250] (Step S1112) The ignition node determination unit 133 increments the counter i by 1. The process returns to step S1103.

[0251] In the flowchart of FIG. 11, when transmitting firing (transmission of information) between nodes, the firing node determination unit 133 preferably updates the retained energy amount information by subtracting the amount of energy indicated by the retained energy amount information contained in the node information that caused the firing. This may also be applied to the retained energy amount information paired with the AXON identifier of the AXON used for the transmission, and the retained energy amount information paired with the Dendrites identifier of the Dendrites used for the transmission. The function for reducing the amount of energy is stored, for example, in the storage unit 11. The function in question is not important. Since the function is a well-known technique, detailed description thereof will be omitted.

[0252] In the flowchart of FIG. 11, the firing node determination unit 133 normally performs processing to pass one or more pieces of feature information received by the firing source node to the firing destination node.

[0253] Next, an example of the firing determination process in step S1106 will be described with reference to the flowchart in FIG.

[0254] (Step S1201) The ignition node determination unit 133 acquires the ignition condition corresponding to the node information acquired in step S1105.

[0255] (Step S1202) The ignition node determination unit 133 acquires one or more pieces of characteristic information. Note that the one or more pieces of characteristic information are characteristic information passed from the node that is the source of ignition.

[0256] (Step S1203) The ignition node determination unit 133 determines whether or not one or more pieces of feature information acquired in step S1202 satisfy the ignition condition acquired in step S1201. If the ignition condition is satisfied, the process proceeds to step S1204, and if the ignition condition is not satisfied, the process proceeds to step S1207.

[0257] (Step S1204) The firing node determination unit 133 determines whether the node information of interest has firing probability information. If it has firing probability information, the process proceeds to step S1205, and if it does not have firing probability information, the process proceeds to step S1206.

[0258] (Step S1205) The firing node determination unit 133 acquires firing probability information included in the node information of interest. Next, the firing node determination unit 133 uses the firing probability information to determine whether or not firing will occur. If firing will occur, the process proceeds to step S1206, and if not, the process proceeds to step S1207.

[0259] (Step S1206) The firing node determination unit 133 assigns "fire" to the judgment result, and returns to the upper level processing.

[0260] (Step S1207) The firing node determination unit 133 assigns "not firing" to the judgment result, and returns to the upper level processing.

[0261] As described above, according to this embodiment, it is possible to simulate the growth of an infant's brain. In other words, according to this embodiment, it is possible to realize a model of the growth of an infant's brain. Note that the growth of an infant's brain refers to the growth of the nodes, or edges, or nodes and edges that make up a neural network, based on image information and sound information.

[0262] The processing in this embodiment may be realized by software. This software may be distributed by software download or the like. Furthermore, this software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software that realizes the NN growing device 1 in this embodiment is the following program. That is, this program includes: a computer that can access an NN storage unit that stores neural network information having two or more pieces of node information each having a node identifier and one or more pieces of edge information each having an edge identifier and specifying connections between the nodes; a start point storage unit that stores one or more pieces of firing start point information each having an information identifier that identifies feature information of image information and one or more node identifiers that identify the node that will fire first when the feature information is received; and a goal storage unit that stores goal information that specifies goals corresponding to two or more states, including positive and negative; an information receiving unit that receives image information and sound information; a state determination unit that determines one state from the two or more states using the sound information received by the information receiving unit; and a program that executes the program when the information receiving unit receives the sound information. the program for causing the program to function as a growth unit that acquires the goal information paired with the one state determined by the state determination unit, and performs processing to grow node information or edge information corresponding to one or more node identifiers among the one or more node identifiers determined by the firing node determination unit, using the goal information; a feature acquisition unit that uses the received image information to acquire one or more feature information for the image information; an ignition node determination unit that determines a node identifier of a node to be ignited, the node identifier corresponding to each of the one or more feature information acquired by the feature acquisition unit, from the starting point storage unit;

[0263] In this embodiment, the NN growing device 1 may have a structure as shown in the block diagram of Fig. 13. That is, the processing unit 13 of the NN growing device 1 may also include a window determining unit .

[0264] The window determination unit 130 determines a sliding window (hereinafter referred to as "window" as appropriate), which is a partial area of ​​a still image or a still image constituting a video received by the information receiving unit 121. The window is the area that the baby focuses on and looks at, so it may also be called a focus area. Furthermore, it is preferable that the size of the window is constant, but it may also be variable. Furthermore, the shape of the window is, for example, rectangular, but may also be other shapes such as ellipse or circle.

[0265] The window determination unit 130 sequentially determines windows corresponding to the reference point while changing the reference point of the window, which is, for example, a reference point in a still image constituting a still image or a moving image received by the information receiving unit 121. The reference point of a window is, for example, the center of gravity of the window or a specific end point (for example, the coordinate value of the upper left corner).

[0266] The window determination unit 130 passes, for example, a partial image, which is an image within the determined window, to the feature acquisition unit 132. The partial image is an image of a partial area of ​​a still image, which is a still image or a still image constituting a moving image, received by the information receiving unit 121.

[0267] For example, the window determination unit 130 generates a random number corresponding to the value of X, obtains the value of X using the random number, and generates a random number corresponding to the value of Y, obtains the value of Y using the random number. For example, the window determination unit 130 generates a random number corresponding to the previous reference point (X1, Y1) and the value of X (X R ), a random number corresponding to the value of Y (Y R ) and the next reference point (X1+X R ,Y1+Y R ), or (X1-X R ,Y1-Y R ) is acquired. In this case, the acquired random number is the amount of change in the reference point. Note that the window determination unit 130 may, for example, regularly acquire a new reference point relative to the previous reference point.

[0268] The reference point is, for example, an initial reference point, a first reference point, or a second reference point. The initial reference point is a reference point used when a window is first determined from image information. The initial reference point may be, for example, the center point of the image information, but is not limited to this. The initial reference point may be determined, for example, by obtaining X and Y coordinate values ​​using random numbers. The first reference point is a point slightly moved from a previously determined reference point (old reference point) and is the reference point of the next window. The amount of change at the slightly moved point is, for example, the value of a generated random number, but is not limited to this and may be fixed. The second reference point is a reference point used when the firing node satisfies the target condition, and is a point slightly moved from the old reference point. Note that the amount of change at the first reference point is greater than the amount of change at the second reference point.

[0269] For example, the window determination unit 130 acquires a partial image of the size of a window, centered on a reference point, from a still image constituting a still image or video accepted by the information acceptance unit 121. For example, the window determination unit 130 acquires area information (for example, upper left coordinate values ​​and lower right coordinate values, coordinate values ​​of a center point and a radius) that specifies an area of ​​the partial image of the size of a window, centered on a reference point.

[0270] The process of the window determination unit 130 that determines a window using such random numbers corresponds to a saccade, which is a movement of the eyeball.

[0271] For example, the window determination unit 130 determines whether the node identifier of each of the one or more fired nodes matches any of one or more focus conditions, and if the focus condition is matched, determines the image (here, typically a partial image) that caused the one or more nodes to fire as the window of interest. After determining the window of interest, the window determination unit 130 generates a random number within a small range of values, determines a window corresponding to a second reference point that is shifted by the random number from the old reference point of the window of interest, and repeats the process of setting the window as the window of interest.

[0272] The focus condition is information for determining whether an image is one that an infant will be attracted to or wants to see. The focus condition has, for example, one or more node identifiers. The focus condition has, for example, two or more node identifiers (a set of identifiers for a node group) and a condition regarding the threshold of the fired nodes (for example, "a percentage of nodes in a specific node group that is equal to or greater than the threshold has fired," or "a number of nodes in a specific node group that is equal to or greater than the threshold has fired").

[0273] The node identifier of the fired node is the node identifier acquired by the firing node determination unit 133.

[0274] It is preferable that the window determination unit 130 determines a new window until, for example, a transition condition is satisfied. The transition condition is a condition for using the next image information, etc. The transition condition is, for example, "a window has been determined a threshold number of times from one image information," "a threshold time or more has elapsed since the reception of one image information," or "the next image information, etc. has been received." However, the transition condition is not limited.

[0275] The feature acquisition unit 132 in FIG. 13 acquires one or more pieces of feature information for a partial image by using the partial image corresponding to the sliding window sequentially determined by the window determination unit 130, for example.

[0276] The feature acquisition unit 132 acquires the amount of movement between two partial images, for example, by using a first partial image in a window determined by the window determination unit 130, which is an area in one piece of image information, and a second partial image in a window determined by the window determination unit 130, which is an area in the image information temporally preceding the one piece of image information. Such amount of movement is an example of feature information of the moving image acquired by the feature acquisition unit 132.

[0277] The feature acquisition unit 132 may acquire, for example, a first movement amount which is the movement amount between a first partial image and a second partial image immediately preceding the first partial image, a second movement amount which is the movement amount between the first partial image and a second partial image preceding the first partial image, and an Nth movement amount which is the movement amount between the first partial image and an Nth partial image preceding the first partial image by N, and form a movement amount vector (first movement amount, second movement amount, . . . , Nth movement amount). Such a movement amount vector is also an example of feature information of a moving image acquired by the feature acquisition unit 132.

[0278] The amount of movement between images is information that specifies the amount of movement between two images, and is, for example, a motion vector or optical flow.

[0279] The window determination unit 130 can usually be realized by a processor, memory, etc. The processing procedure of the window determination unit 130 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may also be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0280] The NN growth device 1 in Fig. 13 may, for example, perform the operation of the flowchart shown in Fig. 14. The operation example of the NN growth device 1 shown in Fig. 14 is a case where the node determination unit 130 is functioning. In the flowchart of Fig. 14, explanations of the same steps as in the flowchart of Fig. 2 will be omitted.

[0281] (Step S1401) The window determination unit 130 performs a window determination process. An example of the window determination process will be described with reference to the flowchart in FIG.

[0282] (Step S1402) The feature acquiring unit 132 acquires one or more pieces of feature information using the partial image acquired in step S1401.

[0283] (Step S1403) Window determination unit 130 determines whether or not a transition condition is met, which is a condition for using the next image information, etc. If the transition condition is met, the process returns to step S201, and if the transition condition is not met, the process goes to step S1401.

[0284] Next, an example of the window determination process in step S1401 will be described with reference to the flowchart in FIG.

[0285] (Step S1501) The window determination unit 130 determines whether or not a previous reference point exists, which is a reference point for the image information received in S201 and is the reference point for the previously determined window. If a previous reference point exists, the process proceeds to step S1502; if a previous reference point does not exist, the process proceeds to step S1510.

[0286] (Step S1502) The window determination unit 130 assigns 1 to a counter i.

[0287] (Step S1503) The window determination unit 130 determines whether or not the i-th condition of interest exists in the storage unit 11. If the i-th condition of interest exists, the process proceeds to step S1504, and if not, the process proceeds to step S1508.

[0288] (Step S1504) The window determination unit 130 acquires the node identifiers of one or more firing nodes corresponding to the i-th condition of interest.

[0289] (Step S1505) The window determination unit 130 determines whether or not the one or more node identifiers acquired in step S1504 satisfy the i-th focus condition. If the i-th focus condition is satisfied, the process proceeds to step S1506; if not, the process proceeds to step S1513.

[0290] (Step S1506) The window determination unit 130 acquires the old reference point.

[0291] (Step S1507) The window determination unit 130 acquires a second reference point based on the old reference point. The process proceeds to step S1511. The second reference point is a point at a position slightly moved from the old reference point.

[0292] (Step S1508) The window determination unit 130 acquires the old reference point. (Step S1509) The window determination unit 130 acquires a first reference point based on the old reference point. The process proceeds to step S1511. The first reference point is a point located a short distance from the old reference point. It is also preferable that the distances moved from the old reference point when determining the first and second reference points are different.

[0293] (Step S1510) The window determination unit 130 acquires an initial reference point for one piece of image information. Then, the process proceeds to step S1511.

[0294] (Step S1511) Window determination unit 130 acquires area information that identifies the area of ​​the window using the acquired reference point. Note that the acquired reference point is the initial reference point, the first reference point, or the second reference point.

[0295] (Step S1512) Window determination unit 130 acquires a partial image, which is an image of the area specified by the area information acquired in step S1511, from the image information accepted in step S201, and returns to the upper-level processing.

[0296] (Step S1513) The window determination unit 130 increments the counter i by 1. The process returns to step S1503.

[0297] (Embodiment 2) In this embodiment, an information processing device will be described that uses neural network information generated by the NN generating device 1 to obtain firing patterns for received image information and / or sound information, and outputs information for the firing patterns.

[0298] In this embodiment, an information processing device in which the speed of information transmission between nodes changes depending on temperature information will be described.

[0299] 16 is a block diagram of information processing device 2 according to the present embodiment. Information processing device 2 includes storage unit 21, reception unit 22, processing unit 23, and output unit 24. Storage unit 21 includes NN storage unit 113. Reception unit 22 includes information reception unit 221 and temperature reception unit 222. Processing unit 23 includes feature acquisition unit 231, information transmission unit 232, firing pattern acquisition unit 233, and output information acquisition unit 234. Output unit 24 includes information output unit 241.

[0300] Various types of information are stored in the storage unit 21 that constitutes the information processing device 2. The various types of information include, for example, neural network information, one or more pieces of firing start point information, and one or more pieces of output management information.

[0301] The firing start point information is information that includes an information identifier that identifies the characteristic information of the received information, and one or more node identifiers that identify the node that will be fired first when the characteristic information is received.

[0302] Reception information is information received by the information receiving unit 221. Reception information includes image information or sound information. Reception information may be two or more types of information. Reception information may include, for example, tactile information and smell information. Tactile information is information related to the sense of touch. Smell information is information related to smell.

[0303] The output management information is information that includes output conditions and output information, and may be information that pairs the output conditions and the output information.

[0304] An output condition is a condition used to determine output information. An output condition is a condition for output using a firing pattern. An output condition may be the firing pattern itself, or information having a firing pattern and output probability information. Output probability information is information regarding the probability for obtaining output information. An output condition may be information regarding a firing pattern and a lower limit of the number of node identifiers that the applied firing pattern has, or information regarding a lower limit of the ratio of node identifiers that the applied firing pattern has. A firing pattern has one or more node identifiers. A firing pattern is a pattern of firing one or more nodes. Output information is information corresponding to a firing pattern.

[0305] The output information is, for example, emotional information relating to the emotions of a person (especially an infant), behavioral information relating to the body movements of a person (especially an infant), etc. Emotional information is, for example, happy, sad, frightened, surprised, etc. Emotional information is, for example, an ID that identifies an emotion. Emotional information may also be the above-mentioned state identifier. Behavioral information is, for example, information reflected in the movements of an avatar (character). Behavioral information is, for example, information reflected in the movements of an infant avatar. Note that the technology for operating an avatar is well known, so a detailed description will be omitted.

[0306] The output condition may be a condition that uses the firing pattern and information about one or more pieces of external information. The external information is information from the outside. The external information may also be called user context. Examples of the external information include temperature, weather, smell, sound, light, etc.

[0307] The NN storage unit 113 stores neural network information accumulated by the NN growing device 1.

[0308] The reception unit 22 receives various types of information, such as reception information and temperature information.

[0309] The information receiving unit 221 receives received information. The information receiving unit 221 receives, for example, image information captured by a camera. The information receiving unit 221 may also receive sound information captured by a microphone.

[0310] Here, reception is a concept that includes reception of information acquired by devices such as cameras and microphones, reception of information transmitted via wired or wireless communication lines, and reception of information read from recording media such as optical disks, magnetic disks, and semiconductor memories.

[0311] The temperature receiving unit 222 receives temperature information. The temperature information is information that specifies a temperature. The temperature is, for example, the temperature of the external environment.

[0312] Here, reception is a concept that includes reception of information input from input devices such as a microphone, keyboard, mouse, or touch panel, reception of information transmitted via a wired or wireless communication line, and reception of information read from recording media such as an optical disk, magnetic disk, or semiconductor memory.

[0313] The processing unit 23 performs various types of processing, such as processing performed by a feature acquisition unit 231, an information transmission unit 232, an ignition pattern acquisition unit 233, and an output information acquisition unit 234.

[0314] The feature acquisition unit 231 acquires one or more pieces of feature information for the image information, using the image information accepted by the information acceptance unit 221. The processing performed by the feature acquisition unit 231 may be the same as the processing performed by the feature acquisition unit 132.

[0315] The information transmission unit 232 determines, from one or more pieces of firing start point information, a node identifier corresponding to one or more pieces of feature information acquired by the feature acquisition unit 231. Such a node identifier is the identifier of a node that will fire. Moreover, such a node identifier is the identifier of a node that will fire in the first stage.

[0316] Next, the information transmission unit 232 determines the node identifier of a node that is connected by an edge to the node identified by the one or more determined node identifiers, receives the feature information, and is to fire.

[0317] The information transmission unit 232 performs an information transmission process, which is a process of passing feature information from one firing node to the next firing node. The next firing node is a node that is connected to the one firing node by one edge and is determined to fire.

[0318] The information transmission unit 232 acquires node information of nodes connected to one firing node by an edge. Next, the information transmission unit 232 determines whether the node information satisfies the firing condition. Then, the information transmission unit 232 composes and stores firing information including the node identifier of the node information that satisfies the firing condition.

[0319] For example, when the information transmission unit 232 determines that the node information satisfies the firing condition, the information transmission unit 232 determines whether or not the node information will fire at the probability indicated by the firing probability information of the node information. Then, when the information transmission unit 232 determines that the node information will fire based on the firing condition and the probability indicated by the firing probability information, the information transmission unit 232 configures and stores firing information including the node identifier of the node information.

[0320] It is preferable that the information transmission unit 232 changes the processing time for performing the information transmission process according to the temperature information received by the temperature reception unit 222. For example, the information transmission unit 232 performs the information transmission process more quickly when the temperature information indicates a high temperature than when the temperature information indicates a lower temperature. For example, the information transmission unit 232 delays the information transmission process when the temperature information indicates a low temperature compared to when the temperature information indicates a higher temperature. For example, the information transmission unit 232 delays the information transmission process for a predetermined time when a delay condition is met. Delaying the information transmission process means, for example, waiting for a predetermined time. The delay condition is a condition for delaying the process and is a condition based on the temperature information. The delay condition is, for example, "temperature information<=threshold value" or "temperature information<threshold value."

[0321] It is preferable that the information transmission unit 232 increases the firing probability information included in the node information of the firing node, because the more a node fires, the easier it becomes to fire that node.

[0322] The firing pattern acquisition unit 233 acquires a firing pattern using one or more node identifiers determined by the information transmission unit 232. A firing pattern is a collection of information that identifies one or more firing nodes. A firing pattern is usually information that identifies nodes that fire simultaneously. A firing pattern has one or more node identifiers.

[0323] It is preferable that the firing pattern acquisition unit 233 acquires a firing pattern having one or more node identifiers of the nodes that finally fired. Note that the node that finally fired is a node that, among the nodes that fired, did not transmit information to other nodes connected by edges.

[0324] The output information acquisition unit 234 acquires output information corresponding to the firing pattern acquired by the firing pattern acquisition unit 233. The output information acquisition unit 234 refers to one or more pieces of output management information in the storage unit 21, for example, and acquires output information corresponding to the output condition satisfied by the firing pattern acquired by the firing pattern acquisition unit 233.

[0325] The output information acquisition unit 234, for example, refers to one or more pieces of output management information in the storage unit 21 and determines a firing pattern of an output condition satisfied by the firing pattern acquired by the firing pattern acquisition unit 233. Then, the output information acquisition unit 234 determines whether or not to acquire output information, for example, with a probability based on output probability information paired with the determined firing pattern, and if it is determined to acquire the output information, acquires the output information contained in the output management information.

[0326] The output unit 24 outputs various types of information, such as output information.

[0327] The information output unit 241 outputs the output information acquired by the output information acquisition unit 234. Here, output is a concept that includes display on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage in a recording medium, and delivery of processing results to another processing device, another program, etc.

[0328] The storage unit 21 and the NN storage unit 113 are preferably non-volatile recording media, but can also be realized as volatile recording media.

[0329] There is no restriction on the process by which information is stored in the storage unit 21 etc. For example, information may be stored in the storage unit 21 etc. via a recording medium, information transmitted via a communication line etc. may be stored in the storage unit 21 etc., or information input via an input device may be stored in the storage unit 21 etc.

[0330] The reception unit 22, the information reception unit 221, and the temperature reception unit 222 are realized, for example, by a camera, a microphone, a wireless or wired communication means, a means for receiving broadcasts, a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0331] The processing unit 23, the feature acquisition unit 231, the information transmission unit 232, the firing pattern acquisition unit 233, and the output information acquisition unit 234 can usually be realized by a processor, a memory, etc. The processing procedures of the processing unit 23, etc. are usually realized by software, and the software is recorded on a recording medium such as a ROM. However, they may also be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0332] The output unit 24 and the information output unit 241 may or may not include output devices such as a display, a speaker, etc. The output unit 24, etc. may be realized by driver software for an output device, or a combination of driver software for an output device and an output device, etc.

[0333] Next, an example of the operation of the information processing device 2 will be described with reference to the flowchart of FIG.

[0334] (Step S1701) Information receiving unit 221 determines whether or not reception information has been received. If reception information has been received, the process proceeds to step S1702, and if reception information has not been received, the process returns to step S1701.

[0335] (Step S1702) The feature acquisition unit 231 acquires one or more pieces of feature information from the reception information accepted in step S1701. The feature acquisition unit 231 acquires one or more pieces of feature information from the image information accepted in step S1701, for example. The feature acquisition unit 231 acquires one or more pieces of feature information from the sound information accepted in step S1701, for example.

[0336] (Step S1703) The temperature receiving unit 222 acquires temperature information.

[0337] (Step S1704) The information transmission unit 232 performs information transmission processing within the neural network. An example of the information transmission processing will be described with reference to the flowchart in FIG.

[0338] (Step S1705) The firing pattern acquisition unit 233 acquires a firing pattern having one or more node identifiers of the node that fired last in step S1704. Note that the node that fired last is a node that fired and did not pass its feature information to other nodes.

[0339] (Step S1706) The output information acquisition unit 234 acquires output information corresponding to the firing pattern acquired in step S1705.

[0340] (Step S1707) If the output information can be acquired in step S1706, the process proceeds to step S1708, and if not, the process returns to step S1701.

[0341] (Step S1708) The information output unit 241 outputs the information acquired in step S1706. The process returns to step S1701.

[0342] In the flowchart of FIG. 17, the processing unit 23 may have a state determination unit 131 and a growth unit 134 and perform the above-described growth processing.

[0343] In the flowchart of FIG. 17, the process ends when the power is turned off or an interrupt occurs to end the process.

[0344] Next, an example of the information transmission process in step S1704 will be described with reference to the flowchart in FIG.

[0345] (Step S1801) The information transmission unit 232 assigns 1 to a counter i.

[0346] (Step S1802) The information transmission unit 232 determines whether or not the i-th feature information exists among the feature information acquired in step S1702. If the i-th feature information exists, the process proceeds to step S1803, and if not, the process returns to the upper level process.

[0347] (Step S1803) The information transmission unit 232 refers to one or more pieces of firing start point information in the storage unit 21, and acquires node identifiers contained in each of the one or more pieces of firing start point information that the i-th feature information satisfies. The information transmission unit 232 composes firing information including the node identifiers, and stores the information in the storage unit 21. It is preferable that the information transmission unit 232 acquires timer information indicating the time of firing from a clock (not shown), composes firing information including the timer information and the node identifiers, and stores the information in the storage unit 21. The one or more node identifiers are identifiers of nodes that fire in the first stage. Also, the node identifiers are firing node identifiers. It is also possible that the information transmission unit 232 is not capable of acquiring the firing node identifiers.

[0348] (Step S1804) The information transmission unit 232 assigns 1 to the counter j.

[0349] (Step S1805) The information transmission unit 232 determines whether or not the j-th ignition node identifier exists among the ignition node identifiers acquired in step S1803. If the j-th ignition node identifier exists, the process proceeds to step S1806; if not, the process proceeds to step S1810.

[0350] (Step S1806) The information transmission unit 232 performs a process of adding the i-th feature information to the node identified by the j-th firing node identifier. Note that the process of adding the i-th feature information to the node is, for example, a process of writing the i-th feature information to the node information of the node, and a process of associating the i-th feature information with the node information of the node.

[0351] (Step S1807) The information transmission unit 232 updates the number of times information included in the node information corresponding to the j-th ignition node identifier to increase it. For example, the information transmission unit 232 reads out the number of times information included in the node information corresponding to the j-th ignition node identifier, and overwrites the number of times information with the number of times information added by 1.

[0352] (Step S1808) The information transmission unit 232 performs next transmission processing using the j-th ignition node identifier as the node identifier of interest. An example of the next transmission processing will be described with reference to the flowchart of FIG.

[0353] The next transmission process is a process of transmitting feature information for the node identified by the target node identifier to the node that is connected to the node identified by the target node identifier by an edge and that will fire. The process of transmitting feature information is an information transmission process.

[0354] (Step S1809) The information transmission unit 232 increments the counter j by 1. The process returns to step S1805.

[0355] (Step S1810) The information transmission unit 232 increments the counter i by 1. The process returns to step S1802.

[0356] Next, an example of the next transmission process in step S1808 will be described with reference to the flowchart in FIG.

[0357] (Step S1901) The information transmission unit 232 determines whether the acquired temperature information matches the delay condition. If the delay condition is met, the process proceeds to step S1902, and if the delay condition is not met, the process proceeds to step S1903.

[0358] (Step S1902) The information transmission unit 232 waits. Note that it is preferable that the wait time is determined in advance, but this is not limiting.

[0359] (Step S1903) The information transmission unit 232 acquires all edge information including the ignition node identifier of interest from the NN storage unit 113.

[0360] (Step S1904) The information transmission unit 232 assigns 1 to the counter i.

[0361] (Step S1905) The information transmission unit 232 determines whether or not the i-th edge information exists among the edge information acquired in step S1901. If the i-th edge information exists, the process proceeds to step S1906, and if not, the process returns to the upper level process.

[0362] (Step S1906) The information transmission unit 232 determines whether or not the node identifier of another node exists in the i-th edge information. If the node identifier of another node exists, the process proceeds to step S1907; if not, the process proceeds to step S1914. The node identifier of another node in the edge information is the node identifier of the node to which the edge is connected. If the node identifier of another node exists in the i-th edge information, the edge is connected to two nodes.

[0363] (Step S1907) The information transmission unit 232 acquires the node identifier of another node in the i-th edge information. Next, the information transmission unit 232 acquires the node information of the node identified by the node identifier from the NN storage unit 113.

[0364] (Step S1908) The information transmission unit 232 uses the node information acquired in step S1907 to determine whether or not the node corresponding to the node information will fire. An example of such firing determination processing will be described with reference to the flowchart in FIG.

[0365] (Step S1909) If the determination result in step S1908 is "fire", the information transmission unit 232 proceeds to step S1910, and if the determination result is "not fire", the information transmission unit 232 proceeds to step S1914.

[0366] (Step S1910) The information transmission unit 232 acquires ignition information having the node identifier included in the node information acquired in step S1907, and accumulates the ignition information in the storage unit 11.

[0367] (Step S1911) The information transmission unit 232 changes the firing probability information included in the node information acquired in step S1907. Here, the information transmission unit 232 changes the firing probability information so that the firing probability specified by the firing probability information increases.

[0368] (Step S1912) The information transmission unit 232 determines whether or not to terminate the transmission of information between nodes. If the transmission is to be terminated, the process proceeds to step S1914, and if the transmission is not to be terminated, the process proceeds to step S1913. The transmission is terminated, for example, when the node in question is the terminal node in the NN. Furthermore, when the transmission is to be terminated, the node identifiers contained in the firing information accumulated immediately before in step S1910 are the node identifiers that constitute the firing pattern.

[0369] (Step S1913) The information transmission unit 232 performs next transmission processing with the node in question as the node of interest. An example of the next transmission processing is shown in FIG.

[0370] (Step S1914) The information transmission unit 232 increments the counter i by 1. The process returns to step S1905.

[0371] In the flowchart of FIG. 19, when transmitting information between nodes, the information transmission unit 232 preferably updates the retained energy amount information by subtracting the amount of energy indicated by the retained energy amount information included in the node information that caused the ignition. This may also be applied to the retained energy amount information paired with the AXON identifier of the AXON used for the transmission, and the retained energy amount information paired with the Dendrites identifier of the Dendrites used for the transmission. The function for reducing the amount of energy is stored, for example, in the storage unit 21. The function in question is not important. Because the function is a well-known technique, detailed description thereof will be omitted.

[0372] Next, an example of the firing determination process in step S1908 will be described with reference to the flowchart in FIG.

[0373] (Step S2001) The information transmission unit 232 acquires the firing condition corresponding to the node information acquired in step S1905. The firing condition may be different for each node or may be common to two or more nodes.

[0374] (Step S2002) The information transfer unit 232 acquires one or more pieces of characteristic information. Here, the one or more pieces of characteristic information are transferred from the node that is the source of firing.

[0375] (Step S2003) The information transmission unit 232 determines whether or not the one or more pieces of feature information acquired in step S2002 satisfy the firing condition acquired in step S2001. If the firing condition is satisfied, the process proceeds to step S2004, and if the firing condition is not satisfied, the process proceeds to step S2007.

[0376] (Step S2004) The information transmission unit 232 determines whether the node information of interest has firing probability information. If it has firing probability information, the process proceeds to step S2005, and if it does not have firing probability information, the process proceeds to step S2006.

[0377] (Step S2005) The information transmission unit 232 acquires firing probability information included in the node information of interest. Next, the information transmission unit 232 uses the firing probability information to determine whether or not firing will occur. If firing will occur, the process proceeds to step S2006, and if not, the process proceeds to step S2007.

[0378] (Step S2006) The information transmission unit 232 assigns "fire" to the determination result, and returns to the upper level processing.

[0379] (Step S2007) The information transmission unit 232 assigns "does not fire" to the determination result, and returns to the upper level processing.

[0380] As described above, according to this embodiment, it is possible to simulate the operation of an infant's growing brain.

[0381] In this embodiment, the information processing device may also realize the growth process performed by the neural network growth device 1. In this case, the information processing device can output output information in response to received information while growing a neural network. In this case, the information processing device is the information processing device 3. In addition to the configuration of the information processing device 2, the information processing device 3 has a state determination unit 131 and a growth unit 134 that the neural network growth device 1 has. A block diagram of the information processing device 3 in this case is shown in FIG. 21.

[0382] 21, the processing of the firing node determination unit 133 of the NN growing device is performed by the information transmission unit 232. Also, the feature acquisition unit 231 is the same as the feature acquisition unit 132.

[0383] The software that realizes the information processing device 2 in this embodiment is the following program. In other words, this program causes a computer that can access an NN storage unit in which neural network information accumulated by the NN growing device 1 is stored to function as an information receiving unit that receives received information, which is one or more pieces of information selected from image information and sound information; a feature acquisition unit that acquires one or more pieces of feature information for the received information received by the information receiving unit; an information transmission unit that determines a node identifier of a node that will fire, which is a node identifier corresponding to each of the one or more pieces of feature information acquired by the feature acquisition unit, from a start point storage unit that stores one or more pieces of firing start point information having an information identifier that identifies the feature information of the received information and one or more node identifiers that identify a node that will fire first when the feature information is accepted, and determines the node identifier of a node that is connected by an edge to each of the nodes identified by the one or more node identifiers and that will fire, which is a node to which the feature information is passed; a firing pattern acquisition unit that acquires a firing pattern using the one or more node identifiers determined by the information transmission unit; an output information acquisition unit that acquires output information corresponding to the firing pattern acquired by the firing pattern acquisition unit; and an information output unit that outputs the output information.

[0384] 22 shows the appearance of a computer that executes the programs described in this specification to realize the NN growth device 1, information processing device 2, and information processing device 3 of the various embodiments described above. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 22 is an overview of this computer system 300, and FIG. 23 is a block diagram of the system 300.

[0385] In FIG. 22, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, a monitor 304, a microphone 305, and a camera 306.

[0386] 23, computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing instructions of application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data. Although not shown here, computer 301 may further include a network card for providing connection to a LAN.

[0387] A program that causes the computer system 300 to execute the functions of the NN growth device 1 and the like of the above-described embodiment may be stored on a CD-ROM 3101, inserted into the CD-ROM drive 3012, and then transferred to the hard disk 3017. Alternatively, the program may be transmitted to the computer 301 via a network (not shown) and stored on the hard disk 3017. The program is loaded into the RAM 3016 when executed. The program may also be loaded directly from the CD-ROM 3101 or the network.

[0388] The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 301 to execute the functions of the NN growth device 1 of the above-described embodiment. The program only needs to include instructions that call appropriate functions (modules) in a controlled manner and achieve desired results. How the computer system 300 operates is well known, and a detailed description thereof will be omitted.

[0389] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmission step (processing that can only be performed by hardware).

[0390] The computer that executes the program may be a single computer or a plurality of computers, that is, it may perform centralized processing or distributed processing.

[0391] Furthermore, in each of the above embodiments, it goes without saying that two or more communication means present in one device may be physically realized by one medium.

[0392] Furthermore, in each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0393] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention. [Industrial Applicability]

[0394] As described above, the neural network growing device according to the present invention has the effect of being able to simulate the growth of an infant's brain, and is useful as a neural network growing device, etc.

Claims

1. an NN storage unit in which neural network information is stored, the neural network information having two or more pieces of node information each having a node identifier and one or more pieces of edge information each having an edge identifier and specifying a connection between the nodes; a start point storage unit for storing one or more pieces of firing start point information each having an information identifier for identifying feature information of image information and one or more node identifiers for identifying a node that will be first fired when the feature information is received; a goal storage unit for storing goal information that identifies goals corresponding to two or more states including positive and negative states; an information receiving unit that receives image information and sound information; a state determination unit that determines one state from the two or more states using the sound information received by the information receiving unit; a feature acquisition unit that acquires one or more pieces of feature information for the image information using the image information accepted by the information acceptance unit; an ignition node determination unit that determines, from the starting point storage unit, a node identifier of a node that will ignite, the node identifier corresponding to the one or more pieces of feature information acquired by the feature acquisition unit, and that is connected by an edge to the node identified by the one or more node identifiers and receives the feature information, the node identifier of a node that will ignite; a growth unit that acquires the goal information that pairs with the one state determined by the state determination unit, and performs processing to grow node information or edge information corresponding to each of one or more node identifiers among the one or more node identifiers determined by the firing node determination unit using the goal information.

2. a window determination unit that sequentially determines a sliding window, which is a partial area of ​​a still image, from the still image included in the image information received by the information receiving unit; The feature acquisition unit 2. The neural network growing apparatus according to claim 1, wherein the window determining unit sequentially uses the partial images corresponding to the determined sliding windows to obtain one or more pieces of feature information for the partial images.

3. the characteristic information has an information identifier for identifying the information and an information amount indicating the size of the information, The firing node determination unit 2. The NN growing device according to claim 1, which determines whether one or more pieces of feature information passed from one or more other nodes connected by edges satisfy a firing condition for one or more pieces of feature information, and determines the node identifier of the node determined to satisfy the firing condition.

4. the node information includes node position information that identifies the position of the node; the goal information includes goal position information that identifies the position of the goal, or goal direction information that indicates the direction of the goal, The growth portion is 2. The neural network growing device according to claim 1, wherein edge generation processing is performed to generate and store edge information for edges extending from nodes identified by one or more node identifiers among the one or more node identifiers determined by the firing node determination unit in a direction indicated by the goal information paired with the one state determined by the state determination unit.

5. The firing node determination unit accumulating frequency information relating to the number of times of the determined node identifier in association with the node identifier; The growth portion is 5. The neural network growing device according to claim 4, wherein the edge generation process is performed on a node identified by a node identifier corresponding to the number information that matches an edge generation condition.

6. the node information includes node position information that identifies the position of the node; the goal information includes goal position information that identifies the position of the goal, or goal direction information that indicates the direction of the goal, The growth portion is 2. The NN growing device according to claim 1, wherein edge growing processing is performed to acquire and store edge information obtained by growing edges extending from nodes identified by one or more node identifiers among the one or more node identifiers determined by the firing node determination unit in a direction indicated by the goal information paired with the one state determined by the state determination unit.

7. The firing node determination unit accumulating frequency information relating to the number of times of the determined node identifier in association with the node identifier; The growth portion is 7. The NN growing device according to claim 6, wherein the edge growing process is performed on a node identified by a node identifier corresponding to the number information that matches an edge generation condition.

8. the node is soma, The edge has an AXON and dendrites, The NN growing apparatus according to claim 1 , wherein the edge information includes AXON information having an AXON identifier and AXON position information indicating the position of the AXON, and dendrite information having a dendrite identifier and dendrite position information indicating the position of the dendrite.

9. a neural network storage unit for storing neural network information accumulated by the neural network growing device according to any one of claims 1 to 8; an information receiving unit that receives reception information, which is one or more pieces of information selected from image information and sound information; a characteristic acquisition unit that acquires one or more pieces of characteristic information for the received information received by the information receiving unit; an information transmission unit that determines a node identifier of a node to be ignited, the node identifier corresponding to the one or more pieces of feature information acquired by the feature acquisition unit, from a start point storage unit that stores one or more pieces of ignition start point information having an information identifier that identifies the feature information of the received information and one or more node identifiers that identify a node that will be ignited first when the feature information is accepted, and that is connected by an edge to the nodes identified by the one or more node identifiers and that receives the feature information; and a firing pattern acquisition unit that acquires firing patterns using one or more node identifiers determined by the information transmission unit; an output information acquisition unit that acquires output information corresponding to the ignition pattern acquired by the ignition pattern acquisition unit; and an information output unit that outputs the output information.

10. Further comprising a temperature receiving unit that receives temperature information, The information transmission unit 10. The information processing device according to claim 9, wherein an information transmission process is performed in which the fired node passes the feature information corresponding to the fired node to the next fired node, and the processing time for performing the information transmission process is changed depending on the temperature information received by the temperature receiving unit.

11. a starting point storage unit storing one or more pieces of firing starting point information having an information identifier that identifies feature information of image information and one or more node identifiers that identify a node that will be fired first when the feature information is received; a goal storage unit storing goal information that identifies a goal corresponding to each of two or more states including positive and negative; an information receiving unit; a state determining unit; a feature acquiring unit; an firing node determining unit; and a growing unit, an information receiving step in which the information receiving unit receives image information and sound information; a state determination step in which the state determination unit determines one state from the two or more states by using the sound information received in the information receiving step; a feature acquisition step in which the feature acquisition unit acquires one or more pieces of feature information for the image information by using the image information accepted in the information acceptance step; an ignition node determination step in which an ignition node determination unit determines, from the starting point storage unit, a node identifier of a node to be ignited, which is a node identifier corresponding to the one or more pieces of feature information acquired in the feature acquisition step, and determines a node identifier of a node to be ignited, which is connected by an edge to the node identified by the one or more node identifiers and receives the feature information; a growth step in which the growth unit acquires the goal information paired with the one state determined by the state determination unit, and processes the node information or edge information corresponding to each of one or more node identifiers among the one or more node identifiers determined by the firing node determination unit using the goal information.

12. a computer that can access an NN storage unit that stores neural network information having two or more pieces of node information each having a node identifier and one or more pieces of edge information each having an edge identifier and specifying a connection between the nodes; a start point storage unit that stores one or more pieces of firing start point information each having an information identifier that identifies feature information of image information and one or more node identifiers that identify a node that will fire first when the feature information is received; and a goal storage unit that stores goal information that specifies a goal corresponding to each of two or more states including positive and negative; an information receiving unit that receives image information and sound information; a state determination unit that determines one state from the two or more states using the sound information received by the information receiving unit; a feature acquisition unit that acquires one or more pieces of feature information for the image information using the image information accepted by the information acceptance unit; an ignition node determination unit that determines, from the starting point storage unit, a node identifier of a node that will ignite, the node identifier corresponding to the one or more pieces of feature information acquired by the feature acquisition unit, and that is connected by an edge to the node identified by the one or more node identifiers and receives the feature information, the node identifier of a node that will ignite; A program for functioning as a growth unit that acquires the goal information that pairs with the one state determined by the state determination unit, and performs processing to grow node information or edge information corresponding to each of one or more node identifiers among the one or more node identifiers determined by the ignition node determination unit using the goal information.

Images