Information processing device, information processing method, and program

Abstract

This information processing device 1 comprises: an MNFP storage unit 113 that stores one or more motor neuron firing patterns (MNFPs), which is information specifying one or more firing nodes at each of two or more points in time, in association with action determination information; an MNFP determination unit 131 that applies input information to a neural network (NN) to acquire one or more firing nodes out of two or more nodes of the NN in time series, and sequentially determines one or more MNFPs corresponding to the one or more firing nodes in time series from the MNFP storage unit 113; an operation determination unit 132 that sequentially acquires operation determination information corresponding to one or more MNFPs determined by the MNFP determination unit 131; and an operation unit 133 that sequentially uses the one or more pieces of operation determination information to operate an operation target 2.

Description

[Technical Field]

[0001] The present invention relates to an information processing device or the like that simulates the movement of a living thing such as a human being in response to its cognition. [Background technology]

[0002] BACKGROUND ART Conventionally, there has been a technique for controlling the motion of a moving object such as a robot or an avatar using a neural network (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-13786 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the prior art, there was no mechanism that could appropriately simulate movements according to the perception of living creatures such as humans. [Means for solving the problem]

[0005] The information processing device of the first invention is an information processing device comprising: an NN storage unit in which a neural network having two or more nodes identified by node identifiers is stored; an MNFP storage unit in which one or more motor neuron firing patterns (MNFPs) are stored, which are information that identifies one or more firing nodes at each of two or more time points in correspondence with action decision information that determines the action of an object to be operated; an input accepting unit that accepts input information; an MNFP determination unit that applies the input information to the neural network, acquires one or more firing node identifiers of the two or more nodes that the neural network has in time series, and sequentially determines one or more MNFPs corresponding to the one or more firing node identifiers in the time series from the one or more MNFPs in the MNFP storage unit; an action decision unit that sequentially acquires action decision information corresponding to each of the one or more MNFPs determined by the MNFP determination unit; and an action unit that sequentially uses the one or more action decision information acquired by the action decision unit to operate the object to be operated.

[0006] With this configuration, it is possible to simulate movements according to the perception of living creatures such as humans.

[0007] In addition, compared to the first invention, the information processing device of the second invention further includes a muscle storage unit in which two or more muscle information pieces are stored, the muscle information being information about muscles that move when one or more nodes are fired, the information being associated with one or more firing node identifiers and having a muscle identifier and a maximum power; the action determination unit includes a muscle determination means that sequentially determines one or more muscle information pieces associated with one or more firing node identifiers identified by each of the one or more MNFPs determined by the MNFP determination unit, and an intensity acquisition means that acquires a firing number, which is the number of firing node identifiers associated with the muscle information, for each of the one or more muscle information pieces determined by the muscle determination means, and sequentially acquires the intensity of the movement of the muscle identified by the muscle information using the firing number and the maximum power of the muscle information; and the action unit is an information processing device that sequentially moves the action target based on one or more action muscle information pieces being pairs of muscle identifiers included in each of the one or more muscle information pieces determined by the muscle determination means and the intensity acquired by the intensity acquisition means.

[0008] With this configuration, it is possible to simulate muscle movements that correspond to the perception of a living being such as a human.

[0009] Furthermore, the information processing device of the third invention, compared to the information processing device of the first invention, further comprises a joint storage unit in which one or more pieces of joint information having joint identifiers are stored in association with two or more muscle identifiers, the movement determination unit further comprises angle acquisition means that refers to the joint storage unit and uses the one or more movement muscle information to sequentially acquire the angles of the joints identified by one or more joint identifiers that are associated with the muscle identifiers contained in the one or more pieces of muscle information determined by the muscle determination means, and the movement unit outputs the angle for each of the one or more joints to the object to be moved, thereby moving the object to be moved.

[0010] With this configuration, it is possible to simulate joint movements that correspond to the perception of a living being such as a human.

[0011] Furthermore, in the information processing device of the fourth invention, compared to the third invention, the object of operation is an avatar, the joint information is information about the avatar's joints and has joint positions that specify the positions of the joints of the avatar, and the information processing device further includes an avatar storage unit in which avatar information for outputting the avatar is stored, and the operation unit uses the avatar information to create an avatar in which the joints specified by the joint positions of each of the one or more joints are bent according to the angles corresponding to the joints, and outputs the avatar.

[0012] With this configuration, joint movements according to the perception of a living being such as a human can be simulated using an avatar.

[0013] Furthermore, the information processing device of the fifth invention differs from the first invention in that the object of operation is the face of an avatar, the information is associated with the node identifiers of one or more nodes, and the information is about a mesh that moves when one or more nodes are fired, and the information further includes a muscle storage unit in which two or more pieces of muscle information having a muscle identifier that identifies the mesh of the avatar's face are stored, the operation determination unit includes muscle determination means that sequentially determines one or more pieces of muscle information associated with each of one or more firing node identifiers identified by each of one or more MNFPs determined by the MNFP determination unit, and the operation unit is an information processing device that sequentially stretches and contracts the meshes identified by the muscle identifiers of the muscle information determined by the muscle determination means.

[0014] With this configuration, it is possible to simulate facial movements according to the perception of a living being such as a human.

[0015] In addition, the information processing device of the sixth invention is different from the fifth invention in that the action determination unit includes a muscle determination means that sequentially determines one or more muscle information associated with one or more firing node identifiers identified by each of one or more MNFPs determined by the MNFP determination unit, and an intensity acquisition means that acquires a firing number, which is the number of firing node identifiers associated with the muscle information, for each of the one or more muscle information determined by the muscle determination means, and sequentially acquires the intensity of the muscle movement identified by the muscle information using the firing number, and the action unit is an information processing device that sequentially moves the action target based on one or more action muscle information, which is a pair of the muscle identifier included in each of the one or more muscle information determined by the muscle determination means and the intensity acquired by the intensity acquisition means.

[0016] With this configuration, facial movements can be appropriately simulated according to the perception of a living creature such as a human.

[0017] In addition, the information processing device of the seventh invention, compared to the first invention, is an information processing device in which the action determination information is an action identifier that identifies the action, and further includes a module storage unit in which an action module for operating the action target is stored in correspondence with the action identifier, the action determination unit acquires the action identifier corresponding to the MNFP determined by the MNFP determination unit, and the action unit executes the action module associated with the action identifier acquired by the action determination unit to operate the action target.

[0018] With this configuration, it is possible to simulate movements according to the perception of living creatures such as humans.

[0019] In addition, the information processing device of the eighth invention is an information processing device according to any one of the second to fourth inventions, further comprising a muscle change unit that performs a change process to increase the maximum power paired with the muscle identifier when the muscle identifier determined by the muscle determination means satisfies a predetermined increase condition.

[0020] With this configuration, muscle movements corresponding to the perception of a living being such as a human can be simulated, taking into account the characteristics of the muscles.

[0021] Furthermore, the information processing device of the ninth invention is an information processing device according to any one of the second to fourth inventions, further comprising a muscle change unit that performs a change process to reduce the maximum power paired with a muscle identifier when a muscle identifier not determined by the muscle determination means satisfies a predetermined reduction condition.

[0022] With this configuration, muscle movements corresponding to the perception of a living being such as a human can be simulated, taking into account the characteristics of the muscles.

[0023] In addition, the information processing device of the tenth invention is an information processing device that, in addition to any one of the first to ninth inventions, further includes a learning unit that performs a learning process to acquire the MNFP when the input receiving unit receives input information, receive action determination information, and store the MNFP in the MNFP storage unit in association with the action determination information.

[0024] With this configuration, it is possible to learn an MNFP for controlling movements according to the perception of living organisms such as humans.

[0025] In addition, the information processing device of the eleventh invention is an information processing device according to any one of the first to ninth inventions, wherein each of one or more of the two or more nodes in the neural network corresponds to a firing condition and to one or more connecting node identifiers that identify other nodes to which the firing is propagated, and the MNFP determination unit is equipped with an initial determination means that acquires one or more feature information from input information accepted by the input accepting unit, determines a firing condition that matches the one or more feature information, and acquires one or more firing node identifiers corresponding to the firing conditions, a propagation means that acquires one or more connecting node identifiers corresponding to each of the one or more firing node identifiers acquired by the initial determination means, and then sequentially acquires one or more connecting node identifiers corresponding to each of the one or more connecting node identifiers, and an MNFP determination means that acquires firing information in chronological order having one or more firing node identifiers from the one or more firing node identifiers and one or more connecting node identifiers acquired by the initial determination means and the propagation means, and sequentially determines one or more MNFPs corresponding to the firing information in the time series from one or more MNFPs in the MNFP storage unit.

[0026] With this configuration, it is possible to simulate movements according to the perception of living creatures such as humans.

[0027] Furthermore, the information processing device of the twelfth invention is an information processing device according to any one of the first to eleventh inventions, in which the number of time points for at least two MNFPs among two or more MNFPs is different.

[0028] With this configuration, it is possible to appropriately simulate movements according to the perception of living creatures such as humans. [Effects of the Invention]

[0029] According to the information processing device of the present invention, it is possible to appropriately simulate the movement of a living thing such as a human being in accordance with its cognition. [Brief explanation of the drawings]

[0030] [Figure 1] Block diagram of operating system A according to the first embodiment. [Figure 2] A flowchart illustrating an example of the operation of the information processing device 1. [Figure 3] A flowchart illustrating an example of the initial determination process [Figure 4] A flowchart illustrating an example of the MNFP determination process [Figure 5] A flowchart illustrating an example of the operation process [Figure 6] Flowchart illustrating an example of angle acquisition processing [Figure 7] A flowchart illustrating an example of the firing propagation process. [Figure 8] A flowchart illustrating an example of the firing elimination process [Figure 9] A schematic diagram illustrating a specific example of the operation of the operating system A. [Figure 10] The NN management table [Figure 11] Figure showing the MNFP management table [Figure 12] Figure showing the muscle management chart [Figure 13] Figure showing the joint management table [Figure 14] FIG. 10 shows an example of the ignition information. [Figure 15] Block diagram of operating system B in embodiment 2 [Figure 16] A flowchart illustrating an example of operation processing performed by the information processing device 3. [Figure 17] Block diagram of operating system C according to the third embodiment. [Figure 18] A flowchart illustrating an example of operation processing performed by the information processing device 3. [Figure 19] A flowchart illustrating an example of the learning process [Figure 20] Flowchart illustrating an example of the MNFP accumulation process [Figure 21]Overview of the computer system in the above embodiment [Figure 22] Block diagram of the computer system DETAILED DESCRIPTION OF THE INVENTION

[0031] Hereinafter, embodiments of an information processing device and the like will be described with reference to the drawings. Note that components with the same reference numerals in the embodiments perform similar operations, and therefore repeated description may be omitted.

[0032] (Embodiment 1) In this embodiment, an information processing device capable of appropriately simulating movements according to the cognition of a living thing such as a human will be described. Specifically, in this embodiment, an information processing device will be described that receives input information such as an image or sound, applies the input information to one or more stored neural networks, acquires one or more motor neuron firing patterns (hereinafter referred to as "MNFPs" as appropriate) that specify the time-series firing states of one or more nodes that are outputs from the neural networks, acquires action-determination information corresponding to the MNFPs, and performs actions corresponding to the one or more action-determination information on an action target.

[0033] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.

[0034] FIG. 1 is a block diagram of an operating system A according to the present embodiment.

[0035] The operating system A includes an information processing device 1 and an operating target 2. The information processing device 1 may also include the operating target 2.

[0036] The information processing device 1 is a device that receives input information and causes an operation target 2 to operate in accordance with the input information.

[0037] The moving object 2 is an object that performs a movement. The moving object 2 may be, for example, an avatar or a robot, but the type of the moving object is not important.

[0038] The information processing device 1 includes a storage unit 11, a reception unit 12, and a processing unit 13. The storage unit 11 includes an NN storage unit 111, an firing start point storage unit 112, an MNFP storage unit 113, an avatar storage unit 114, a muscle storage unit 115, a joint storage unit 116, and an firing information storage unit 117. The reception unit 12 includes an input reception unit 121. The processing unit 13 includes an MNFP determination unit 131, a movement determination unit 132, a movement unit 133, a muscle change unit 134, and a learning unit 135. The MNFP determination unit 131 includes an initial determination means 1311, a propagation means 1312, and an MNFP determination means 1313. The movement determination unit 132 includes a muscle determination means 1321, a strength acquisition means 1322, and an angle acquisition means 1323.

[0039] Various types of information are stored in the storage unit 11 constituting the information processing device 1. The various types of information include, for example, a neural network (hereinafter referred to as "NN" where appropriate) described below, firing start point information described below, MNFP described below, avatar information described below, muscle information described below, joint information described below, and elimination conditions. Note that if the motion target 2 is something other than an avatar, there is no need for avatar information to be stored in the storage unit 11.

[0040] The deletion condition is a condition for an ignited node to become a non-ignited node. The deletion condition is usually a condition related to the time elapsed since ignition. For example, the deletion condition is that the time elapsed since ignition is equal to or longer than a threshold. Note that an ignited node is a node that is ignited. A non-ignited node is a node that is not ignited.

[0041] The NN storage unit 111 stores one or more neural networks (hereinafter referred to as "NN" where appropriate). The NN has two or more pieces of node information identified by node identifiers. The node information is information about the nodes that make up the NN. The node information has, for example, a node identifier and a firing condition. The node information has, for example, one or more connecting node identifiers. The node information may have, for example, a propagation probability associated with each of one or more connecting node identifiers. The node information may also have, for example, one or more original identifiers.

[0042] A node identifier is information that identifies a node. The node identifier is, for example, the node ID or node name.

[0043] The firing condition is a condition for the node to fire. The firing condition can also be said to be information about the condition for the node to fire. The firing condition is, for example, a condition related to one or more pieces of characteristic information. For example, the characteristic information may be an information identifier that identifies the information, information that has an information identifier and an information amount that indicates the size of the information, or the information amount alone.

[0044] The information identifier is, for example, information that specifies the type of image feature, such as red "R" that constitutes color, green "G" that constitutes color, or blue "B" that constitutes color. The information identifier is, for example, information that specifies the type of sound feature, such as "frequency" or "amplitude." The information identifier is, for example, a class of character string. Note that a class is the result of character string classification. For example, a class is "positive" or "negative." For example, a class is "joy," "anger," "sadness," or "pleasure." Note that the technology for classifying character strings into classes is a well-known technology, and therefore a detailed description thereof will be omitted.

[0045] The amount of information is, for example, a numerical value greater than zero.

[0046] Examples of firing conditions include "feature information >= 0.5," "feature information > 0.7," "information amount >= 0.5," "information amount > 0.7," "(information identifier = A & information amount >= 0.5) & (information identifier = B & information amount > 0.8)," etc. The feature information constituting the firing condition is, for example, a feature amount, but may also be the input information itself. The feature amount is, for example, a feature amount of an image resulting from image analysis, such as "R," "G," or "B." The feature amount is, for example, a feature amount of audio resulting from audio analysis, such as frequency or amplitude. It is preferable that the firing condition corresponds to a firing probability. The firing probability is information related to the probability of firing. The firing probability may be the firing probability itself, or a value obtained by converting the firing probability using a function or the like. When the firing condition is satisfied, the firing probability is referenced, and it is preferable that the node fires or does not fire according to the probability indicated by the firing probability.

[0047] The process according to the probability such as the firing probability or the propagation probability may be, for example, a process similar to a lottery with weighting according to the probability. The probability here may be replaced with likelihood.

[0048] The connecting node identifier is identification information of the connecting node. The connecting node is a node connected to the node of interest, and is the node to which the firing of the node of interest is propagated after the firing.

[0049] The propagation probability is the probability that a firing will propagate.

[0050] The source identifier is identification information of the source node. The source node is a node connected to the node of interest, and is a node that propagates the firing to the node of interest after the source node fires.

[0051] One or more pieces of firing start point information are stored in the firing start point storage unit 112. The firing start point information is information for specifying the node that fires first when input information is received.

[0052] The firing start point information includes, for example, one or more firing node identifiers. The firing start point information also includes, for example, a firing condition and one or more firing node identifiers. The firing node identifier is information that identifies the firing node.

[0053] The firing condition here is a condition related to the input information. The firing condition is, for example, a condition related to one or more pieces of feature information. The firing condition has, for example, an information identifier that identifies the feature information. The firing condition is, for example, information indicating specific feature information or a range of specific information. The firing condition is, for example, red "R" that constitutes a color, green "G" that constitutes a color, blue "B" that constitutes a color, the wave number of a sound "30", the frequency of a sound "60", the frequency of a sound being "between threshold A and threshold B", or the class of a character string being a specific class (for example, "negative"), etc.

[0054] The firing start point storage unit 112 may not exist, and the node that fires first when input information is received may be specified using the firing condition paired with the node identifier that identifies the node information of the NN in the NN storage unit 111. In other words, the firing start point storage unit 112 is useful for speeding up the process of specifying the node that fires first when input information is received.

[0055] One or more motor neuron firing patterns (hereinafter referred to as "MNFP") are stored in the MNFP storage unit 113. Each of the one or more MNFPs here corresponds directly or indirectly to action determination information.

[0056] MNFP is information for controlling the behavior of the operating object 2. MNFP is information for specifying one or more firing nodes for each of two or more time points. MNFP is information for specifying a set of firing nodes in a time series. Note that a set of firing nodes is usually two or more firing nodes, but it can also be one firing node.

[0057] MNFP is, for example, information on a matrix with two axes, a time axis and a node identifier. In other words, MNFP is, for example, a set of vectors corresponding to two or more time points. Here, a vector is information that specifies one or more firing node identifiers. For example, a vector is "(n1, n2, , nn) ​​= (1, 0, , 1)". Note that "n1", "n2", , , and "nn" are node identifiers. A value of "1" indicates that the node identified by the corresponding node identifier is a firing node. A value of "0" indicates that the node identified by the corresponding node identifier is a non-firing node.

[0058] An MNFP is, for example, a set of two or more vectors each having a time identifier that identifies two or more time points and one or more firing node identifiers. Here, the two or more time points are usually consecutive time points. An example of an MNFP is "<time identifier>t1 <vector>(1,0,...,1)", "<time identifier>t2 <vector>(1,1,...,1)", "<time identifier>tn <vector>(0,1,...,0)".

[0059] The data structure of the MNFP does not matter. It is also preferable that the number of time points of at least two of the two or more MNFPs in the MNFP storage unit 113 are different. A different number of time points means that the lengths of the time series are different. If the MNFP is a matrix, a different number of time points means that the number of rows or columns of at least two MNFPs are different.

[0060] The movement determination information is information for determining the movement of the moving object 2. The movement determination information is, for example, one or more pieces of moving muscle information, one or more angles of each joint, or one or more movement identifiers.

[0061] The operating muscle information is information that specifies the muscle to be operated. The operating muscle information is a pair of a muscle identifier and strength. The muscle identifier is information that identifies the muscle. The muscle identifier is, for example, the muscle ID or muscle name. Strength is information that indicates the strength when the muscle is contracted. The strength may be the strength when the muscle is contracted or the strength when the muscle is stretched.

[0062] The joint angle is information that specifies the angle of the joint to be operated. The joint angle is usually associated with a joint identifier. The joint identifier is information that identifies the joint. The joint identifier is, for example, the ID or name of the joint.

[0063] An operation identifier is information that identifies an operation. For example, an operation identifier is the name of a module that operates operation target 2, or the ID of a module that operates operation target 2. A module may be thought of as including not only an execution module but also functions, methods, etc.

[0064] Avatar storage unit 114 stores avatar information. Avatar information is information for outputting an avatar. The data structure of the avatar information is not important. However, avatar information usually includes model information. Model information is information for configuring the display of the avatar. Model information includes, for example, mesh information, bone information, and material information. Model information has, for example, a data structure of glTF (GL Transmission Format)? However, the data structure of the model information can be of any type, including VRM, OBJ, FBX, STL, GLB, COLLADA, etc.

[0065] If the information processing device 1 has the avatar storage unit 114, the motion target 2 is an avatar. In this case, it may be considered that the motion target 2 is stored in the avatar storage unit 114.

[0066] Two or more pieces of muscle information are stored in the muscle storage unit 115. The muscle information is information about muscles that move when one or more nodes fire. The muscle information is information associated with one or more firing node identifiers, and is information that includes a muscle identifier and maximum power. Muscle movement means that the muscle contracts or stretches.

[0067] The maximum power is information that specifies the maximum power of a muscle. The maximum power is, for example, a numerical value. The range that the maximum power can take may or may not be determined.

[0068] The joint storage unit 116 stores one or more pieces of joint information. The joint information is information related to a joint. The joint information usually has a joint identifier. The joint identifier is information that identifies a joint. The joint identifier is, for example, a joint ID or a joint name. The joint information is associated with the muscle identifier of each of two or more muscles connected to the joint. The muscle identified by the muscle identifier corresponding to the joint information is the muscle that moves the joint identified by the joint information. The joint information has, for example, a joint position, and the joint position is information that identifies the position of the joint. The joint position is usually the relative position of the joint within the avatar. The joint position is, for example, the coordinate values ​​(x, y) of the center of gravity of the joint. However, the data structure of the joint position is not important.

[0069] If the motion object 2 is an avatar, the joint information may be included in the avatar information.

[0070] Two or more pieces of ignition information are stored in the ignition information storage unit 117. Time-series ignition information is stored in the ignition information storage unit 117. The time-series ignition information is a collection of two or more pieces of ignition information at each time point.

[0071] The firing information here is information that identifies the currently firing node. The firing information is information about the result of firing. The firing information is information that identifies the node that fired at one or more time points. The time-series firing information has the same structure as, for example, an MNFP. In other words, the firing information is information that identifies, for example, one or more firing nodes at two or more time points.

[0072] The reception unit 12 receives various types of information and instructions. The various types of information and instructions are, for example, input information, which will be described later. The means for inputting the various types of information and instructions may be any means, such as a touch panel, a keyboard, a mouse, or a menu screen.

[0073] The input receiving unit 121 receives input information. The input information is, for example, an image or a sound. However, the input information may also be a character string such as a sentence.

[0074] Here, reception is a concept that includes reception of information input from input devices such as a keyboard, mouse, or touch panel, reception of information transmitted via a wired or wireless communication line, and reception of information read from recording media such as an optical disk, magnetic disk, or semiconductor memory.

[0075] The processing unit 13 performs various types of processing. The various types of processing are, for example, processing performed by the MNFP determination unit 131, the movement determination unit 132, the movement unit 133, the muscle change unit 134, and the learning unit 135. The processing unit 13 performs, for example, a firing elimination process, which will be described later. The firing elimination process may be performed by a firing elimination unit (not shown).

[0076] The MNFP determination unit 131 applies the input information received by the input receiving unit 121 to the NN stored in the NN storage unit 111, acquires one or more firing node identifiers from the two or more nodes that the NN has in chronological order, and sequentially determines one or more MNFPs corresponding to the one or more firing node identifiers in the chronological order from one or more MNFPs in the MNFP storage unit 113.

[0077] The application process for applying input information to a NN is, for example, the following process. That is, MNFP determination unit 131 acquires one or more pieces of feature information from the input information. Next, MNFP determination unit 131 references the NN and acquires one or more ignition node identifiers corresponding to the ignition information satisfied by the one or more pieces of feature information. Next, MNFP determination unit 131 sequentially acquires one or more ignition node identifiers through a propagation process performed by propagation means 1312, which will be described later. That is, MNFP determination unit 131 acquires one or more ignition node identifiers in chronological order. This process is realized by applying the input information to the NN.

[0078] Note that one or more MNFPs corresponding to one or more firing node identifiers in the time series are MNFPs that satisfy the correspondence condition.

[0079] An MNFP that satisfies the correspondence condition is, for example, an MNFP that matches one or more firing node identifiers in the time series. However, an MNFP that satisfies the correspondence condition may also include an MNFP whose similarity to one or more firing node identifiers in the time series is equal to or greater than a threshold. The similarity between a matrix that is one or more firing node identifiers in the time series and an MNFP that is a matrix is ​​obtained, for example, using the proportion of matching elements among the elements of the two matrices. The similarity between a matrix that is one or more firing node identifiers in the time series and an MNFP that is a matrix is ​​obtained, for example, by calculating the similarity of the vectors for each row of the matrix, and taking the representative value (for example, the average or median) of the two or more similarities as the similarity between the two matrices. The method for calculating the similarity between the two matrices is not important.

[0080] When the MNFP is a matrix with two axes, a time axis and a node identifier, MNFP determination unit 131 configures a matrix with, for example, two or more time points as rows or columns, and with columns or rows in which values ​​corresponding to one or more firing node identifiers are "1" and values ​​corresponding to non-firing node identifiers are "0." Next, MNFP determination unit 131 sequentially determines the MNFP corresponding to the matrix from one or more MNFPs in MNFP storage unit 113.

[0081] The initial determination means 1311 acquires one or more pieces of characteristic information from the input information accepted by the input accepting unit 121. Next, the initial determination means 1311 determines one or more firing conditions that match the one or more pieces of characteristic information, and acquires one or more firing node identifiers corresponding to each of the one or more firing conditions.

[0082] When the input information is an image, the acquired feature information is, for example, red "R" that constitutes a color, green "G" that constitutes a color, blue "B" that constitutes a color, pixel values, and an object identifier that is the result of object recognition for the image. When the input information is sound, the acquired feature information is, for example, frequency and amplitude. When the input information is a character string, the acquired feature information is, for example, the class of the character string.

[0083] The initial determination means 1311, for example, acquires one or more pieces of characteristic information from the input information accepted by the input accepting unit 121. Next, the initial determination means 1311 refers to the ignition start point storage unit 112 and acquires from the ignition start point storage unit 112 one or more ignition node identifiers for which the acquired one or more pieces of characteristic information satisfy the ignition condition.

[0084] In addition, when the firing condition corresponds to the firing probability, the initial determination means 1311, for example, acquires one or more pieces of feature information from the input information accepted by the input accepting unit 121, and determines whether or not a node identified by a node identifier corresponding to a firing condition that matches the one or more pieces of feature information will fire according to the probability indicated by the firing probability, and acquires a firing node identifier that identifies the node only if it is determined that the node will fire.

[0085] The propagation means 1312 acquires one or more connecting node identifiers corresponding to the one or more firing node identifiers acquired by the initial determination means 1311. Next, the propagation means 1312 acquires one or more connecting node identifiers corresponding to the one or more connecting node identifiers. Next, the propagation means 1312 acquires one or more connecting node identifiers connected to the one or more connecting node identifiers, and so on, propagating the firing.

[0086] When the firing probability is used, the propagation means 1312 acquires one or more connecting node identifiers corresponding to one or more firing node identifiers acquired by the initial determination means 1311. The propagation means 1312 also acquires information identifiers included in one or more pieces of feature information corresponding to one or more firing node identifiers acquired by the initial determination means 1311. The propagation means 1312 may acquire the amount of information corresponding to each information identifier. Preferably, the propagation means 1312 subtracts the amount of information included in the feature information corresponding to the firing node identifier acquired by the initial determination means 1311 using a predetermined arithmetic expression (e.g., "x 0.9") and acquires the reduced amount of information. Next, the propagation means 1312 determines whether the acquired information identifier, the acquired amount of information, or the acquired information identifier and the information amount match the firing condition paired with one or more connecting node identifiers. The propagation means 1312 then acquires only the connecting node identifier paired with the firing condition that is determined to match as the firing node identifier.

[0087] If the NN has a propagation probability associated with each of one or more node identifiers, the propagation means 1312 acquires, in accordance with the probability indicated by the propagation probability, a connecting node identifier corresponding to each of one or more firing node identifiers acquired by the initial determination means 1311. Also, if the NN has a propagation probability associated with each of one or more node identifiers, the propagation means 1312 acquires, in accordance with the probability indicated by the propagation probability, a connecting node identifier corresponding to each of the acquired one or more connecting node identifiers.

[0088] Furthermore, in a NN, when there is a propagation probability associated with each of one or more node identifiers, it is preferable that the propagation means 1312 acquires or does not acquire a connection node identifier that pairs with a firing condition that is determined to match as a firing node identifier according to the probability indicated by the propagation probability.

[0089] MNFP determination means 1313 acquires, in time series, firing information having one or more firing node identifiers from among the one or more firing node identifiers and one or more connection node identifiers acquired by initial determination means 1311 and propagation means 1312. Acquiring in time series means acquiring at two or more points in time. Next, MNFP determination means 1313 sequentially determines one or more MNFPs corresponding to the time-series firing information from one or more MNFPs in MNFP storage unit 113. Note that the MNFP corresponding to the time-series firing information is an MNFP that satisfies the correspondence condition.

[0090] Operation determination unit 132 sequentially acquires operation determination information corresponding to each of one or more MNFPs determined by MNFP determination unit 131 from MNFP storage unit 113. In other words, operation determination unit 132 acquires operation determination information corresponding to each of one or more MNFPs determined by MNFP determination unit 131 for each of two or more time points.

[0091] The muscle determination means 1321 refers to the muscle storage unit 115 and sequentially determines one or more pieces of muscle information associated with one or more ignition node identifiers identified by each of the one or more MNFPs determined by the MNFP determination unit 131. That is, the muscle determination means 1321 determines, for each of two or more time points, one or more pieces of muscle information associated with one or more ignition node identifiers identified by each of the one or more MNFPs determined by the MNFP determination unit 131. Note that the one or more ignition node identifiers identified by the MNFP are preferably usually one or more ignition node identifiers at the most recent time point possessed by the MNFP. However, the one or more ignition node identifiers identified by the MNFP may also be all ignition node identifiers possessed by the MNFP at two or more time points.

[0092] The strength acquisition means 1322 acquires the strength of each of the one or more pieces of muscle information determined by the muscle determination means 1321 .

[0093] For example, the intensity acquisition means 1322 acquires the number of firings, which is the number of firing node identifiers associated with each muscle information, for each of the one or more muscle information determined by the muscle determination means 1321. Next, for example, the intensity acquisition means 1322 acquires the maximum power of each muscle information from the muscle storage unit 115 for each of the one or more muscle information determined by the muscle determination means 1321. Next, the intensity acquisition means 1322 sequentially acquires intensities, for example, using the number of firings and the maximum power. Note that the intensity acquisition means 1322 typically acquires a higher intensity as the number of firings increases. Also, the intensity acquisition means 1322 typically acquires a higher intensity as the maximum power increases. For example, the intensity acquisition means 1322 acquires intensities using an increasing function with the number of firings and the maximum power as parameters. Note that an arithmetic expression representing such an increasing function is stored in the storage unit 11. For example, the intensity acquisition means 1322 acquires intensities corresponding to the number of firings and the maximum power from a correspondence table. The correspondence table is a table having two or more pieces of correspondence information, each of which has the number of firings, the maximum power, and the intensity. The intensity acquisition unit 1322, for example, acquires from the correspondence table the intensity paired with the number of firings and the maximum power that is most similar to the number of firings and the maximum power.

[0094] The intensity acquisition means 1322, for example, acquires the maximum power of each piece of muscle information from the muscle storage unit 115 for each piece of one or more pieces of muscle information determined by the muscle determination means 1321. Next, the intensity acquisition means 1322 sequentially acquires the intensity, for example, using the maximum power. The intensity acquisition means 1322 normally acquires a higher intensity as the maximum power increases. For example, the intensity acquisition means 1322 acquires the percentage (%) of force used by the muscle based on the number of firings, and multiplies the percentage by the maximum power to calculate the intensity. Furthermore, the intensity acquisition means 1322 may, for example, use a hill-type model to determine an attenuation rate that specifies a change in intensity over time. It is preferable to calculate the attenuation rate by using the calculated attenuation rate, and to acquire the time series intensity so as to approach the maximum power or to acquire the time series intensity so as to go from the maximum power to 0.

[0095] For example, the intensity acquiring means 1322 acquires the intensity corresponding to each of the one or more pieces of muscle information each time the muscle determining means 1321 determines the one or more pieces of muscle information. As a result, the intensity acquiring means 1322 successively acquires the intensity corresponding to each of the one or more pieces of muscle information.

[0096] Through the above process, the strength acquisition means 1322 sequentially acquires one or more pieces of muscle information for movement. Note that the muscle information for movement is a set of a muscle identifier and a strength.

[0097] The angle acquisition means 1323 refers to the joint storage unit 116 and uses one or more pieces of moving muscle information acquired by the strength acquisition means 1322 to sequentially acquire the angles of the joints identified by one or more joint identifiers corresponding to the muscle identifiers contained in the one or more pieces of muscle information determined by the muscle determination means 1321.

[0098] The angle acquisition unit 1323 calculates the angle of each of one or more joints using the strength corresponding to one or more muscle identifiers paired with the joint identifier of the joint. The angle acquisition unit 1323 determines the bending direction and angle of the joint using, for example, a physical calculation that calculates torque.

[0099] The angle acquisition unit 1323 sequentially acquires a joint angle set, which is a set of pairs of joint identifiers and angles, for example. Note that the joint angle set may also be a set of pairs of angles and joint positions.

[0100] The action unit 133 sequentially uses one or more pieces of action determination information acquired by the action determination unit 132 to cause the action target 2 to act.

[0101] The movement unit 133 sequentially moves the movement target 2 based on one or more pieces of movement muscle information, which is, for example, a pair of a muscle identifier included in each piece of muscle information determined by the muscle determination means 1321 and an intensity acquired by the intensity acquisition means 1322.

[0102] The movement unit 133, for example, outputs the angle for each of the one or more joints determined by the angle acquisition means 1323 to the movement target 2, and moves the movement target 2. In other words, the movement unit 133, for example, sequentially uses the sets of joint angles acquired by the angle acquisition means 1323 to move the movement target 2.

[0103] When the motion object 2 is an avatar, the motion unit 133 uses the avatar information to construct an avatar in which, for example, the joints specified by the joint positions of one or more joints are bent according to the angles determined by the angle acquisition means 1323, and outputs the avatar.

[0104] If the object 2 to be moved is a robot, the operation unit 133 sends an instruction to the robot to bend the joints identified by the joint positions of one or more joints according to the angles determined by the angle acquisition means 1323, thereby causing the robot to move.

[0105] When the muscle identifier determined by the muscle determination means 1321 satisfies a predetermined increase condition, the muscle change unit 134 performs a change process to increase the maximum power paired with the muscle identifier.

[0106] The increase conditions are, for example, that the muscle determination means 1321 has determined a muscle identifier, that the number of times the muscle determination means 1321 determines a muscle identifier is equal to or greater than a threshold, or that the number of times per unit time the muscle determination means 1321 determines a muscle identifier is equal to or greater than a threshold. In the change process for increasing the maximum power, the degree to which the maximum power is increased does not matter. For example, when the increase conditions are met, the muscle change unit 134 increases the maximum power paired with the muscle identifier by a threshold number (e.g., "1"). For example, when the increase conditions are met, the muscle change unit 134 increases the maximum power paired with the muscle identifier by a percentage of the threshold (e.g., "10%).

[0107] When a muscle identifier not determined by the muscle determination means 1321 satisfies a predetermined reduction condition, the muscle change unit 134 performs a change process to reduce the maximum power paired with the muscle identifier.

[0108] The reduction condition is, for example, a muscle identifier that has not been determined by the muscle determination means 1321 for a period of time equal to or longer than the threshold time. In the change process for reducing the maximum power, the degree to which the maximum power is reduced is not important. For example, when the reduction condition is satisfied, the muscle change unit 134 reduces the maximum power paired with the muscle identifier by a threshold number (for example, "1"). For example, when the reduction condition is satisfied, the muscle change unit 134 reduces the maximum power paired with the muscle identifier by a percentage of the threshold (for example, "15%").

[0109] Learning unit 135 acquires the MNFP when input receiving unit 121 receives input information. Learning unit 135 also receives action determination information. Learning unit 135 also associates the MNFP with the received action determination information and stores it in MNFP storage unit 113. This process is referred to as a learning process.

[0110] The storage unit 11, NN storage unit 111, firing start point storage unit 112, MNFP storage unit 113, avatar storage unit 114, muscle storage unit 115, and joint storage unit 116 are preferably non-volatile recording media, but can also be realized with volatile recording media.

[0111] There is no restriction on the process by which information is stored in the storage unit 11 etc. For example, information may be stored in the storage unit 11 etc. via a recording medium, information transmitted via a communication line etc. may be stored in the storage unit 11 etc., or information input via an input device may be stored in the storage unit 11 etc.

[0112] The reception unit 12 and the input reception unit 121 can be realized by a device driver for an input means such as a touch panel or a keyboard, control software for a menu screen, etc. However, the reception unit 12 etc. may also be realized by a wireless or wired communication means.

[0113] The processing unit 13, MNFP determination unit 131, movement determination unit 132, movement unit 133, muscle change unit 134, learning unit 135, initial determination means 1311, propagation means 1312, MNFP determination means 1313, muscle determination means 1321, strength acquisition means 1322, and angle acquisition means 1323 can typically be realized by a processor, memory, or the like. The processing procedures of the processing unit 13, etc., are typically realized by software, and the software is recorded on a recording medium such as a ROM. However, they may also be realized by hardware (dedicated circuitry). The processor may be a CPU, MPU, GPU, or the like, and the type is not important.

[0114] Next, an example of the operation of the information processing device 1 will be described with reference to the flowchart of FIG.

[0115] (Step S201) Input receiving unit 121 determines whether or not input information has been received. If input information has been received, the process proceeds to step S202, and if input information has not been received, the process returns to step S201.

[0116] (Step S202) The initial determination means 1311 acquires one or more pieces of feature information from the input information accepted in step S201. Note that the technology for acquiring one or more pieces of feature information from input information such as an image, sound, or character string is a publicly known technology.

[0117] (Step S203) The initial determination means 1311 performs an initial determination process. An example of the initial determination process will be described with reference to the flowchart in Fig. 3. The initial determination process is a process of determining a node to be fired first using one or more pieces of feature information acquired in step S202.

[0118] (Step S204) The MNFP determination means 1313 performs an MNFP determination process. An example of the MNFP determination process will be described using the flowchart in Fig. 4. The MNFP determination process is a process for determining an MNFP corresponding to one or more ignition node identifiers in a time series.

[0119] (Step S205) Operation determination unit 132 determines whether or not one or more MNFPs have been determined in step S204. If an MNFP has been determined, the process proceeds to step S206, and if an MNFP has not been determined, the process proceeds to step S207.

[0120] (Step S206) The operation determination unit 132 and the like perform an operation according to the determined MNFP. An example of such operation processing will be described with reference to the flowchart in FIG.

[0121] (Step S207) Propagation means 1312 determines whether or not the propagation of the ignition has finished. If the propagation of the ignition has finished, the process returns to step S201, and if the propagation of the ignition has not finished, the process proceeds to step S208.

[0122] The propagation means 1312 determines that the propagation of the firing has ended, for example, when it determines that there are no connecting nodes for all of the one or more nodes that fired immediately before. Also, the propagation means 1312 determines that the propagation of the firing has ended, for example, when a predetermined time has elapsed after the input information was accepted.

[0123] (Step S208) The propagation means 1312 performs the ignition propagation process. Return to step S204. An example of the ignition propagation process will be described with reference to the flowchart in Fig. 7. The ignition propagation process is a process for determining the node that will ignite at the next time point.

[0124] (Step S209) The processing unit 13 performs a firing elimination process. An example of the firing elimination process will be described with reference to the flowchart in Fig. 8. The firing elimination process is a process of changing a node in an firing state into a non-firing state.

[0125] In the flowchart of FIG. 2, the process ends when the power is turned off or an interrupt occurs to end the process.

[0126] Next, an example of the initial determination process in step S203 will be described with reference to the flowchart in FIG.

[0127] (Step S301) The initial determination means 1311 assigns 1 to a counter i.

[0128] (Step S302) The initial determination means 1311 determines whether or not the i-th node exists among the candidate nodes that will fire at the initial point in time. If the i-th node exists, the process proceeds to step S303, and if not, the process returns to the upper level process.

[0129] The initial determination means 1311 determines, for example, whether or not an i-th node identifier exists among the node identifiers held by one or more pieces of firing start point information stored in the firing start point storage unit 112. The initial determination means 1311 also determines, for example, whether or not an i-th node identifier exists in the NN in the NN storage unit 111. The initial determination means 1311 also determines, for example, whether or not an i-th node identifier exists, which is a node identifier in the NN in the NN storage unit 111 and pairs with firing information using feature information.

[0130] (Step S303) The initial determination means 1311 acquires the firing condition paired with the node identifier of the i-th node from the firing start point storage unit 112 or the NN storage unit 111.

[0131] (Step S304) The initial determination means 1311 determines whether or not the one or more pieces of acquired feature information satisfy the firing condition acquired in step S303. If the firing condition is satisfied, the process proceeds to step S305, and if the firing condition is not satisfied, the process proceeds to step S306.

[0132] (Step S305) The initial determination means 1311 performs a process of associating the i-th node with the current time point and setting the i-th node as the firing node. The process of setting the i-th node as the firing node is, for example, a process of writing "1" into an element of a vector associated with the current time point and corresponding to the node identifier of the i-th node. The vector is stored in the firing information storage unit 117, for example.

[0133] (Step S306) The initial determination means 1311 performs a process of associating the i-th node with the current time and not treating the i-th node as an ignition node. The process of not treating the i-th node as an ignition node is, for example, a process of writing "0" into an element of a vector associated with the current time, which corresponds to the node identifier of the i-th node. The vector is stored, for example, in the ignition information storage unit 117.

[0134] (Step S307) The initial determination means 1311 increments the counter i by 1. The process returns to step S302.

[0135] By performing the process in the flowchart of FIG. 3, for example, a vector that identifies an ignition node at an initial point in time can be obtained.

[0136] Next, an example of the MNFP determination process in step S204 will be described with reference to the flowchart in FIG.

[0137] (Step S401) The MNFP determining means 1313 assigns 1 to a counter i.

[0138] (Step S402) MNFP determination means 1313 determines whether or not the i-th MNFP exists in MNFP storage unit 113. If the i-th MNFP exists, the process proceeds to step S403, and if not, the process returns to the upper level processing.

[0139] (Step S 403 ) MNFP determination means 1313 acquires the i-th MNFP from MNFP storage unit 113 .

[0140] (Step S404) The MNFP determination means 1313 acquires the number of time points of the i-th MNFP acquired in step S403. The number of time points is, for example, the number of rows or columns constituting the matrix of the i-th MNFP.

[0141] (Step S405) The MNFP determination means 1313 obtains a matrix having ignition vectors for the number of time points obtained in step S404, going back from the most recent time point. This matrix is ​​the target to be compared with the MNFP, and is therefore called a target matrix. The target matrix is ​​information that identifies ignition nodes that have ignited based on input information, and is time-series information. The MNFP determination means 1313 obtains the target matrix from, for example, the ignition information storage unit 117.

[0142] (Step S406) The MNFP determination means 1313 determines whether or not a target matrix having ignition vectors for the number of time points was acquired in step S405. If a target matrix was acquired, the process proceeds to step S407; if not, the process proceeds to step S409.

[0143] (Step S407) The MNFP determination means 1313 determines whether the target matrix and the i-th MNFP satisfy the correspondence condition. If the correspondence condition is satisfied, the process proceeds to step S408, and if the correspondence condition is not satisfied, the process proceeds to step S409. The correspondence condition may be, for example, that the target matrix and the i-th MNFP match, or that the similarity between the target matrix and the i-th MNFP is equal to or greater than a threshold.

[0144] (Step S408) The MNFP determination means 1313 temporarily stores the MNFP identifier that identifies the i-th MNFP in a buffer (not shown).

[0145] (Step S409) The MNFP determining means 1313 increments the counter i by 1. The process returns to step S402.

[0146] Next, an example of the operation process in step S206 will be described with reference to the flowchart in FIG.

[0147] (Step S501) The operation determination unit 132 assigns 1 to a counter i.

[0148] (Step S502) Operation determination unit 132 determines whether or not the i-th MNFP identifier stored in the buffer in step S408 exists. If the i-th MNFP identifier exists, the process proceeds to step S503; if not, the process proceeds to step S511.

[0149] (Step S503) The operation determination unit 132 assigns 1 to a counter j.

[0150] (Step S504) Operation determination unit 132 determines whether or not the jth ignition node identifier owned by the MNFP identified by the i-th MNFP identifier exists. If the jth ignition node identifier exists, the process proceeds to step S505; if not, the process proceeds to step S510. Note that the jth ignition node identifier here is preferably the ignition node identifier at the latest point in time (current point in time) among the ignition node identifiers owned by the MNFP.

[0151] (Step S505) The operation determination unit 132 assigns 1 to a counter k.

[0152] (Step S506) The movement determination unit 132 determines whether the k-th muscle identifier exists among the muscle identifiers corresponding to the j-th firing node identifier. If the k-th muscle identifier exists, the process proceeds to step S507; if not, the process proceeds to step S509.

[0153] (Step S507) The movement determination unit 132 increases the firing number (firing number of the k-th muscle), which is the number of firing nodes corresponding to the k-th muscle identifier. Note that the amount of increase is, for example, 1, but is not limited to this.

[0154] (Step S508) The operation determination unit 132 increments the counter k by 1. The process returns to step S506.

[0155] (Step S509) The operation determination unit 132 increments the counter j by 1. The process returns to step S504.

[0156] (Step S510) The operation determination unit 132 increments the counter i by 1. The process returns to step S502.

[0157] (Step S511) The operation determination unit 132 assigns 1 to the counter l.

[0158] (Step S512) The movement determination unit 132 determines whether or not the lth joint exists. If the lth joint exists, the process proceeds to step S513, and if not, the process proceeds to step S515.

[0159] (Step S513) The angle acquisition means 1323 performs angle acquisition processing for the l-th joint. An example of the angle acquisition processing will be described with reference to the flowchart in FIG.

[0160] (Step S514) The operation determination unit 132 increments the counter 1 by 1. The process returns to step S512.

[0161] (Step S515) The operation unit 133 creates a joint angle set using the joint angles determined in step S513.

[0162] (Step S516) The operation unit 133 uses the set of joint angles acquired in step S515 to operate the operation target 2. The process returns to the upper level process.

[0163] The operation unit 133, for example, uses the avatar information in the avatar storage unit 114 to create an avatar so that the angles of each joint are as indicated by the joint angle set, and outputs the avatar. The operation unit 133, for example, moves the robot so that the angles of each joint are as indicated by the joint angle set.

[0164] Next, an example of the angle acquisition process in step S513 will be described with reference to the flowchart in FIG.

[0165] (Step S601) The operation determination unit 1322 assigns 1 to a counter i.

[0166] (Step S602) The muscle determination means 1321 refers to the joint storage unit 116 and determines whether or not there is an i-th muscle identifier paired with the joint identifier of the l-th joint in step S512. If there is an i-th muscle identifier, the process proceeds to step S603; if there is no i-th muscle identifier, the process proceeds to step S607.

[0167] (Step S603) The intensity acquisition unit 1322 acquires the number of firings paired with the i-th muscle identifier.

[0168] (Step S604) The intensity acquisition means 1322 determines whether the number of firings acquired in step S603 is equal to or greater than 1. If the number of firings is equal to or greater than 1, the process proceeds to step S605, and if the number of firings is 0 or the number of firings cannot be acquired, the process proceeds to step S607.

[0169] (Step S605) The strength acquisition means 1322 acquires the maximum power paired with the i-th muscle identifier from the muscle storage unit 115.

[0170] (Step S606) The strength acquisition means 1322 acquires the strength of the muscle identified by the i-th muscle identifier, using the number of firings acquired in step S603 and the maximum power acquired in step S605.

[0171] (Step S607) The operation determination unit 132 increments the counter i by 1. The process returns to step S602.

[0172] (Step S608) The angle acquisition means 1323 uses the strength of one or more muscles acquired in step S606 to determine whether the angle of the l-th joint can be acquired in step S512. If the angle can be acquired, the process proceeds to step S609, and if the angle cannot be acquired, the process returns to the upper process.

[0173] If the angle cannot be acquired, it means that the intensity could not be acquired in step S606.

[0174] (Step S609) The angle acquisition means 1323 acquires the angle of the l-th joint in step S512, and returns to the upper level processing.

[0175] In the flowchart of FIG. 6, the angle acquisition means 1323 normally associates the joint identifier with the angle and stores them in a buffer (not shown).

[0176] Next, an example of the firing propagation process in step S208 will be described with reference to the flowchart in FIG.

[0177] (Step S701) The propagation means 1312 assigns 1 to the counter i.

[0178] (Step S702) The propagation means 1312 determines whether or not the i-th ignition node that fired immediately before exists. If the i-th ignition node exists, the process proceeds to step S703; if not, the process returns to the upper process.

[0179] (Step S703) The propagation means 1312 assigns 1 to the counter j.

[0180] (Step S704) The propagation means 1312 determines whether or not the jth connection node corresponding to the ith firing node exists. If the jth connection node exists, the process proceeds to step S705; if not, the process proceeds to step S712.

[0181] The propagation means 1312 normally determines whether or not there exists a jth connecting node identifier that pairs with the node identifier that identifies the i-th firing node, by referring to the NN storage unit 111. If there exists a jth connecting node identifier, the process proceeds to step S705; if there does not exist a jth connecting node identifier, the process proceeds to step S712.

[0182] (Step S705) The propagation means 1312 acquires from the NN storage unit 111 the firing condition paired with the j-th connection node identifier.

[0183] (Step S706) The propagation means 1312 acquires feature information corresponding to the i-th firing node. The propagation means 1312 may acquire feature information containing an amount of information obtained by subtracting the amount of information contained in the feature information corresponding to the i-th firing node. Next, the propagation means 1312 determines whether the acquired feature information satisfies the firing condition paired with the j-th connecting node identifier. If the firing condition is satisfied, the process proceeds to step S707; if the firing condition is not satisfied, the process proceeds to step S710. The acquired feature information may be, for example, an amount of information, an information identifier, or an amount of information and an information identifier.

[0184] (Step S707) The propagation means 1312 determines whether or not a propagation probability paired with the j-th connection node identifier exists in the NN storage unit 111. If a propagation probability exists, the process proceeds to step S708, and if not, the process proceeds to step S709.

[0185] (Step S708) The propagation means 1312 determines whether the node identified by the j-th connection node identifier will fire according to the propagation probability. If it will fire, the process proceeds to step S709, and if it will not fire, the process proceeds to step S710.

[0186] (Step S709) The propagation means 1312 sets the node identified by the j-th connection node identifier as the firing node.

[0187] (Step S710) The propagation means 1312 does not set the node identified by the j-th connection node identifier as the firing node.

[0188] (Step S711) The propagation means 1312 increments the counter j by 1. The process returns to step S704.

[0189] (Step S712) The propagation means 1312 increments the counter i by 1. The process returns to step S702.

[0190] 7, the propagation unit 1312 determines whether the acquired feature information satisfies the firing condition. However, if the firing condition is satisfied in step S706, the propagation unit 1312 may determine whether to proceed to step S707 or step S710 based on the firing probability.

[0191] Next, an example of the firing elimination process in step S209 will be described with reference to the flowchart in FIG. 8.

[0192] (Step S801) The processing unit 13 assigns 1 to a counter i.

[0193] (Step S802) The processing unit 13 determines whether or not there is firing information at the i-th time point, which is firing information corresponding to at least one firing node identifier and is stored in the firing information storage unit 117. If there is firing information at the i-th time point, the process proceeds to step S803; if there is no firing information, the process returns to the upper processing.

[0194] (Step S803) The processing unit 13 assigns 1 to the counter j.

[0195] (Step S804) The processing unit 13 determines whether or not the jth ignition node exists among the ignition nodes identified by the ignition information at the i-th time point. If the jth ignition node exists, the process proceeds to step S805; if not, the process proceeds to step S808.

[0196] (Step S805) The processing unit 13 determines whether the j-th firing node at the i-th time point satisfies the elimination condition. If the j-th firing node satisfies the elimination condition, the process proceeds to step S806; if not, the process proceeds to step S807.

[0197] (Step S806) The processing unit 13 sets the i-th firing node at the i-th time point as a non-firing node.

[0198] (Step S807) The processing unit 13 increments the counter j by 1. The process returns to step S804.

[0199] (Step S808) The processing unit 13 increments the counter i by 1. The process returns to step S802.

[0200] A specific example of the operation of the operating system A in this embodiment will be described below with reference to the schematic diagram of the operating system A in FIG.

[0201] The input receiving unit 121 of the information processing device 1 receives input information 901. The input information 901 is, for example, audio or images.

[0202] Furthermore, the initial determination means 1311 of the information processing device 1, for example, senses input information 901 and acquires one or more pieces of feature information (902). Then, the MNFP determination unit 131 applies the NN (903) in the NN storage unit 111 to the one or more pieces of feature information, and sequentially determines a set of one or more firing nodes. Then, the MNFP determination unit 131 constructs time-series firing information. Next, the MNFP determination unit 131 sequentially determines one or more MNFPs in the MNFP storage unit 113 using the time-series firing information. Then, the operation unit 133 sequentially acquires action determination information corresponding to each of the one or more sequentially determined MNFPs. Next, the operation unit 133 sequentially uses the one or more pieces of action determination information acquired to cause the action target 2 to operate. Note that in FIG. 9, six NNs (903) are connected.

[0203] More specifically, for example, the storage unit 11 of the information processing device 1 stores the following information:

[0204] The NN storage unit 111 stores the NN management table shown in FIG. 10. The NN management table is a table for managing NNs. The NN management table is a table for managing information on one or more nodes that make up an NN. Here, the node information includes a "node identifier," a "firing condition," a "connecting node identifier," a "propagation probability," and an "elimination condition." If the "elimination condition" is "-," the default elimination condition is adopted as the elimination condition for the corresponding node. The default elimination condition may be, for example, that three seconds or more have passed since ignition.

[0205] It is assumed that the firing start point storage unit 112 stores a set of firing start point information "N001, N002, N003, N004, N005, . . . ".

[0206] MNFP storage unit 113 stores the MNFP management table shown in FIG. 11. The MNFP management table manages two or more records each having an MNFP identifier and an MNFP. Here, the MNFP is a matrix with two axes: node identifier and time point. At the times "t1," "t2," "t3," "t4," and "t5," a smaller number indicates an earlier time point (older time point). Also, in the MNFP with MNFP identifier "M01" in FIG. 11, at time point "t1," the two nodes identified by "N001" and "N002" are firing nodes, and the nodes identified by "N015" and "N389" are non-firing nodes.

[0207] It is assumed that avatar storage unit 114 stores avatar information that configures the avatar of motion target 2.

[0208] The muscle storage unit 115 stores the muscle management table shown in FIG. 12. The muscle management table manages one or more pieces of muscle information. Here, the muscle information has a "muscle identifier," "maximum power," and "firing node identifier." In other words, the muscle information is information about the muscle that operates when the node fires. This indicates that the muscle identified by the muscle identifier "MU001" operates when the node identified by the node identifier "N001" or "N002" is in the firing state. Furthermore, the maximum power of the muscle identified by the muscle identifier "MU001" is "8."

[0209] The joint storage unit 116 stores a joint management table shown in FIG. 13. The joint management table manages one or more pieces of joint information. The joint information here includes a "joint identifier," a "joint position," and a "muscle identifier." The "joint position" here is a coordinate value indicating the center of gravity of the joint. In FIG. 13, the joint identified by the joint identifier "J001" indicates a joint that operates according to the strength of the muscles identified by the muscle identifiers "MU001" and "MU002."

[0210] In this situation, it is assumed that the information processing device 1 receives input information (for example, an image).

[0211] Next, the initial determination means 1311 acquires two or more pieces of characteristic information (for example, "R", "G", and "B") from the received input information.

[0212] Next, the initial determination means 1311 performs the initial determination process as follows. That is, the initial determination means 1311 refers to the set of firing start point information "N001, N002, N003, N004, N005,..." in the firing start point storage unit 112 and the NN management table (FIG. 10), and acquires node identifiers "N001, N002, N003,..." of nodes that fire for two or more pieces of feature information (for example, "R", "G", and "B"). Then, it is assumed that the vector (1, 1, 1, 0, 0,..., 0,..., 0,...) constituting the firing node identifiers is stored in the firing information storage unit 117 in association with the time point "t1". This information is the information in the column corresponding to "t1" in FIG. 14.

[0213] Next, the MNFP determination means 1313 performs the MNFP determination process as follows: That is, the MNFP determination means 1313 refers to Fig. 11 and determines that for each MNFP in the MNFP management table, the ignition information vector (1,1,1,0,0,...,0,...,0,...) corresponding to "t1" in Fig. 14 does not match (here, a perfect match).

[0214] Next, the propagation means 1312 performs a propagation process, for example, as follows. That is, the propagation means 1312 acquires connecting node identifiers (e.g., "N011" and "N012" paired with "N001"), "N013", "N014", and "N015" paired with "N002", "N016" and "N017" paired with "N003") paired with the identifiers of each ignition node ("N001", "N002", "N003") that fired at time "t1". Next, the propagation means 1312 acquires a propagation probability paired with each connecting node identifier, and if the propagation probability is less than 1, performs a lottery according to the propagation probability to determine whether the connecting node identifier will be the ignition node identifier. Furthermore, the propagation means 1312 determines that the connecting node identifier paired with a propagation probability of "1" will be the ignition node identifier. Then, the propagation means 1312 acquires a vector (1, 1, . . . , 0, . . . , 1, . . . ), which is a set of ignition node identifiers, in association with the time point "t2." The propagation means 1312 also associates the vector with the time point "t2" and adds it to the ignition information storage unit 117 (see the column corresponding to "t2" in FIG. 14).

[0215] If there is a firing condition paired with the connecting node identifier, the propagation means 1312 determines whether the connecting node identifier is a firing node identifier only when the firing condition is satisfied, as described above.

[0216] Next, it is assumed that the MNFP determination means 1313 determines that the MNFP matching the two columns of the matrix for time points "t1" and "t2" does not exist in the MNFP management table (FIG. 11).

[0217] Next, it is assumed that the propagation means 1312 further advances the propagation process, and that the set of ignition information shown in FIG.

[0218] Next, the MNFP determination means 1313 determines that the set (matrix) of ignition information shown in Fig. 14 corresponds to (here, is an exact match with) the MNFP "M001" in the MNFP management table (Fig. 11). In other words, the MNFP determination means 1313 determines the MNFP identifier "M001."

[0219] Next, the muscle determination means 1321 acquires the firing node identifiers ("N001", "N002", "N015", "N389") at the most recent time point "t5" of the MNFP identified by the MNFP identifier "M001".

[0220] Next, the muscle determination means 1321 refers to the muscle management table (FIG. 12) and acquires muscle identifiers ("MU001", "MU002", . . . ) that are paired with each of the acquired firing node identifiers.

[0221] Next, the intensity acquisition means 1322 refers to the column "t5" in Fig. 14 and acquires, for each acquired muscle identifier, the firing count, which is the number of fired nodes among the firing node identifiers paired with the muscle identifier. For example, the intensity acquisition means 1322 acquires a firing count of "2" for muscle identifier "MU001" and a firing count of "1" for muscle identifier "MU002."

[0222] Furthermore, the intensity obtaining means 1322 obtains the maximum power paired with each obtained muscle identifier from the muscle management table (FIG. 12). Next, the intensity obtaining means 1322 obtains the intensity for each obtained muscle identifier using the number of firings and the maximum power.

[0223] Next, the angle acquisition means 1323 acquires one or more joint identifiers that are paired with the muscle identifier of the muscle whose strength has been acquired from the joint management table (FIG. 13). Then, for each acquired joint identifier, the angle acquisition means 1323 acquires the angle of the joint using the strength of the muscle identified by each of the two or more muscle identifiers that are paired with each joint identifier. In addition, the angle acquisition means 1323 acquires, for each joint identifier, the joint position that is paired with each joint identifier from the joint management table (FIG. 13). Through the above processing, the angle acquisition means 1323 can acquire (angle, joint position) for each moving joint. Note that a set of (angle, joint position) for one or more moving joints is a joint angle set.

[0224] Next, the operation unit 133 acquires the avatar information from the avatar storage unit 114. Next, the operation unit 133 transforms the avatar information so that the angles of the joints correspond to the acquired set of joint angles, and moves the avatar, which is the object of movement 2.

[0225] Thereafter, the operation of the information processing device 1 continues according to the processing explained in the flowchart of FIG. 2 and the like.

[0226] As described above, according to this embodiment, it is possible to appropriately simulate the movement of a living thing such as a human being in response to its cognition using an MNFP.

[0227] In particular, according to this embodiment, muscle movements corresponding to the perception of a living being such as a human can be appropriately simulated.

[0228] Furthermore, according to this embodiment, it is possible to appropriately simulate the movement of the joints in accordance with the perception of a living being such as a human being.

[0229] Furthermore, according to this embodiment, joint movements according to the perception of a living being such as a human can be simulated using an avatar.

[0230] In the specific example of this embodiment, the moving object 2 is an avatar, but it may be a robot or other type of moving object 2. In a robot, once the joints and their angles are determined, the movement of the robot is determined.

[0231] Furthermore, the processing in this embodiment may be realized by software. This software may be distributed by software download or the like. This software may also be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software that realizes the information processing device 1 in this embodiment is the following program. In other words, this program is a program for causing a computer that can access an NN storage unit in which a neural network having two or more nodes identified by node identifiers is stored, and an MNFP storage unit in which one or more control information is stored, which is information including one or more motor neuron firing patterns (MNFPs), which is information that identifies one or more firing nodes at each of two or more time points, and which corresponds to action determination information that determines the action of an object to be operated on, to function as an action unit that includes: an input accepting unit that accepts input information; an MNFP determination unit that applies the input information to the neural network, acquires one or more firing node identifiers of the two or more nodes that the neural network has in a time series, and sequentially determines one or more MNFPs corresponding to the one or more firing node identifiers in the time series from the one or more MNFPs in the MNFP storage unit; an action determination unit that sequentially acquires action determination information corresponding to each of the one or more MNFPs determined by the MNFP determination unit; and a program for causing the object to operate by sequentially using the one or more action determination information acquired by the action determination unit.

[0232] (Embodiment 2) This embodiment differs from the first embodiment in that the moving object is an object without joints. The moving object is, for example, a face. In this embodiment, the movement of muscles corresponding to the moving object is simulated.

[0233] In this embodiment, when the muscles to be operated are facial muscles, an information processing device will be described that acquires the strength of expansion and contraction for each mesh of the face and changes the face using the strength for each mesh.

[0234] 15 is a block diagram of an operating system B according to the present embodiment. The operating system B includes an information processing device 3 and an operating target 4. The information processing device 3 may also include the operating target 4.

[0235] The moving object 4 is an object that performs a movement. The moving object 4 may be, for example, an avatar or a robot, but the type of the moving object 4 is not important.

[0236] The information processing device 3 includes a storage unit 31, a reception unit 12, and a processing unit 33. The storage unit 31 includes an NN storage unit 111, an firing start point storage unit 112, an MNFP storage unit 113, an avatar storage unit 314, and a muscle storage unit 315. The processing unit 33 includes an MNFP determination unit 131, a movement determination unit 332, a movement unit 333, a muscle change unit 334, and a learning unit 135. The movement determination unit 332 includes muscle determination means 1321 and strength acquisition means 1322.

[0237] Various types of information are stored in the storage unit 31 that constitutes the information processing device 3. The various types of information include, for example, neural networks (NNs), firing start point information, MNFPs, avatar information, and muscle information.

[0238] The avatar storage unit 314 stores avatar information for outputting avatars. The avatar information here does not need to have joints. The avatar information is, for example, information for configuring the face of a living creature such as a human. As mentioned above, the data structure of the avatar information is not important. However, the avatar information usually includes model information. The model information is information for configuring the display of the avatar. The model information includes, for example, mesh information.

[0239] The muscle storage unit 315 stores two or more pieces of muscle information. Here, the muscle information is information about a mesh that moves when one or two or more nodes are fired. The muscle information is information associated with one or two or more firing node identifiers. The muscle information has, for example, a muscle identifier. The muscle information has, for example, a muscle identifier, maximum power, and mesh position. Note that a mesh is a partial area of ​​the avatar's face. The muscle identifier here is information that identifies a mesh that is a partial area of ​​the avatar's face. The muscle identifier here may also be called a mesh identifier. The mesh position is information that specifies the position of the mesh.

[0240] The processing unit 33 performs various types of processing. The various types of processing are, for example, processing performed by the movement determination unit 332, the movement unit 333, and the muscle change unit 334.

[0241] Movement determination unit 332 sequentially acquires movement determination information corresponding to each of one or more MNFPs determined by MNFP determination unit 131. Movement determination unit 332 performs the processing normally performed by muscle determination means 1321 and strength acquisition means 1322.

[0242] The action unit 333 sequentially uses one or more pieces of action determination information acquired by the action determination unit 332 to cause the action target 4 to act.

[0243] The movement unit 333 sequentially moves the movement target 4 based on one or more pieces of movement muscle information, which are pairs of muscle identifiers included in each piece of muscle information determined by the muscle determination means 1321 and intensities acquired by the intensity acquisition means 1322.

[0244] The operation unit 333 sequentially expands or contracts the meshes identified by the muscle identifiers included in the muscle information determined by the muscle determination unit 1321.

[0245] When the muscle identifier determined by the muscle determination means 1321 satisfies a predetermined increase condition, the muscle change unit 334 performs a change process to increase the maximum power paired with the muscle identifier.

[0246] The muscle change unit 334 performs a change process to decrease the maximum power paired with a muscle identifier when the muscle identifier not determined by the muscle determination means 1321 satisfies a predetermined decrease condition. The muscle change unit 334 may perform the same process as the muscle change unit 134.

[0247] The storage unit 31, the avatar storage unit 314, and the muscle storage unit 315 are preferably non-volatile recording media, but may also be realized as volatile recording media.

[0248] There is no restriction on the process by which information is stored in the storage unit 11 etc. For example, information may be stored in the storage unit 31 etc. via a recording medium, information transmitted via a communication line etc. may be stored in the storage unit 31 etc., or information input via an input device may be stored in the storage unit 11 etc.

[0249] The processing unit 33, the movement determination unit 332, the movement unit 333, and the muscle change unit 334 can usually be realized by a processor, a memory, etc. The processing procedures of the processing unit 33, etc. are usually realized by software, and the software is recorded on a recording medium such as a ROM. However, they may also be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0250] Next, a description will be given of an example of the operation of the information processing device 3 constituting the operating system B. The example of the operation of the information processing device 3 is the same as the operation described using the flowchart in FIG. 2, except for the operation process in step S206.

[0251] An example of the operational processing performed by the information processing device 3 will be described below with reference to the flowchart of Fig. 16. In the flowchart of Fig. 16, the description of the same steps as in the flowchart of Fig. 5 will be omitted.

[0252] (Step S1601) The operation determination unit 132 assigns 1 to a counter l.

[0253] (Step S1602) The movement determination unit 132 determines whether or not there is an l-th muscle identifier whose corresponding firing count is equal to or greater than 1. If there is an l-th muscle identifier, the process proceeds to step S1603; if there is not, the process proceeds to step S1607.

[0254] (Step S1603) The movement determination unit 132 acquires the number of firings corresponding to the lth muscle identifier. If there is a maximum power corresponding to the lth muscle identifier, the movement determination unit 132 acquires the number of firings and the maximum power.

[0255] (Step S1604) The intensity acquisition unit 1322 acquires the intensity using the number of firings acquired in step S1603. Alternatively, the intensity acquisition unit 1322 may acquire the intensity using the number of firings and the maximum power.

[0256] (Step S1605) The movement determination unit 132 temporarily stores the l-th muscle identifier and the strength acquired in step S1604 in a pair in a buffer (not shown).

[0257] (Step S1606) The operation determination unit 132 increments the counter 1 by 1. The process returns to step S1602.

[0258] (Step S1607) The operation unit 133 uses one or more pairs of muscle identifiers and strengths stored in a buffer (not shown) to contract the muscles (here, typically meshes) identified by each muscle identifier by the corresponding strength, thereby moving the object of operation 4 (for example, the face of an avatar), and then returns to the upper level processing.

[0259] As described above, according to this embodiment, facial movements corresponding to received input information, that is, facial movements corresponding to the perception of living creatures such as humans, can be appropriately simulated.

[0260] (Embodiment 3) In this embodiment, the way in which the action object is made to act differs from the above embodiment: In this embodiment, the action object performs an action in accordance with the acquired action identifier.

[0261] 17 is a block diagram of an operating system C according to the present embodiment. The operating system C includes an information processing device 5 and an operating target 6. The information processing device 5 may also include the operating target 6.

[0262] The moving object 6 is an object that performs a movement. The moving object 6 is, for example, an avatar or a robot, but the type is not important.

[0263] The information processing device 5 includes a storage unit 51, a reception unit 12, and a processing unit 53. The storage unit 51 includes an NN storage unit 111, an ignition starting point storage unit 112, an MNFP storage unit 113, an avatar storage unit 114, and a module storage unit 511. The processing unit 53 includes an MNFP determination unit 131, an action determination unit 532, an action unit 533, and a learning unit 135.

[0264] Various types of information are stored in the storage unit 51 that constitutes the information processing device 5. The various types of information include, for example, neural networks (NNs), firing start point information, MNFPs, avatar information, and modules.

[0265] The module storage unit 511 stores one or more operation modules. An operation module is a program for operating the operation target 6. An operation module may be, for example, an execution module, a function, or a method, but the type is not important. An operation module is associated with an operation identifier. An operation identifier is information that identifies an operation. An operation identifier is, for example, an operation module name or an operation module ID.

[0266] Processing unit 53 performs various types of processing. The various types of processing are, for example, processing performed by MNFP determination unit 131, operation determination unit 532, operation unit 533, and learning unit 135.

[0267] The operation determination unit 532 sequentially acquires operation determination information corresponding to each of the one or more MNFPs determined by the MNFP determination unit 131.

[0268] Operation determination section 532 sequentially acquires operation identifiers corresponding to one or more MNFPs determined by MNFP determination section 131. Note that an operation identifier is associated with each MNFP in MNFP storage section 113.

[0269] The action unit 533 sequentially uses one or more pieces of action determination information acquired by the action determination unit 532 to cause the action target 6 to act.

[0270] The operation unit 533 sequentially executes the operation modules associated with the operation identifiers acquired by the operation determination unit 532, and causes the operation target 6 to operate.

[0271] The storage unit 51 and the module storage unit 511 are preferably non-volatile recording media, but may also be realized as volatile recording media.

[0272] There is no restriction on the process by which information is stored in the storage unit 51 etc. For example, information may be stored in the storage unit 51 etc. via a recording medium, information transmitted via a communication line etc. may be stored in the storage unit 51 etc., or information input via an input device may be stored in the storage unit 51 etc.

[0273] The processing unit 53, the operation determination unit 532, and the operation unit 533 can usually be realized by a processor, a memory, etc. The processing procedures of the processing unit 53, etc. are usually realized by software, and the software is recorded on a recording medium such as a ROM. However, they may also be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0274] Next, a description will be given of an example of the operation of the information processing device 5 constituting the operating system C. The example of the operation of the information processing device 3 is the same as the operation described using the flowchart in FIG. 2, except for the operation process in step S206.

[0275] An example of the operation process performed by the information processing device 3 will be described below with reference to the flowchart of Fig. 18. In the flowchart of Fig. 18, the description of the same steps as those in the flowchart of Fig. 5 will be omitted.

[0276] (Step S1801) The action determining unit 532 acquires the action identifier corresponding to the i-th MNFP from the MNFP storage unit 113.

[0277] (Step S1802) Operation unit 533 acquires the operation module identified by the operation identifier acquired in step S1801 from module storage unit 511, and executes the operation module. Then, the process proceeds to step S510.

[0278] As described above, according to this embodiment, it is possible to easily simulate the movement of a living thing such as a human being in response to its cognition.

[0279] An example of the learning process performed by the information processing device 1 or the information processing device 3 in the above-described embodiment will be described below with reference to the flowchart in Fig. 19. In the flowchart in Fig. 19, the description of the same steps as those in Fig. 2 will be omitted. Note that the learning process is a process of learning an MNFP.

[0280] (Step S1901) The learning unit 135 determines whether or not input information and movement determination information to be used for learning have been received. If such information has been received, the process proceeds to step S1902, and if not, the process returns to step S1901. The movement determination information here is, for example, movement muscle information.

[0281] (Step S1902) The learning unit 135 determines whether or not the learning conditions are satisfied. If the learning conditions are satisfied, the process proceeds to step S1903, and if the learning conditions are not satisfied, the process proceeds to step S208.

[0282] The learning conditions are conditions for storing MNFPs in MNFP storage unit 113. The learning conditions are, for example, a time point number condition and an ignition information condition. The time point number condition is a condition related to the number of time points in the ignition information stored in ignition information storage unit 117, for example, that the number of time points is equal to or greater than a threshold. The ignition information condition is a condition related to the content of the ignition information, for example, that the number of ignition nodes is equal to or greater than a threshold.

[0283] (Step S1903) The learning unit 135 acquires the MNFP stored in the firing information storage unit 117 at the current time point. The MNFP here is information that specifies a set of firing node identifiers at each time point from the initial time point to the current time point, and is, for example, the matrix described above. The matrix has two axes: a time axis and a node identifier.

[0284] (Step S1904) Learning unit 135 performs MNFP accumulation processing. An example of the MNFP accumulation processing will be described with reference to the flowchart in Fig. 20. Note that the MNFP accumulation processing is processing for accumulating MNFPs in MNFP storage unit 113.

[0285] In the flowchart of FIG. 19, the learning unit 135 may perform the processes in steps S203, S208, and S209.

[0286] Next, an example of the MNFP accumulation process in step S1904 will be described with reference to the flowchart in FIG.

[0287] (Step S2001) The learning unit 135 acquires one or more muscle identifiers included in the received movement determination information. The one or more muscle identifiers are identifiers of muscles that will move when the input information is received.

[0288] (Step S2002) The learning unit 135 assigns 1 to the counter i.

[0289] (Step S2003) Learning unit 135 determines whether the i-th muscle identifier exists among the muscle identifiers acquired in step S2001. If the i-th muscle identifier exists, the process proceeds to step S2004; if not, the process proceeds to step S2007.

[0290] (Step S2004) The learning unit 135 refers to the muscle storage unit 315 and acquires one or more firing node identifiers corresponding to the i-th muscle identifier.

[0291] (Step S2005) The learning unit 135 sets the node value, which is the node identifier at the last point in time in the MNFP acquired in step S1903 and corresponds to one or more firing node identifiers acquired in step S2004, to the firing node value (e.g., "1").

[0292] (Step S2006) The learning unit 135 increments the counter i by 1. The process returns to step S2003.

[0293] (Step S2007) Learning unit 135 stores the MNFP changed in step S2005 in MNFP storage unit 113. The process returns to the upper level processing.

[0294] Next, an example of the learning process performed by the information processing device 5 will be described. The learning process here differs from the learning process described with reference to FIG. 19 only in the MNFP accumulation process.

[0295] The MNFP accumulation process performed by the learning unit 135 of the information processing device 5 is a process of associating the MNFP acquired in S1903 with the action determination information accepted in S1901 and accumulating them in the MNFP storage unit 113. Note that the action determination information here is an action identifier.

[0296] 21 shows the appearance of a computer that executes the programs described herein to realize the information processing device 1 and the like according to the various embodiments described above. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 21 is an overview of this computer system 300, and FIG. 22 is a block diagram of the system 300.

[0297] In FIG. 21, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0298] 22, computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing instructions of application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data. Although not shown here, computer 301 may further include a network card for providing connection to a LAN.

[0299] A program that causes computer system 300 to execute the functions of information processing device 1 and the like of the above-described embodiment may be stored on CD-ROM 3101, inserted into CD-ROM drive 3012, and then transferred to hard disk 3017. Alternatively, the program may be transmitted to computer 301 via a network (not shown) and stored on hard disk 3017. The program is loaded into RAM 3016 when executed. The program may also be loaded directly from CD-ROM 3101 or the network.

[0300] The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 301 to execute the functions of the information processing device 1 of the above-described embodiment. The program only needs to include instructions that call appropriate functions (modules) in a controlled manner and achieve desired results. How the computer system 300 operates is well known, and a detailed description thereof will be omitted.

[0301] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmission step (processing that can only be performed by hardware).

[0302] The computer that executes the program may be a single computer or a plurality of computers, that is, it may perform centralized processing or distributed processing.

[0303] Furthermore, in each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0304] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention. [Industrial Applicability]

[0305] As described above, the information processing device 1 according to the present invention has the effect of being able to appropriately simulate the movement of a living thing such as a human being in accordance with its cognition, and is useful as a simulation device or the like. [Explanation of symbols]

[0306] 1, 3, 5 Information processing equipment 2, 4, 6 operation target 11, 31, 51 storage compartment 12 Reception 13, 33, 53 Processing section 111 NN storage area 112 Firing start point storage section 113 MNFP storage area 114, 314 Avatar storage area 115, 315 muscle storage area 116 Joint storage section 117 Firing information storage unit 121 Input reception section 131 MNFP Determination Department 132, 332, 532 Operation decision unit 133, 333, 533 Operating part 134, 334 Muscle modification section 135 Learning Department 511 Module storage section 1311 Initial determination means 1312 Means of propagation 1313 MNFP determination means 1321 Muscle determination means 1322 Intensity acquisition means 1323 Angle acquisition means

Claims

1. a NN storage unit in which a neural network having two or more nodes identified by node identifiers is stored; a MNFP storage unit that stores one or more motor neuron firing patterns (MNFPs), which are information that identifies one or more firing nodes at each of two or more time points, in association with motion determination information that determines a motion of a motion target; an input receiving unit that receives input information; an MNFP determination unit that applies the input information to the neural network, acquires one or more ignition node identifiers from two or more nodes included in the neural network in a time series, and sequentially determines one or more MNFPs corresponding to the one or more ignition node identifiers in the time series from the one or more MNFPs in the MNFP storage unit; an operation determination unit that sequentially acquires operation determination information corresponding to each of the one or more MNFPs determined by the MNFP determination unit; an operation unit that sequentially uses the one or more pieces of operation determination information acquired by the operation determination unit to operate the operation target, a muscle storage unit for storing two or more pieces of muscle information, the muscle information being information about muscles that move when the one or more nodes are fired, the information being associated with one or more firing node identifiers, and the information having a muscle identifier and a maximum power; The operation determination unit a muscle determination means for sequentially determining one or more pieces of muscle information associated with one or more firing node identifiers specified by the one or more MNFPs determined by the MNFP determination unit; an intensity acquisition means for acquiring, for each of the one or more pieces of muscle information determined by the muscle determination means, a firing number, which is the number of firing node identifiers corresponding to the muscle information, and sequentially acquiring the intensity of the movement of the muscle specified by the muscle information using the firing number and a maximum power of the muscle information; The operating unit An information processing device that sequentially moves the object to be moved based on one or more pieces of movement muscle information that are pairs of a muscle identifier included in each of the one or more pieces of muscle information determined by the muscle determination means and the strength acquired by the strength acquisition means.

2. The method further includes a joint storage unit in which one or more pieces of joint information having a joint identifier are stored in association with two or more muscle identifiers, The operation determination unit and angle acquisition means for referring to the joint storage unit and using the one or more pieces of motion muscle information to sequentially acquire angles of joints identified by one or more joint identifiers associated with muscle identifiers included in the one or more pieces of muscle information determined by the muscle determination means; The operating unit The information processing apparatus according to claim 1 , wherein the angle for each of the one or more joints is output to the object to be moved, thereby moving the object to be moved.

3. the action target is an avatar, the joint information is information about the joints of the avatar and includes joint positions that identify positions of the joints of the avatar; further comprising an avatar storage unit in which avatar information for outputting the avatar is stored; The operating unit The information processing apparatus according to claim 2 , further comprising: using the avatar information to create an avatar in which the joints specified by the joint positions of the one or more joints are bent in accordance with the angles corresponding to the joints; and outputting the avatar.

4. a NN storage unit in which a neural network having two or more nodes identified by node identifiers is stored; a MNFP storage unit that stores one or more motor neuron firing patterns (MNFPs), which are information that identifies one or more firing nodes at each of two or more time points, in association with motion determination information that determines a motion of a motion target; an input receiving unit that receives input information; an MNFP determination unit that applies the input information to the neural network, acquires one or more ignition node identifiers from two or more nodes included in the neural network in a time series, and sequentially determines one or more MNFPs corresponding to the one or more ignition node identifiers in the time series from the one or more MNFPs in the MNFP storage unit; an operation determination unit that sequentially acquires operation determination information corresponding to each of the one or more MNFPs determined by the MNFP determination unit; an operation unit that sequentially uses the one or more pieces of operation determination information acquired by the operation determination unit to operate the operation target, the object of action is the face of an avatar, a muscle storage unit for storing two or more pieces of muscle information, the muscle information being associated with the node identifiers of one or more nodes and relating to meshes that move when the one or more nodes are fired, the muscle information having a muscle identifier that identifies a mesh of the face of the avatar; The operation determination unit a muscle determination means for sequentially determining one or more pieces of muscle information associated with one or more firing node identifiers identified by the one or more MNFPs determined by the MNFP determination unit; The operating unit an information processing device that sequentially stretches or shrinks meshes identified by muscle identifiers included in the muscle information determined by the muscle determination means;

5. The operation determination unit a muscle determination means for sequentially determining one or more pieces of muscle information associated with one or more firing node identifiers specified by the one or more MNFPs determined by the MNFP determination unit; an intensity acquisition means for acquiring, for each of the one or more pieces of muscle information determined by the muscle determination means, a firing number, which is the number of firing node identifiers corresponding to the muscle information, and sequentially acquiring the intensities of the movements of the muscles identified by the muscle information using the firing numbers; The operating unit An information processing device as described in claim 4, wherein the object to be moved is moved sequentially based on one or more movement muscle information which is a pair of a muscle identifier of each of the one or more muscle information determined by the muscle determination means and the strength acquired by the strength acquisition means.

6. the action determination information is an action identifier for identifying an action, a module storage unit in which an operation module for operating the operation target is stored in association with an operation identifier; The operation determination unit acquiring an operation identifier corresponding to the multi-function peripheral (MNFP) determined by the multi-function peripheral (MNFP) determination unit; The operating unit The information processing apparatus according to claim 1 , wherein the action determining unit executes the action module associated with the acquired action identifier to cause the action target to operate.

7. 7. The information processing device according to claim 1, further comprising a muscle change unit that performs a change process to increase the maximum power paired with the muscle identifier when the muscle identifier determined by the muscle determination means satisfies a predetermined increase condition.

8. 8. The information processing device according to claim 1, further comprising a muscle change unit that performs a change process to reduce the maximum power paired with a muscle identifier not determined by the muscle determination means when the muscle identifier satisfies a predetermined reduction condition.

9. 9. The information processing device according to claim 1, further comprising a learning unit that performs a learning process to acquire an MNFP when the input receiving unit receives input information, receive operation determination information, and store the MNFP in the MNFP storage unit in association with the operation determination information.

10. each of one or more nodes among the two or more nodes included in the neural network is associated with a firing condition and with one or more connecting node identifiers that identify other nodes to which the firing propagates; The MNFP determination unit an initial determination means for acquiring one or more pieces of characteristic information from the input information accepted by the input accepting unit, determining a firing condition that matches the one or more pieces of characteristic information, and acquiring one or more firing node identifiers corresponding to the firing condition; a propagation means for acquiring one or more connection node identifiers corresponding to the one or more firing node identifiers acquired by the initial determination means, and then successively acquiring one or more connection node identifiers corresponding to the one or more connection node identifiers; 10. The information processing device according to claim 1, further comprising: an MNFP determination means for acquiring, in time series, firing information having one or more firing node identifiers from among one or more firing node identifiers and one or more connection node identifiers acquired by the initial determination means and the propagation means, and sequentially determining one or more MNFPs corresponding to the firing information in time series from the one or more MNFPs in the MNFP storage unit.

11. The information processing device according to claim 1 , wherein the number of time points of at least two of the two or more MNFPS is different.

12. An information processing method, comprising the steps of: causing a computer to execute all of the processes performed by the information processing device according to claim 1 .

13. Computer, A program for causing an information processing device to function as the information processing device according to any one of claims 1 to 11.

Images