Information processing device, information processing method, and information processing program

Abstract

This technology aims to prevent the prediction accuracy of neural networks from becoming low. [Solution] The information processing device includes: an acquisition unit that acquires training data including multiple sets of input variables including multiple elements and output variables including multiple elements; a neural network group configuration unit that constitutes multiple neural networks, each corresponding to one of the elements included in the output variables; and an input layer correction unit that corrects the input layer of each of the multiple neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] Techniques for generating neural networks are known. For example, Non-Patent Document 1 describes using a type of recurrent neural network called an Echo State Network (ESN) for identifying complex non-linear dynamic systems. ESN achieves lightweight and global optimality by learning only the output layer.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, among the elements included in the output variable of the neural network, there may be elements in which the contribution of specific elements included in the input variable is large and the contribution of other elements is low. In such a case, in the technique described in Non-Patent Document 1, there is a problem that the influence of the elements of the input variable with low contribution is overly large, and as a result, the prediction accuracy of the output variable decreases.

[0005] This disclosure has been made in view of the above-mentioned problems, and one exemplary purpose thereof is to provide a technique to prevent the prediction accuracy of neural networks from becoming low. [Means for solving the problem]

[0006] An identification device for a dynamic system relating to an illustrative aspect of this disclosure includes: acquisition means for acquiring training data including multiple sets of input variables including multiple elements and output variables including multiple elements; neural network group configuration means for configuring multiple neural networks, each corresponding to one of the elements included in the output variable; and input layer correction means for correcting the input layer of each of the multiple neural networks based on the correlation between the elements of the output variable corresponding to each neural network and the input variable.

[0007] An information processing method relating to an illustrative aspect of this disclosure includes: an acquisition process in which at least one processor acquires training data comprising multiple sets of input variables comprising multiple elements and output variables comprising multiple elements; a neural network group configuration process in which the at least one processor constitutes a plurality of neural networks, each corresponding to one of the elements included in the output variables; and an input layer correction process in which the at least one processor corrects the input layer of each of the plurality of neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables.

[0008] An illustrative aspect of the present disclosure is an information processing program for causing a computer to function as an information processing device, wherein the computer functions as: an acquisition means for acquiring training data including multiple sets of input variables including multiple elements and output variables including multiple elements; a neural network group configuration means for configuring multiple neural networks, each corresponding to one of the elements included in the output variables; and an input layer correction means for correcting the input layer of each of the multiple neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables. [Effects of the Invention]

[0009] One exemplary aspect of this disclosure is that it can provide a technique to prevent the prediction accuracy of neural networks from becoming low. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the configuration of the information processing device related to this disclosure. [Figure 2] This is a flowchart showing the flow of the information processing method related to this disclosure. [Figure 3] This is a block diagram showing the configuration of the information processing device related to this disclosure. [Figure 4] This figure shows an example of output and input variables related to this disclosure. [Figure 5] This flowchart shows the flow of the learning phase execution method related to this disclosure. [Figure 6] This figure shows an example of a neural network related to this disclosure. [Figure 7] This figure shows an example of the estimated phase execution method related to this disclosure. [Figure 8] This figure shows the data observed by the dynamic system related to this disclosure. [Figure 9] This figure shows the evaluation results related to this disclosure. [Figure 10]This figure shows the evaluation results related to this disclosure. [Figure 11] This figure shows the correlation coefficient and coefficient of determination calculated in this disclosure. [Figure 12] This is a block diagram showing the configuration of a computer that functions as an information processing device related to this disclosure. [Figure 13] This diagram schematically shows the configuration of an ESN related to conventional technology. [Modes for carrying out the invention]

[0011] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining some or all of the technologies (things or methods) employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technologies employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. In addition, the effects mentioned in each of the exemplary embodiments shown below are examples of effects that can be expected in that exemplary embodiment and do not define the scope of the present invention. That is, embodiments that do not produce the effects mentioned in each of the exemplary embodiments shown below may also be included in the scope of the present invention.

[0012] [First Exemplary Embodiment] A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form for each of the exemplary embodiments described later. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems occur. Furthermore, each technology shown in the drawings referenced to explain this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems occur.

[0013] <Configuration of the information processing device> The configuration of the information processing device 1 will be explained with reference to Figure 1. Figure 1 is a block diagram showing the configuration of the information processing device 1. As shown in Figure 1, the information processing device 1 comprises an acquisition unit 11, a neural network group configuration unit 12, and an input layer correction unit 13.

[0014] The acquisition unit 11 acquires training data that includes multiple sets of input variables containing multiple elements and output variables containing multiple elements. The neural network group configuration unit 12 configures multiple neural networks, each corresponding to one of the elements included in the output variable. The input layer correction unit 13 corrects the input layer of each of the multiple neural networks based on the correlation between the elements of the output variable corresponding to each neural network and the input variable.

[0015] <Effects of Information Processing Devices> As described above, the information processing device 1 employs a configuration comprising: an acquisition unit 11 that acquires training data containing multiple sets of input variables containing multiple elements and output variables containing multiple elements; a neural network group configuration unit 12 that constitutes multiple neural networks, each corresponding to one of the elements included in the output variables; and an input layer correction unit 13 that corrects the input layer of each of the multiple neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables. Therefore, the information processing device 1 has the effect of preventing the prediction accuracy of the neural network from becoming low.

[0016] <Information Processing Flowchart> The flow of the information processing method S1 will be explained with reference to Figure 2. Figure 2 is a flowchart showing the flow of the information processing method S1. As shown in Figure 2, the information processing method S1 includes an acquisition process S11, a neural network group configuration process S12, and an input layer correction process S13.

[0017] In the acquisition process S11, at least one processor acquires training data that includes multiple sets of input variables containing multiple elements and output variables containing multiple elements. In the neural network group configuration process S12, the at least one processor configures multiple neural networks, each corresponding to one of the elements included in the output variables. In the input layer correction process S13, the at least one processor corrects the input layer of each of the multiple neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables.

[0018] (Effects of information processing methods) As described above, the information processing method S1 employs a configuration that includes: an acquisition process S11 in which at least one processor acquires training data containing multiple sets of input variables containing multiple elements and output variables containing multiple elements; a neural network group configuration process S12 in which the at least one processor constitutes multiple neural networks, each corresponding to one of the elements included in the output variables; and an input layer correction process S13 in which the at least one processor corrects the input layer of each of the multiple neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables. Therefore, the information processing method S1 has the effect of preventing the prediction accuracy of the neural network from becoming low.

[0019] [Second exemplary embodiment] A second exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same function as those described in the above-described exemplary embodiment are denoted by the same reference numerals, and their descriptions are omitted as appropriate. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems arise. Furthermore, each technology shown in the drawings referenced to describe this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems arise.

[0020] <Overview of Information Processing Devices> This section describes the overview of the information processing device 1A. The information processing device 1A is a device that generates multiple neural networks. These multiple neural networks are used, for example, to identify dynamical systems. An example of a dynamical system is a system that controls the drive of a forklift. However, the systems identified by the information processing device 1A are not limited to systems that control the drive of a forklift. The identification method performed by the information processing device 1A is applicable to systems in various other technological fields.

[0021] (Identification of dynamic systems) Here, we will explain the identification of dynamic systems. A static system is one in which the output x(t) at a given time t is determined solely by the input u(t) at that time, whereas a dynamic system is one in which the output x(t) at a given time t is determined not only by the input u(t) at that time but also by past inputs, the current state, and other factors. A dynamic system can be described by differential equations as an example.

[0022] Dynamic system identification refers to the process of estimating the governing equations (equations of motion) of an unknown dynamic system from the relationships (data) between various inputs and outputs. There are two types of dynamic system identification: parametric identification and nonparametric identification. Parametric identification is a method that estimates parameters by assuming the structure of the dynamic system. Nonparametric identification is a method that also estimates the structure of the dynamic system itself.

[0023] One parametric identification method involves assuming governing equations, defining unknown parameters, and calculating the unknown parameters that minimize modeling errors. However, for complex dynamic systems that output things like friction between a vehicle and the road surface, or play (gap) in machinery, it is impossible to formulate differential equations. Furthermore, even if differential equations could be formulated, it would be impossible to estimate the unknown parameters. This is because the differential equations may be nonlinear and unsolvable, or unknown parameters cannot be estimated from observational information alone, or the moment of inertia, for example, cannot be measured. On the other hand, approximation with easily solvable differential equations (e.g., second-order linear differential equations) is also an option, but in this case, there is a problem in that the gap with reality (modeling error) becomes large.

[0024] Therefore, in recent years, research has focused on data-driven, non-parametric system identification utilizing machine learning. In particular, ESNs (Echo State Networks) are being used as a powerful tool for identifying non-linear and complex dynamic systems.

[0025] (ESN) An ESN is a type of recurrent neural network (RNN). ESNs achieve lightweight design and global optimization by training only the output layer. Figure 13 shows a schematic diagram of an ESN configuration. In the example in Figure 13, the ESN network 300 has an input layer W in Reservoir layer A, output layer W outIt consists of. The reservoir layer A is a sparse and randomly connected recurrent neural network. In network 300, the input layer W in and the reservoir layer A are randomly generated, and the input layer W in and the reservoir layer A are not learned. In network 300, only the output layer W out learns to reduce the computational cost and realizes real-time performance.

[0026] (Mathematical Structure of ESN) As an example, the dynamics of ESN are represented by the following equation (1).

Equation

[0027] Here, let the number of input nodes of the input layer W in be m, the number of output nodes of the output layer W out be n, and the number of neurons in the reservoir layer A be p. In equation (1), the explanatory variable u(t) is the input to the ESN and has m elements. The target variable y(t) is the output from the ESN and has n elements. r(t) is the latent variable. The input layer W in is a p×m weight matrix, the reservoir layer A is a p×p weight matrix, and the output layer W out is an n×p weight matrix. The parameter α∈(0,1] is the leakage rate, which forces the evolution of the reservoir layer to slow down when α→0. The activation function ψ() is a non-linear function such as the tanh function, etc.

[0028] At this time, the learning of the output layer W out of ESN is performed, as an example, by the following equation (2).

Equation

[0029] In equation (2), the matrix Y = [y(t1),…,y(t N )], and the matrix R = [r(t1),…,r(t N)]. That is, matrix Y is a matrix in which the output variable y(t) is arranged according to the observed values, and matrix R is a matrix in which the latent variable r(t) is arranged according to the observed values. Parameter β is a normalization parameter that guarantees non-singularity.

[0030] In RNNs, training up to the reservoir layer requires error propagation and optimization, resulting in high computation time. Therefore, RNNs cannot be used for real-time system identification. In contrast, with ESNs, training is limited to the output layer, allowing the analytical optimal solution to be calculated in a single step, dramatically reducing computational cost. The reservoir layer of an ESN is initialized randomly and never trained, yet it can model complex nonlinear systems with sufficient accuracy.

[0031] On the other hand, since ESNs do not optimize the input layer, the latent variables generated from the input layer may not meet the desired performance; that is, the range of values ​​and the amount of change of the latent variables may lack the expressive power necessary to accurately estimate the output variables. Therefore, the information processing device 1A improves the accuracy of modeling dynamic systems by multiplying the randomly generated input layer by predetermined weights.

[0032] <Configuration of the information processing device> The configuration of the information processing device 1A will be described with reference to the drawings. Figure 3 is a block diagram showing the configuration of the information processing device 1A. The information processing device 1A comprises a control unit 10A, a storage unit 20A, a communication unit 30A, an input unit 40A, and an output unit 50A.

[0033] (Communications Department) The communication unit 30A communicates with external devices of the information processing device 1A via a communication line. The specific configuration of the communication line is not limited to this exemplary embodiment, but examples of communication lines include wireless LAN (Local Area Network), wired LAN, WAN (Wide Area Network), public telephone network, mobile data communication network, or a combination thereof. The communication unit 30A transmits data supplied from the control unit 10A to other devices and supplies data received from other devices to the control unit 10A.

[0034] (Input section) The input unit 40A is configured to receive input to the information processing device 1A, and may include, for example, an input device such as a keyboard, mouse, touch panel, camera, or microphone. Alternatively, the input unit 40A may be configured to receive data from the input device via an interface such as USB (Universal Serial Bus).

[0035] (Output section) The output unit 50A is configured to output from the information processing device 1A, and may include, for example, an output device such as a display, printer, touch panel, or speaker. The output unit 50A may also be configured to have an interface such as USB, and to output data to the output device via this interface.

[0036] (Storage part) The memory unit 20A stores various types of information that the control unit 10A references. Examples of such information include training data 201 and neural network group 202. The neural network group 202 is a collection of neural networks. Here, when we say that a neural network is stored in the memory unit 20A, we mean that the parameters that define the neural network are stored in the memory unit 20A.

[0037] (Training data) Training data 201 is data used for machine learning of multiple neural networks included in the neural network group 202. Training data 201 has m elements u j An input variable u containing (j=1,2,…,m) and n elements y i The output variable y includes (i=1,2,…,n) and contains multiple sets of values.

[0038] Here, the relationship between each element of the output variable y and each element of the input variable u will be explained with reference to the diagram. Figure 4 shows an example of the output variable y and input variable u. In the example in Figure 4, an example of the output variable y and input variable u is shown when the dynamic system being identified is a forklift control system. In the example in Figure 4, the input variable u is a 5-dimensional vector containing components a, b, c, cosθ, and sinθ. Components a, b, and c are parameters of the accelerator lever, steering wheel, and brake pedal, respectively. The output variable y is a 3-dimensional vector containing components of x-axis velocity, y-axis velocity, and angular velocity.

[0039] In the example in Figure 4, each element (component) of the output variable y is connected by a line to the element of the input variable u that has a large contribution to that element. Elements that are not connected by a line are not unrelated, but their influence is small. As shown in Figure 4, for each element of the output variable y, the element of the input variable that has a large contribution is different.

[0040] (Neural network group) The neural network group 202 is a collection of neural networks. A neural network is, for example, an ESN, but is not limited to this. Each of the multiple neural networks contains elements y in the output variable y. i It corresponds to one of the following. More specifically, the neural network group 202 is, for example, a set of n neural networks 2021, 2022, ..., 202n. In this case, it consists of n neural networks 202i (i=1,2,...,n) and elements y of the output variable y. iThere is a one-to-one correspondence between them. However, the number of neural networks is not limited to n; it can be more or less than n.

[0041] The input to the neural networks included in neural network group 202 is the input variable u. On the other hand, the output of the neural networks included in the group is not the output variable y, but differs for each of the multiple neural networks.

[0042] More specifically, for example, if the neural network group 202 is a set of n neural networks 2021, 2022, ..., 202n, then the output of neural network 2021 is element y1 of the output variable y, the output of neural network 2022 is element y2 of the output variable y, and so on. i In this case, with conventional ESNs, the output variable y is

number

number

[0043] Each of the multiple neural networks has an input layer W in Reservoir layer A, output layer W out It consists of: Input layer W in The first layer has m input nodes. The reservoir layer A has p neurons. The output layer W out It has one or more output nodes. However, the output layer W out The number of output nodes is less than the number of elements n in the output variable y. Reservoir layer A is a sparsely and randomly connected recurrent neural network. Input layer Win Reservoir layer A is generated randomly, and reservoir layer A is not trained.

[0044] (Control Unit) As shown in Figure 3, the control unit 10A includes a learning phase execution unit 110A and an estimation phase execution unit 120A. The learning phase execution unit 110A includes an acquisition unit 11A, a neural network group configuration unit 12A, a correlation calculation unit 13A, an input layer correction unit 14A, and a training unit 15A. The estimation phase execution unit 120A includes a second acquisition unit 16A and an output control unit 17A. The acquisition unit 11A, the neural network group configuration unit 12A, the input layer correction unit 14A, the second acquisition unit 16A, and the output control unit 17A are examples of the acquisition means, neural network group configuration means, input layer correction means, second acquisition means, and output control means according to this disclosure, respectively.

[0045] (Acquisition Department) The acquisition unit 11A acquires the training data 201. For example, the acquisition unit 11A may acquire the training data 201 by reading it from a storage location specified by the user of the information processing device 1A (which may be a storage device within the information processing device 1A or a storage device outside the information processing device 1A). Alternatively, the acquisition unit 11A may acquire the training data 201 by receiving it from another device via the communication unit 30A. Furthermore, the acquisition unit 11A may acquire the training data 201 input to the input unit 40A.

[0046] (Neural network group configuration) The neural network group component 12A constitutes multiple neural networks. More specifically, as an example, there is a one-to-one correspondence between multiple neural networks and the elements of the output variable, and the neural network group component 12A constitutes a number of neural networks equal to the number of elements contained in the output variable.

[0047] (Correlation calculation unit / Input layer correction unit) The correlation calculation unit 13A calculates the correlation between the input variables and the output variables. As an example, the correlation calculation unit 13A calculates the correlation coefficient or coefficient of determination between the input variables and the output variables. The input layer correction unit 14A corrects the input layer of each of the multiple neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables. More specifically, the input layer correction unit 14A increases the weight values, which are elements of the input layer of each of the multiple neural networks, as their contribution to the elements of the output variables corresponding to each neural network increases.

[0048] (Training Department) The training unit 15A trains the output layers of multiple neural networks whose input layers have been corrected by the input layer correction unit 14A, using the training data 201. As an example, the training unit 15A may train the output layers using the above equation (2).

[0049] (Second acquisition part) The second acquisition unit 16A acquires input data. For example, the second acquisition unit 16A may acquire input data by reading it from a storage location specified by the user of the information processing device 1A (which may be a storage device within the information processing device 1A or a storage device outside the information processing device 1A). Alternatively, the second acquisition unit 16A may acquire input data by receiving input data from another device via the communication unit 30A. Furthermore, the second acquisition unit 16A may acquire input data input to the input unit 40A.

[0050] (Output control unit) The output control unit 17A outputs output data that includes elements obtained by inputting the input data acquired by the second acquisition unit 16A into each of the multiple neural networks. For example, each of the multiple neural networks is an element y of the output variable y. i In the case of the above, the output control unit 17A inputs the input data to each of the n neural networks to obtain the n elements y1, y2, ..., y n Outputs output data that includes this data.

[0051] The output control unit 17A may output data by writing it to a storage location specified by the user of the information processing device 1A (which may be a storage device within the information processing device 1A or a storage device outside the information processing device 1A). Alternatively, the output control unit 17A may transmit the output data via the communication unit 30A, or it may output the output data to an output device such as a display.

[0052] <Flowchart of the learning phase execution method> Figure 5 is a flowchart showing the flow of the learning phase execution method S1A, which is an example of a learning phase execution method performed by the information processing device 1A. In this example, we will explain the case in which n neural networks 202i (i=1,2,…,n) are generated as multiple neural networks.

[0053] In step S101, the acquisition unit 11A acquires training data 201. In step S102, the neural network group configuration unit 12A configures n neural networks 202i (i=1,2,…,n).

[0054] Figure 6 shows an example of n neural networks 202i configured by the neural network group configuration unit 12A. In the example in Figure 6, the neural network 202i has an input layer W i in Reservoir layer A i Output layer W i out Includes.

[0055] Input layer W i in It has m input nodes and input layer W i in The input variable u is input to the input layer W. i in The input variable u can be expressed, for example, by the following equation (5).

number

[0056] Input layer W of neural network 202i i in and reservoir layer A i These are generated randomly. The neural network group component 12A is, as an example,

number

[0057] At this time, the input layer W i in Variable W generated from i in u is represented as follows:

number

[0058] In step S103, the correlation calculation unit 13A calculates the elements u of the input variable u j and the elements of the output variable y i The correlation calculation unit 13A calculates a coefficient representing the correlation with the element u of the input variable u, as an example. j and the elements of the output variable y i Correlation coefficient φ(y i ,u j ) is calculated, and the diagonal matrix Φ of the following equation (8) i To construct.

number

[0059] In step S104, the input layer correction unit 14A corrects each input layer W of the multiple neural networks 202i. i in The element is the weight value w iqjThe elements y of the output variable y corresponding to each neural network 202i i and the element u of the input variable u j The correction is performed using the correlation coefficient or coefficient of determination of the input layer. The input layer correction unit 14A, as an example, uses the element y of the output variable y. i Input layer W of neural network 202i for i in We define it as follows: Here, W i 0 This is an input layer generated by random numbers.

number

[0060] As described above, input layer W i in By defining the element y of the output variable, i This allows us to construct latent variables using input weights corresponding to their contributions.

[0061] In step S105, the training unit 15A controls the output layer W of the neural network 202i. i out The training unit 15A, as an example, uses the above formula (2) to train the output layer W i out To teach it.

[0062] <Flowchart of the estimated phase execution method> Figure 7 is a flowchart showing the flow of the estimation phase execution method S2A performed by the information processing device 1A. In step S201, the second acquisition unit 16A acquires input data. In step S202, the output control unit 17A inputs the input variables acquired by the second acquisition unit 16A to each of the neural networks 202i to obtain the output y i Output data that includes elements (i=1,2,…,n).

[0063] <Examples> An example in which a dynamic system for driving a forklift is identified using the information processing device 1A will be described. In this example, the input variable u and output variable y are as follows. The input variable u includes elements a, s, b, acosθ, asinθ, and as. Element a is the accelerator lever, element s is the steering wheel, element b is the brake pedal, element acosθ is the x-axis component of the accelerator, element asinθ is the y-axis component of the accelerator, and element as is the angular velocity component. The output variable y includes elements vx, vy, and ω. Element v x , v y ω represents the x-axis velocity, y-axis velocity, and angular velocity, respectively.

number

[0064] In this example, a neural network with the same number of elements as the output variable y was generated, and the parameters of the neural network were set as follows. • Dimension of latent variable p = 500 • Spectral radius ρ(A) = 0.9 • Sparsity rate S(A) = 0.7 Leakage rate α = 0.2 ·Regularization coefficient λ=0.01

[0065] In the above parameters, the spectral radius of matrix A in the reservoir layer (the maximum absolute value of the eigenvalues ​​of matrix A) must be less than 1. This is because, in an ESN, the influence of past inputs must decrease over time in the reservoir layer.

[0066] Figure 8 shows data observed in a dynamic system. Figure 8 shows data obtained by remotely controlling a forklift for 30 minutes. Inputs and outputs were saved as data when the forklift was remotely controlled to perform various actions (movements). 80% of this data was used as training data, and the remaining 20% ​​was used as validation data.

[0067] The conventional identification method and the identification method of the information processing device 1A were evaluated and compared. Figures 9 and 10 show the evaluation results. Figure 9 is a graph showing the valid time of the conventional identification method and the valid time of the identification method of the information processing device 1A. The valid time is the time at which the prediction error first exceeds a predetermined threshold, and a longer valid time indicates higher prediction accuracy (the ability to predict over a long period with small errors). Three types were compared: (1) Kinematics, (2) the conventional ESN, and (3) the method of the information processing device 1A. (1) Knematics is a classical system identification method based on the equations of motion of kinematics. The left figure of Figure 9 shows the valid time when the position error threshold is set to 20 cm. The right figure shows the valid time when the angle error threshold is set to 4°. As shown in Figure 9, the valid time of the identification method of the information processing device 1A is longer than that of the conventional method.

[0068] Figure 10 is a graph showing the RMSE (Root mean square error) of a conventional identification method and an identification method using the information processing device 1A. The RMSE is the square root of the mean square of the error between the true value and the predicted value, and a smaller RMSE indicates higher prediction accuracy (smaller error within a given range). Three types were compared: (1) Kinematics, (2) a normal ESN, and (3) the method using the information processing device 1A. The left graph of Figure 10 shows the RMSE of position, and the right graph shows the RMSE of angle. As shown in Figure 10, it can be seen that the RMSE related to the information processing device 1A is smaller than that of the conventional method.

[0069] Figure 11 shows the calculated correlation coefficient and coefficient of determination. In the example in Figure 11, the elements acosθ of the input variable u and the elements v of the output variable y are shown. x It can be seen that there is a high correlation with this.

[0070] <Effects of Information Processing Devices> By the way, in conventional ESNs, changes in input elements with low contribution (low relation) can sometimes lead to a decrease in the prediction accuracy of the output variable y. For example, a certain element y of the output variable y iIn some cases, the result is largely determined by the first and second elements u1 and u2 of the input variable u, and is hardly influenced by other elements. Even in this case, the 3rd, 4th, ... mth elements u3, u4, ... u m Since these are added in equal balance, this becomes noise during estimation, reducing accuracy.

[0071] In contrast, the information processing device 1A can prevent a decrease in the prediction accuracy of the neural network by correcting the input layer based on the correlation between the elements of the output variable and the input variable.

[0072] Furthermore, in the information processing device 1A, there is a one-to-one correspondence between multiple neural networks and the elements of the output variable, and the neural network group configuration unit 12A employs a configuration that forms a neural network equal to the number of elements included in the output variable. Therefore, the information processing device 1A prevents each element included in the output variable from being unnecessarily influenced by the elements of the input variable, which have a low contribution.

[0073] Furthermore, in the information processing device 1A, the input layer correction unit 14A employs a configuration in which the weight values, which are elements of the input layer of each of the multiple neural networks, are increased in proportion to the contribution of each neural network to the elements of the output variable. Therefore, the information processing device 1A can prevent each element included in the output variable from being greatly influenced by the elements of the input variable with low contribution.

[0074] Furthermore, in the information processing device 1A, the input layer correction unit 14A is configured to correct the weight values, which are elements of the input layer of each of the multiple neural networks, using the correlation coefficient or coefficient of determination between the elements of the output variables corresponding to each neural network and the elements of the input variables. Therefore, the information processing device 1A has the effect of reflecting the correlation between the input variables and the output variables in the input layer.

[0075] Furthermore, the information processing device 1A is configured to include a training unit 15A that trains the output layers of multiple neural networks, whose input layers have been corrected by the input layer correction unit 14A, using the training data 201. Therefore, the information processing device 1A can improve the prediction accuracy of multiple neural networks used to estimate output variables from input variables.

[0076] Furthermore, the information processing device 1A employs a configuration in which multiple neural networks are used to identify the behavior of a dynamic system. Therefore, the information processing device 1A can improve the prediction accuracy of the neural network used to identify the behavior of a dynamic system.

[0077] Furthermore, the information processing device 1A employs a configuration that includes a second acquisition unit 16A for acquiring input data, and an output control unit 17A for outputting output data that includes elements obtained by inputting the input data acquired by the second acquisition unit 16A into each of a plurality of neural networks. As a result, the information processing device 1A can, for example, estimate the output of a dynamic system with greater accuracy from input data which is the input to the dynamic system.

[0078] [Variation] The functions of the information processing devices 1 and 1A described above may be implemented by sharing the responsibilities of multiple devices. For example, the information processing device 1A described above may be implemented as a system in which two or more devices are connected via a communication network. In this case, the system may, for example, have a first device equipped with a learning phase execution unit 110A and a second device equipped with an estimation phase execution unit 120A. In this case, the functions of the information processing device 1A are realized through the cooperation of the first device and the second device.

[0079] Furthermore, although the above-described embodiment described a case in which the neural network group 202 is stored in the storage unit 20A of the information processing device 1A, the neural network group 202 may also be stored in a device other than the information processing device 1A (for example, an external server).

[0080] [Examples of implementation using software] Some or all of the functions of the information processing devices 1 and 1A (hereinafter also referred to as "the above devices") may be implemented by hardware such as integrated circuits (IC chips) or by software.

[0081] In the latter case, each of the above devices is implemented, for example, by a computer that executes instructions for a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 12. Figure 12 is a block diagram showing the hardware configuration of computer C, which functions as each of the above devices.

[0082] Computer C comprises at least one processor C1 and at least one memory C2. Memory C2 stores a program P that causes computer C to operate as each of the above-mentioned devices. In computer C, processor C1 reads program P from memory C2 and executes it, thereby realizing each of the above-mentioned devices.

[0083] For processor C1, for example, a CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination thereof can be used. For memory C2, for example, flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof can be used.

[0084] Computer C may also be equipped with RAM (Random Access Memory) for loading program P at runtime and for temporarily storing various data. Furthermore, computer C may be equipped with communication interfaces for sending and receiving data with other devices. Additionally, computer C may be equipped with input / output interfaces for connecting input / output devices such as keyboards, mice, displays, and printers.

[0085] Furthermore, program P can be recorded on a non-temporary, tangible recording medium M that is readable by computer C. Such a recording medium M could be, for example, tape, disk, card, semiconductor memory, or programmable logic circuitry. Computer C can acquire program P via such a recording medium M. Program P can also be transmitted via a transmission medium. Such a transmission medium could be, for example, a communication network or broadcast waves. Computer C can also acquire program P via such a transmission medium.

[0086] Furthermore, each of the above functions of each of the above devices may be implemented by a single processor in a single computer, by multiple processors in a single computer working together, or by multiple processors in each of multiple computers working together. In addition, the programs for implementing each of the above functions in each of the above devices may be stored in a single memory in a single computer, distributed and stored in multiple memories in a single computer, or distributed and stored in multiple memories in each of multiple computers.

[0087] [Additional Note A] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims. (Note A1) A means for acquiring training data that includes multiple sets of input variables containing multiple elements and output variables containing multiple elements, A neural network group configuration means comprising multiple neural networks, each of which corresponds to one of the elements included in the output variable, Input layer correction means for correcting the input layer of each of the plurality of neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables, An information processing device equipped with the following features.

[0088] (Appendix A2) There is a one-to-one correspondence between the aforementioned multiple neural networks and the elements of the output variables. The neural network group configuration means configures a neural network with a number of elements included in the output variable. The information processing device described in Appendix A1.

[0089] (Note A3) The input layer correction means increases the weight values, which are elements of the input layer of each of the plurality of neural networks, as their contribution to the elements of the output variables corresponding to each neural network increases. The information processing device described in Appendix A1 or A2.

[0090] (Note A4) The input layer correction means corrects the weight values, which are elements of the input layer of each of the plurality of neural networks, using the correlation coefficient or coefficient of determination between the elements of the output variable corresponding to each neural network and the elements of the input variable. The information processing device described in Appendix A3.

[0091] (Note A5) Training means for training each output layer of the plurality of neural networks whose input layers have been corrected by the input layer correction means using the training data, An information processing device described in any one of the appendices A1 to A4, further comprising the above.

[0092] (Note A6) The aforementioned multiple neural networks are used to identify the behavior of a dynamic system. An information processing device as described in one of the appendices A1 to A5.

[0093] (Note A7) A second acquisition means for acquiring input data, The system further comprises an output control means that outputs output data including elements obtained by inputting the input data acquired by the second acquisition means into each of the plurality of neural networks, An information processing device as described in any one of the appendices A1 to A6.

[0094] [Additional Note B] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims. (Note B1) At least one processor performs an acquisition process to acquire training data that includes multiple sets of input variables containing multiple elements and output variables containing multiple elements, The at least one processor performs a neural network group configuration process that constitutes a plurality of neural networks, each of which corresponds to one of the elements included in the output variable, The at least one processor performs an input layer correction process that corrects the input layer of each of the plurality of neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables, Information processing methods including

[0095] (Note B2) There is a one-to-one correspondence between the aforementioned multiple neural networks and the elements of the output variables. In the neural network group configuration process, the at least one processor configures a neural network with a number of elements included in the output variable. The information processing method described in Appendix B1.

[0096] (Note B3) In the input layer correction process, the at least one processor increases the weight values, which are elements of the input layer of each of the plurality of neural networks, as their contribution to the elements of the output variables corresponding to each neural network increases. The information processing method described in Appendix B1 or B2.

[0097] (Note B4) In the input layer correction process, the at least one processor corrects the weight values, which are elements of the input layer of each of the plurality of neural networks, using the correlation coefficient or coefficient of determination between the elements of the output variable corresponding to each neural network and the elements of the input variable. The information processing method described in Appendix B3.

[0098] (Note B5) The at least one processor performs a training process in which it trains the output layers of each of the plurality of neural networks whose input layers have been corrected by the input layer correction process using the training data. An information processing method described in any one of the appendices B1 to B4, further including the above.

[0099] (Note B6) The aforementioned multiple neural networks are used to identify the behavior of a dynamic system. The information processing method described in one of the appendices B1 through B5.

[0100] (Note B7) The aforementioned at least one processor performs a second acquisition process for acquiring input data, The at least one processor further includes an output control process that outputs output data including elements obtained by inputting the input data acquired in the second acquisition process into each of the plurality of neural networks, The information processing method described in any one of the appendices B1 to B6.

[0101] [Additional Note C] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims. (Note C1) A program that makes a computer function as an information processing device. The aforementioned computer, A means for acquiring training data that includes multiple sets of input variables containing multiple elements and output variables containing multiple elements, A neural network group configuration means comprising multiple neural networks, each of which corresponds to one of the elements included in the output variable, Input layer correction means for correcting the input layer of each of the plurality of neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables, An information processing program that functions as such.

[0102] (Note C2) There is a one-to-one correspondence between the aforementioned multiple neural networks and the elements of the output variables. The neural network group configuration means configures a neural network with a number of elements included in the output variable. The information processing program described in Appendix C1.

[0103] (Note C3) The input layer correction means increases the weight values, which are elements of the input layer of each of the plurality of neural networks, as their contribution to the elements of the output variables corresponding to each neural network increases. The information processing program described in Appendix C1 or C2.

[0104] (Note C4) The input layer correction means corrects the weight values, which are elements of the input layer of each of the plurality of neural networks, using the correlation coefficient or coefficient of determination between the elements of the output variable corresponding to each neural network and the elements of the input variable. The information processing program described in Appendix C3.

[0105] (Note C5) The aforementioned computer, Training means for training each output layer of the plurality of neural networks whose input layers have been corrected by the input layer correction means using the training data, An information processing program described in any one of the appendices C1 to C4, which further enables the function of the information processing program.

[0106] (Appendix C6) The aforementioned multiple neural networks are used to identify the behavior of a dynamic system. An information processing program described in any one of the appendices C1 to C5.

[0107] (Note C7) A second acquisition means for acquiring input data, The aforementioned computer, The second acquisition means further functions as an output control means that outputs output data including elements obtained by inputting the input data acquired by the second acquisition means into each of the plurality of neural networks. An information processing program described in any one of the appendices C1 to C6.

[0108] [Additional Note D] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims. (Note D1) It comprises at least one processor, and the at least one processor is The acquisition process involves obtaining training data that includes multiple sets of input variables containing multiple elements and output variables containing multiple elements. A neural network group configuration process that constitutes multiple neural networks, each of which corresponds to one of the elements included in the output variable, An input layer correction process that corrects the input layer of each of the plurality of neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables, An information processing device that performs the following actions. The information processing device may also include memory. Furthermore, the memory may store a program that causes at least one processor to execute each of the aforementioned processes.

[0109] (Note D2) There is a one-to-one correspondence between the aforementioned multiple neural networks and the elements of the output variables. In the neural network group configuration process, the at least one processor configures a neural network with a number of elements included in the output variable. The information processing device described in Appendix D1.

[0110] (Note D3) In the input layer correction process, the at least one processor increases the weight values, which are elements of the input layer of each of the plurality of neural networks, as their contribution to the elements of the output variables corresponding to each neural network increases. The information processing device described in Appendix D1 or D2.

[0111] (Note D4) In the input layer correction process, the at least one processor corrects the weight values, which are elements of the input layer of each of the plurality of neural networks, using the correlation coefficient or coefficient of determination between the elements of the output variable corresponding to each neural network and the elements of the input variable. The information processing device described in Appendix D3.

[0112] (Note D5) The aforementioned at least one processor, A training process in which the output layers of each of the plurality of neural networks whose input layers have been corrected by the input layer correction process are trained using the training data. An information processing device described in any one of the appendices D1 to D4 that further performs the following.

[0113] (Note D6) The aforementioned multiple neural networks are used to identify the behavior of a dynamic system. An information processing device as described in any one of the appendices D1 to D5.

[0114] (Note D7) The second acquisition process involves obtaining input data, The aforementioned at least one processor, Further, the system performs an output control process that outputs output data containing elements obtained by inputting the input data acquired in the second acquisition process into each of the plurality of neural networks. An information processing device described in any one of the appendices D1 to D6.

[0115] [Additional Note E] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims. (Note E1) A program that makes a computer function as an information processing device. To the aforementioned computer, The acquisition process involves obtaining training data that includes multiple sets of input variables containing multiple elements and output variables containing multiple elements. A neural network group configuration process that constitutes multiple neural networks, each of which corresponds to one of the elements included in the output variable, An input layer correction process that corrects the input layer of each of the plurality of neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables, A non-temporary recording medium that stores an information processing program that executes that program. [Explanation of symbols]

[0116] 1. 1A Information Processing Device 11, 11A Acquisition Department 12, 12A Neural Network Group Component 13, 14A Input Layer Correction Section 13A Correlation Calculation Unit 15A Training Department 16A 2nd acquisition part 17A Output Control Unit

Claims

1. A means for acquiring training data that includes multiple sets of input variables containing multiple elements and output variables containing multiple elements, A neural network group configuration means comprising multiple neural networks, each of which corresponds to one of the elements included in the output variable, Input layer correction means for correcting the input layer of each of the plurality of neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables, An information processing device equipped with the following features.

2. There is a one-to-one correspondence between the aforementioned multiple neural networks and the elements of the output variables. The neural network group configuration means configures a neural network with a number of elements included in the output variable. The information processing apparatus according to claim 1.

3. The input layer correction means increases the weight values, which are elements of the input layer of each of the plurality of neural networks, as their contribution to the elements of the output variables corresponding to each neural network increases. The information processing apparatus according to claim 1 or 2.

4. The input layer correction means corrects the weight values, which are elements of the input layer of each of the plurality of neural networks, using the correlation coefficient or coefficient of determination between the elements of the output variable corresponding to each neural network and the elements of the input variable. The information processing apparatus according to claim 3.

5. Training means for training the output layers of each of the plurality of neural networks whose input layers have been corrected by the input layer correction means using the training data, The information processing apparatus according to claim 1 or 2, further comprising the above.

6. The aforementioned multiple neural networks are used to identify dynamic systems. The information processing apparatus according to claim 1 or 2.

7. A second acquisition means for acquiring input data, The system further comprises an output control means that outputs output data including elements obtained by inputting the input data acquired by the second acquisition means into each of the plurality of neural networks, The information processing apparatus according to claim 1 or 2.

8. At least one processor performs an acquisition process to acquire training data that includes multiple sets of input variables containing multiple elements and output variables containing multiple elements, The at least one processor performs a neural network group configuration process that constitutes a plurality of neural networks, each of which corresponds to one of the elements included in the output variable, The at least one processor performs an input layer correction process that corrects the input layer of each of the plurality of neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables, Information processing methods including

9. A program for causing a computer to function as an information processing device, wherein the computer, A means for acquiring training data that includes multiple sets of input variables containing multiple elements and output variables containing multiple elements, A neural network group configuration means comprising multiple neural networks, each of which corresponds to one of the elements included in the output variable, Input layer correction means for correcting the input layer of each of the plurality of neural networks based on the correlation between the elements of the output variables corresponding to each neural network and the input variables, An information processing program designed to function as such.

Images