Reservoir computing device, reservoir computing program, and reservoir utilization calculation method

The reservoir calculation device and program address the challenge of maintaining storage capacity by using an input layer with nodes that reflect and apply signals, ensuring effective information retention and performance even with increased nonlinear conversion characteristics.

WO2024247613A1PCT designated stage expired Publication Date: 2024-12-05NAT UNIV CORP KYUSHU INST OF TECH (JP)
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/016884
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-30
Filing Date
2024-05-02
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Reservoir calculation models face challenges in maintaining a stable storage capacity as nonlinear conversion characteristics increase, making it difficult to achieve both high storage capacity and effective information retention.

Method used

The reservoir calculation device and program incorporate an input layer with multiple nodes that reflect and apply signals from other nodes, allowing past information to influence current outputs, thereby stabilizing storage capacity even with increased nonlinear conversion characteristics.

Benefits of technology

This approach ensures a stable storage capacity and improved information retention capabilities, enhancing the performance of reservoir calculation models in processing time-series information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024016884_05122024_PF_FP_ABST
    Figure JP2024016884_05122024_PF_FP_ABST
Patent Text Reader

Abstract

A reservoir computing device 10 is provided with an input layer 11, a reservoir layer 12 that outputs a signal reflecting signals provided from the input layer 11, and an output layer 13 for deriving an output value on the basis of a signal output from the reservoir layer 12, wherein: the input layer 11 has a plurality of nodes A (nodes 14 to 14f); and at least one node A is provided with a signal reflecting a signal output from another node A in one step, and provides a signal reflecting the provided signal to the reservoir layer in a subsequent step.
Need to check novelty before this filing date? Find Prior Art

Description

Reservoir calculation device, reservoir calculation program, and reservoir utilization calculation method

[0001] The present invention relates to a reservoir calculation device and a reservoir calculation program that perform information processing based on a reservoir calculation model, and a reservoir utilization calculation method.

[0002] In a recurrent neural network with recurrent connections, a reservoir computation model is known as a machine learning computation model suitable for processing time-series information (see Patent Document 1). The reservoir computation model includes a reservoir layer having a plurality of interconnected nodes and an output layer that derives an output value based on information reflecting the state of each node (neuron) in the reservoir layer, and adjusts the connection weights between the reservoir layer and the output layer so that the output value approaches a target value.

[0003] The state of each node in the reservoir layer (hereinafter, the nodes in the reservoir layer will also be simply referred to as "nodes") changes depending on the information input to the reservoir layer and the information provided by other nodes. The information provided by each node to other nodes is based on the state of that node, which reflects the information previously input to the reservoir layer and the information previously provided to the node by other nodes. Therefore, the reservoir layer holds past information. The extent to which the reservoir computational model can hold past information is evaluated as the size of the storage capacity of the reservoir layer.

[0004] Patent No. 7028377

[0005] Suzuki, H., Imura, Ji., Horio, Y. et al. Chaotic Boltzmann machines. Sci Rep 3, 1610 (2013). https: / / doi.org / 10.1038 / srep01610Y. Katori, H. Tamukoh and T. Morie, "Reservoir Computing Based on Dynamics of Pseudo-Billiard System in Hypercube," 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, 2019, pp. 1-8, doi: 10.1109 / IJCNN.2019.8852329. Uemura, D., Yamaguchi, M., Katori, Y., Tamuko, G., Morie, T., Application of CMOS Chaotic Boltzmann Machine Circuits to Reservoir Computation, Institute of Electrical Engineers of Japan, Technical Meeting on Electronic Circuits, ECT-19-089, December 18-19(18), 2019, Nihon University (Tokyo). Kohei Tanaka, "Reservoir Computing: Concepts and Recent Trends," Journal of the Institute of Electronics, Information and Communication Engineers, Vol. 102, February 1, 2019. Tomoyuki Kubota, Kohei Nakajima, and Hirokazu Takahashi. Dynamical anatomy of the narma10 benchmark task. arXiv preprint arXiv:1906.04608, 2019.

[0006] Here, factors for evaluating the reservoir performance of a reservoir calculation model include the storage capacity of the reservoir layer and the nonlinear conversion characteristics of the reservoir layer. Since the storage capacity of the reservoir layer tends to decrease as the nonlinear conversion characteristics of the reservoir layer increase, the challenge for the reservoir calculation model was to ensure that both the nonlinear conversion characteristics and storage capacity of the reservoir layer were above a certain level.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide a reservoir calculation device, a reservoir calculation program, and a reservoir utilization calculation method that can stably provide a certain level of memory capacity even when the nonlinear conversion characteristics are increased.

[0008] A reservoir computing device according to a first invention that meets the above objective comprises an input layer, a reservoir layer that outputs a signal that reflects a signal provided from the input layer, and an output layer that derives an output value based on the signal output from the reservoir layer, wherein the input layer has a plurality of nodes A, and at least one of the nodes A is provided with a signal that reflects a signal output from another node A in one step, and provides a signal that reflects the provided signal to the reservoir layer in a subsequent step.

[0009] A reservoir calculation program according to a second invention that meets the above-mentioned objective causes an information processing terminal that includes input means having a plurality of nodes A, reservoir means that outputs a signal that reflects a signal given from the input means, and output means that derives an output value based on the signal output from the reservoir means to perform processing including a step in which at least one of the nodes A is given a signal that reflects a signal output from another of the nodes A, and a step in which the node A that has received a signal gives a signal that reflects the given signal to the reservoir means.

[0010] A reservoir utilization calculation method according to a third aspect of the present invention, which achieves the above object, derives an output value based on an externally input signal, and includes the steps of: a first node receiving an Mth external input signal from the first node, and outputting the Mth input signal from the first node to a reservoir calculation unit; and a second node receiving an M+1th external input signal from the first node, and outputting the M+1th input signal from the first node to the second node, and further providing a signal reflecting the Mth input signal to the reservoir calculation unit.

[0011] According to the reservoir computing device of the first invention, the input layer has a plurality of nodes A, and at least one node A is given a signal that reflects a signal output from another node A in one step, and then gives a signal that reflects the given signal to the reservoir layer in a subsequent step, so that even if the nonlinear transformation characteristics are enhanced, it is possible to stably provide a certain level of memory capacity or more.

[0012] Furthermore, according to the reservoir calculation program of the second invention, an information processing terminal having an input means having a plurality of nodes A, a reservoir means for outputting a signal reflecting a signal given from the input means, and an output means for deriving an output value based on the signal output from the reservoir means is made to perform processing including a step in which at least one node A is given a signal reflecting a signal output from another node A, and a step in which the node A given a signal gives a signal reflecting the given signal to the reservoir means, so that even if the nonlinear conversion characteristic is enhanced, a certain level of memory capacity or more can be stably provided.

[0013] The reservoir utilization calculation method according to the third aspect of the present invention includes the steps of: the first node receiving an Mth input signal from an external device and providing the Mth input signal to a reservoir calculation unit; and the first node outputting the Mth input signal to a second node; the first node receiving an M+1th input signal from an external device and providing the M+1th input signal to a reservoir calculation unit; and the second node outputting a signal reflecting the Mth input signal to the reservoir calculation unit. Therefore, even if the nonlinear conversion characteristics are enhanced, a certain level of storage capacity or more can be stably provided.

[0014] FIG. 1 is an explanatory diagram of a reservoir calculation device according to a first embodiment of the present invention. FIG. 2 is an explanatory diagram of a reservoir calculation device according to a second embodiment of the present invention. FIG. 3 is an explanatory diagram of a modified example of a reservoir calculation device according to the second embodiment of the present invention. FIG. 4 is an explanatory diagram of a reservoir calculation device according to a third embodiment of the present invention. FIG. 5 is an explanatory diagram of a reservoir calculation device according to a fourth embodiment of the present invention. FIG. 6 is an explanatory diagram showing, in a graph, measurement results of the memory capacity of each model related to a chaos Boltzmann machine. FIG. 7 is an explanatory diagram showing, in a graph, measurement results of the NMSE of each model related to a chaos Boltzmann machine. FIG. 8 is an explanatory diagram showing, in a graph, measurement results of the memory capacity of each model related to an echo state network. FIG. 9 is an explanatory diagram showing, in a graph, measurement results of the NMSE of each model related to an echo state network. FIG. 10 is an explanatory diagram of an example of combining reservoir calculation devices according to the second and third embodiments of the present invention.

[0015] Next, with reference to the accompanying drawings, a description will be given of specific embodiments of the present invention to facilitate understanding of the present invention. As shown in Fig. 1, a reservoir calculation device 10 according to a first embodiment of the present invention is a device comprising an input layer (input unit) 11, a reservoir layer (reservoir calculation unit) 12 that outputs a signal reflecting a signal provided from the input layer 11, and an output layer (output unit) 13 that derives an output value based on the signal output from the reservoir layer 12. The details will be described below.

[0016] As shown in Fig. 1, the reservoir computing device 10 has an input layer 11 with multiple (seven in this embodiment) nodes 14, 14a, 14b, 14c, 14d, 14e, and 14f (examples of node A), a reservoir layer 12 with multiple nodes 15 (examples of node B), and an output layer 13 with multiple nodes 16 (although Fig. 1 shows the output layer 13 as having only one node 16, in reality, there are multiple nodes 16). The nodes 14, 14a, 14b, 14c, 14d, 14e, and 14f are capable of acquiring signals, storing the acquired signals, and outputting signals, respectively.

[0017] In this embodiment, the nodes 14, 14a, 14b, 14c, 14d, 14e, and 14f perform signal processing at a predetermined cycle T (time interval T). Specifically, in the Nth step, the node 14 (the same applies to the nodes 14a, 14b, 14c, 14d, 14e, and 14f) receives (acquires) a new signal, stores that signal, and outputs, in the Nth step, the signal that was received and stored in the (N-1)th (previous) step, where N is a natural number.

[0018] The input layer 11 is designed so that, in a period T, node 14 receives a signal from the outside and outputs the signal toward node 14a and reservoir layer 12, node 14a outputs a signal toward node 14b and reservoir layer 12, node 14c outputs a signal toward node 14d and reservoir layer 12, node 14d outputs a signal toward node 14e and reservoir layer 12, and node 14e outputs a signal toward node 14f and reservoir layer 12.

[0019] Therefore, after the node 14 receives a signal from the outside in the first step, the input layer 11 performs the following processing.

[0020] Second step: A new signal is given to the node 14 from the outside, and the signal given from the outside in the first step is output to the node 14 a and the reservoir layer 12 .

[0021] Third step: Node 14 receives a new signal from the outside, and outputs the signal received from the outside in the second step to node 14a and reservoir layer 12. Node 14a outputs the signal received from node 14 in the second step to node 14b and reservoir layer 12.

[0022] Fourth step: Node 14 receives a new signal from the outside, and in the third step outputs the externally applied signal to node 14a and reservoir layer 12. In the third step, node 14a outputs the signal applied from node 14 to node 14b and reservoir layer 12. In the third step, node 14b outputs the signal applied from node 14a to node 14c and reservoir layer 12.

[0023] Similar processing is performed from the fifth step onwards, with node 14c outputting the signal given from node 14b in the previous step to node 14d and the reservoir layer 12, node 14d outputting the signal given from node 14c in the previous step to node 14e and the reservoir layer 12, node 14e outputting the signal given from node 14d in the previous step to node 14f and the reservoir layer 12, and node 14f outputting the signal given from node 14e in the previous step to the reservoir layer 12.

[0024] Here, it is not necessary for the nodes 14, 14a, 14b, 14c, 14d, 14e, and 14f to output signals at the same timing. For example, the nodes 14, 14a, 14b, and 14c can be designed to output signals given in the previous step with a period of T, and the nodes 14d, 14e, and 14f can be designed to output signals given in the step two steps before with a period of 2T.

[0025] Furthermore, among all the nodes A in the input layer, multiple nodes A may be configured to receive signals from the outside, in which case the nodes A other than the node A receiving the signal from the outside receive signals from other nodes A. Furthermore, it may be possible to output a signal from one node A to multiple nodes A, or for a node A to output a signal that reflects a previously received signal, such as a signal obtained by multiplying a received signal by a constant, to other nodes A.

[0026] Therefore, at least one node A receives a signal that reflects a signal output from another node A in one step, and provides the received signal itself or a signal converted from the received signal, i.e., a signal that reflects the received signal, to the reservoir layer in a subsequent step. Therefore, the input layer can provide to the reservoir layer a signal that has been externally provided to the input layer (node ​​14 in this embodiment) over multiple past steps, or a signal that reflects a signal generated in the input layer over multiple past steps.

[0027] In this embodiment, signals output from the nodes 14, 14a, 14b, 14c, 14d, 14e, and 14f toward the reservoir layer 12 are provided to all or some of the nodes 15. Each node 15 in the reservoir layer 12 acquires a signal, changes its state, and outputs the signal toward one or more targets with a period T. The sources of signals provided to each node 15 are one or more of the nodes 14, 14a, 14b, 14c, 14d, 14e, and 14f, the node 15 itself, and other nodes 15, and the destinations of the signals from each node 15 are the node 15 itself, other nodes 15, and the output layer 13.

[0028] Each node 15 receives a new signal in the Nth step, changes its state depending on the new signal and the state of the node 15 before the new signal was received, and outputs a signal corresponding to the changed state in the N+1th step. Therefore, the reservoir layer 12 is affected by the signals received from the input layer 11 over the past several steps. As described above, in this embodiment, not only the reservoir layer 12 but also the input layer 11 retains past signals (information). Therefore, even if the nonlinear transformation characteristics of the reservoir layer 12 are enhanced, the reservoir computing device 10 as a whole can stably maintain a certain level of storage capacity or more.

[0029] Furthermore, the signal output from node 15 (reservoir layer 12) to output layer 13 is multiplied by the corresponding connection weight and then provided to output layer 13. Note that while FIG. 1 shows a single path for providing a signal from reservoir layer 12 to output layer 13, in reality there are multiple paths, and the system is designed so that each path corresponds to a separate connection weight. Note that there may be only one path for providing a signal from reservoir layer 12 to output layer 13.

[0030] The output layer 13 receives a plurality of signals from the reservoir layer 12 every period T, and derives an output value based on the received signals. In this embodiment, the output layer 13 derives an output value by adding the plurality of signals (i.e., signals obtained by multiplying the signals output from the nodes 15 by connection weights) received from the reservoir layer 12. Therefore, the reservoir computing device 10 derives an output value by performing a product-sum operation on the signals output from the reservoir layer 12. During training, the derived output value is compared with a target value, and the magnitude of the connection weight corresponding to each signal output from the reservoir layer 12 is adjusted so that the output value approaches the target value. Here, for example, if the task executed by the reservoir computing device 10 is a prediction task, the output value is a predicted value.

[0031] Therefore, a reservoir utilization calculation method using the reservoir calculation device 10 to derive an output value based on an externally input signal includes the steps of: providing an Mth input signal, input from an external source to a first node (node ​​14), from the first node to a reservoir calculation unit (reservoir layer 12), and outputting the Mth input signal from the first node to a second node (node ​​14a); providing an M+1th input signal, input from an external source to the first node, from the first node to the reservoir calculation unit, and outputting the M+1th input signal from the first node to the second node, and further providing a signal reflecting the Mth input signal from the second node to the reservoir calculation unit, where M is a natural number.

[0032] Next, a reservoir calculation device 20 according to a second embodiment of the present invention will be described with reference to Fig. 2. As shown in Fig. 2, the reservoir calculation device 20 includes an input layer (input unit) 21, a reservoir layer (reservoir calculation unit) 22 that outputs a signal reflecting a signal provided from the input layer 21, and an output layer (output unit) 23 that derives an output value based on the signal output from the reservoir layer 22.

[0033] The input layer 21 is made up of a plurality of groups (two in this embodiment), and the reservoir layer 22 is also made up of a plurality of groups (two in this embodiment). In this embodiment, the number of groups in the input layer 21 is equal to the number of groups in the reservoir layer 22. The input means 21a, which corresponds to the first group of the input layer 21, has a plurality of nodes 24, 24a, and 24b, and the input means 21b, which corresponds to the second group of the input layer 21, has a plurality of nodes 24c, 24d, and 24e. In this embodiment, the number of nodes in each input means 21a and 21b of the input layer 21 is equal, but may be different, as shown in FIG. 3 .

[0034] The reservoir means 22a corresponding to the first group of the reservoir layer 22 has a plurality of nodes 26, and the reservoir means 22b corresponding to the second group of the reservoir layer 22 has a plurality of nodes 27. The output layer 23 has one node 28. The number of nodes 26 and the number of nodes 27 may be the same or different. The output layer 23 may have a plurality of nodes 28.

[0035] An external signal is applied to node 24, which outputs the signal toward node 24a and reservoir means 22a, node 24a outputs the signal toward node 24b and reservoir means 22a, node 24b outputs the signal toward node 24c and reservoir means 22a, node 24c outputs the signal toward node 24d and reservoir means 22b, node 24d outputs the signal toward node 24e and reservoir means 22b, and node 24e outputs the signal toward reservoir means 22b.

[0036] In the reservoir means 22a, a plurality of nodes 26 behave similarly to the plurality of nodes 15 of the input layer 12 in the reservoir computing device 10, and output signals toward nodes 28 of the output layer 23. In the reservoir means 22b, a plurality of nodes 27 behave similarly to the plurality of nodes 15 of the input layer 12 in the reservoir computing device 10, and output signals toward nodes 28 of the output layer 23.

[0037] Therefore, in this embodiment, each group in the reservoir layer 22 receives signals only from a specific (corresponding) group in the input layer 21, and each group in the reservoir layer 22 outputs a signal to a common node 28 in the output layer 23. The output layer 23 (node ​​28) derives an output value based on the signals received from the reservoir means 22a and 22b. In this embodiment, the signals from the nodes 24, 24a, 24b, 24c, 24d, 24e, 26, and 27 are all output with a period T, but they may be output with different periods. For example, only the signal output from node 24b to node 24c, the signal output from node 24c to node 24d, and the signal output from node 24d to node 24e can be output with a period 2T, which is twice the period T, and the other signal outputs can be output with a period T.

[0038] Next, a reservoir calculation device 30 according to a third embodiment of the present invention will be described with reference to Fig. 4. As shown in Fig. 4, the reservoir calculation device 30 includes an input layer (input unit) 31, a reservoir layer (reservoir calculation unit) 32 that outputs a signal reflecting a signal provided from the input layer 31, and an output layer (output unit) 33 that derives an output value based on the signal output from the reservoir layer 32.

[0039] The input layer 31 has a plurality of (two in this embodiment) nodes 34, 34a (each an example of node A). The reservoir layer 32 has a reservoir means 36 having a plurality of nodes 35 (an example of node B) and nodes 37, 37a (each an example of node C) to which signals are directly applied from the nodes 34, 34a, respectively. The node 37 is not connected to another node 37a in the reservoir layer 32, and the node 37a is not connected to another node 37 in the reservoir layer 32.

[0040] The output layer 33 has one node 38. The reservoir means 36 performs the same processing as the reservoir means 22 or 22a of the reservoir computing device 20 and outputs a signal (one or more signals) to the output layer 33. The reservoir computing device 30 performs the following processing steps.

[0041] First step: A signal is applied to node 34 from the outside.

[0042] Second step: A new signal is given to node 34 from the outside, and the node 34 outputs the signal given from the outside in the first step to node 34a, reservoir means 36, and node 37. The signal output by node 34 may be given directly to node 37, or the signal output by node 34 may be multiplied by a coupling weight and then given to node 37. This also applies to the relationship between node 34a and node 37a.

[0043] Third step: Node 34 receives a new signal from the outside, and outputs the signal received from the outside in the second step toward node 34a, reservoir means 36, and node 37. Node 34a outputs the signal received from node 34 in the second step toward reservoir means 36 and node 37a. Node 37 outputs the signal received from node 34 in the second step toward output layer 33.

[0044] Fourth step: Node 34 receives a new signal from the outside, and outputs the signal received from the outside in the third step toward node 34a, reservoir means 36, and node 37. Node 34a outputs the signal received from node 34 in the third step toward reservoir means 36 and node 37a. Node 37 outputs the signal received from node 34 in the third step toward output layer 33. Node 37a outputs the signal received from node 34 in the third step (i.e., the signal received from node 34 to node 34a in the second step) toward output layer 33.

[0045] In the fourth step and subsequent steps, when the reservoir means 36 starts to output signals to the output layer 33, the signals output from the nodes 37 and 37a and the signal output from the reservoir means 36 in those steps are multiplied by the corresponding connection weights and provided to the output layer 33. Then, the output layer 33 derives an output value based on the signal provided to the output layer 33 by the node 38. As in the reservoir calculation device 10, each connection weight is adjusted by comparing the output value with a target value during learning.

[0046] Therefore, the reservoir layer 32 is provided with a node C (nodes 37 and 37a in this embodiment) that receives a signal from node A (nodes 34 and 34a in this embodiment) in one step and outputs a signal reflecting the received signal (which is the received signal itself in this embodiment, but may be a signal obtained by multiplying the received signal by a predetermined numerical value) to the output layer 33 in a subsequent step. Therefore, in the reservoir computing device 30, not only the reservoir means 36 but also the nodes 37 and 37a have the function of holding a signal reflecting a signal previously received by node 34 from the outside (or a signal previously generated by node 34).

[0047] Next, a reservoir calculation device 40 according to a fourth embodiment of the present invention will be described with reference to Fig. 5. As shown in Fig. 5, the reservoir calculation device 40 includes an input layer (input unit) 41, a reservoir layer (reservoir calculation unit) 42 that outputs a signal reflecting a signal provided from the input layer 41, and an output layer (output unit) 43 that derives an output value based on the signal output from the reservoir layer 42.

[0048] The input layer 41, reservoir layer 42, and output layer 43 each consist of multiple groups (two in this embodiment). In this embodiment, the number of groups in the input layer 41, the number of groups in the reservoir layer 42, and the number of groups in the output layer 43 are equal. The input means 41a constituting the first group of the input layer 41 has multiple nodes 44, 44a, and the input means 41b constituting the second group of the input layer 41 has multiple nodes 44b, 44c, 44d, and 44e. Therefore, the input means 41b has a larger number of nodes and can hold a larger amount of signals (number of signals) than the input means 41a.

[0049] Reservoir means 42a constituting the first group (group Q) of reservoir layer 42 has a plurality of nodes 46, and reservoir means 42b constituting the second group (group P) of reservoir layer 42 has a plurality of nodes 47. The number of nodes 47 is greater than the number of nodes 46, and reservoir means 42b can perform more complex calculations than reservoir means 42a. A signal is given to reservoir means 42a from input means 41a, and a signal is given to reservoir means 42b from input means 41b.

[0050] The output means 43a constituting the first group of the output layer 43 has one (or more) node 48, and the output means 43b constituting the second group of the output layer 43 has one (or more) node 49. The output means 43a corresponds to the reservoir means 42a and derives an output value based on a signal provided from the reservoir means 42a. The output means 43b corresponds to the reservoir means 42b and derives an output value based on a signal provided from the reservoir means 42b.

[0051] The reservoir means 42a is normally in an activated state where various processes (including a process of receiving a signal from the input means 41a and outputting the signal to the output means 43a) are possible, regardless of whether a signal is given from the input means 41a. In contrast, the reservoir means 42b is activated by receiving a predetermined signal from the activation means 50 connected to the output means 43a, where it is in an activated state where various processes (including a process of receiving a signal from the input means 41b and outputting the signal to the output means 43b) are possible, and is switched from the activated state to a sleep state where various processes (including a process of receiving a signal from the input means 41b and outputting the signal to the output means 43b) are not performed under predetermined conditions.

[0052] The condition for the reservoir means 42b to switch from the active state to the sleep state is, for example, that no signal is input from the input means 41b to the reservoir means 42b for a predetermined time (period) or that a predetermined signal is given to the reservoir means 42b. The output means 43a derives a predetermined output value, and the activation means 50 is given that output value, which causes the activation means 50 to emit a signal to activate the reservoir means 42b in the sleep state. In other words, the reservoir means 42b in the sleep state is switched to the active state when the output means 43a, which has received a signal from the reservoir means 42a, derives a predetermined output value.

[0053] In addition, in the activated reservoir means 42b, based on the signals output from nodes 44b to 44e toward the reservoir means 42b, each node 47 behaves in the same manner as the multiple nodes 15 of the reservoir layer 12 in the reservoir computing device 10, and outputs a signal toward the output means 43b.

[0054] In this embodiment, the entire reservoir calculation device 40 including the reservoir means 42b consumes power for various processes, and the amount of power consumed by the reservoir means 42b is greater when it is in an active state than when it is in a sleep state. Therefore, in principle, the reservoir calculation device 40 is preferable from the viewpoint of energy saving compared to a reservoir calculation device in which all of the reservoir means are always active.

[0055] Moreover, in this embodiment, the task of the reservoir means 42a is limited to only activating the reservoir means 42b in a sleep state. The task of the reservoir means 42b is more complex than that of the reservoir means 42a, and the reservoir means 42b has a greater number of nodes than the reservoir means 42a, so the power consumption of the reservoir means 42b in an activated state is greater than that of the reservoir means 42a.

[0056] In this respect, the reservoir computing device 40 has a further advantageous design from the viewpoint of energy saving. To wake up the reservoir means 42b in a sleep state, a signal given from the outside to the reservoir computing device 40 (specifically, the node 44) may be, for example, a signal obtained by converting a predetermined character string into a voice signal or a signal obtained by operating an input device.

[0057] The reservoir computation devices 10, 20, 30, and 40 described so far are designed to perform processing based on the chaotic Boltzmann machine described in Non-Patent Documents 1, 2, and 3. The chaotic Boltzmann machine is a network of nonlinear oscillators based on billiard dynamics, and its application to reservoir computation devices has been proposed. However, this is not limiting. For example, the reservoir computation device may be designed to perform processing based on the echo state network or liquid state machine described in Non-Patent Document 4. The echo state network and liquid state machine are known as representative models for realizing reservoir computation devices.

[0058] Furthermore, the reservoir computing devices 10, 20, 30, and 40 are configured by a computer (an example of an information processing terminal) as hardware and a software program installed on the computer, but the reservoir computing devices can also be configured (embodied as a physical reservoir) by physical implementation (analog circuits, semiconductor optical amplifiers, cellular automata, nanoscale materials, etc.). The signals used in the reservoir computing devices 10, 20, 30, and 40 may be analog signals or digital signals.

[0059] When a reservoir calculation device is configured using a computer and a software program, the software program (reservoir calculation program) causes the computer to function as input means having a plurality of nodes A, reservoir means that outputs a signal reflecting the signal given from the input means, and output means that derives an output value based on the signal output from the reservoir means, in order to derive an output value based on a given signal, and causes the computer to perform processing including a step in which at least one node A receives a signal reflecting the signal output from another node A, and a step in which the node A that has received the signal gives a signal reflecting the given signal to the reservoir means.

[0060] Furthermore, the software program may also cause the computer to execute processing in which there are a plurality of input means and a plurality of reservoir means, and each of the plurality of reservoir means receives a signal only from a specific input means.

[0061] The software program has a plurality of reservoir means, including reservoir means p that switches from an active state in which a signal is received from an input means and outputs a signal to an output means under predetermined conditions to a sleep state in which no signal is output even when a signal is received from the input means, and reservoir means q that is normally in an active state, and the computer can execute processing to switch the reservoir means p in a sleep state to an active state by having the output means that receives a signal from the reservoir means q derive a predetermined output value.

[0062] The software program can also cause the computer to function as node C, which receives a signal directly from node A, and cause node C to output a signal that reflects the signal received by node C to an output means, so that the source of signals to node C is node A only, and the destination of signals from node C is the output means only. Experimental example

[0063] Next, two experiments using computer simulations conducted to confirm the effects of the present invention will be described.

[0064] In the first experiment, the memory capacity and normalized mean square error (NMSE), which are evaluation values ​​of how long input information can be stored, were measured for a reservoir computing model (Comparative Example 1) consisting of a chaos Boltzmann machine and reservoir computing models (Experimental Examples 1 to 4) that apply a chaos Boltzmann machine, as shown in Table 1 below.

[0065]

[0066] Furthermore, the storage capacity and NMSE were measured for the echo state network model (Comparative Example 2) and models applying the echo state network model (Experimental Examples 5 to 8) shown in Table 2 below.

[0067]

[0068] In the model types in Table 1, CBM means a reservoir computing model that applies a chaotic Boltzmann machine, and in the model types in Table 2, ESN means an echo state network.

[0069] The input waveform, target waveform, and predicted waveform were as follows, and 1000 of the total time length of 1500 was used as the learning time, and the remaining time length of 500 was used as the measurement time for memory capacity and NMSE. During the measurement time for memory capacity and NMSE, the memory capacity and NMSE were derived by ridge regression of the predicted waveform portion output from the reservoir computing model and the target waveform portion corresponding to that predicted waveform.

[0070] Input waveform: A uniform random number sequence ranging from 0.0 to 0.2 was generated for a time period of 1500 minutes, and this was used as the input waveform to be input to the target reservoir computing model.

[0071] Target waveform: The waveform generated by applying the uniform random number sequence generated to generate the input waveform to the function of the following formula 1 was used as the target waveform.

[0072]

[0073] In Equation 1, y is the target waveform and u is the input waveform. T represents the maximum number of steps that indicates how far back past steps can be reflected, and in this experiment, T=10.

[0074] Predicted waveform: The input waveform was input into the reservoir computing model, and the output waveform was used as the predicted waveform.

[0075] Furthermore, for each reservoir computing model, the connection weight value Win between the input layer and the reservoir layer, the connection weight value Wrec between nodes within the reservoir layer, and the connection weight value Wout between the reservoir layer and the output layer were set as follows:

[0076] Win: A matrix was generated using uniform random numbers between -1 and 1, and this was multiplied by αi to obtain Win. Wrec: Three values, -1, 0, and 1, were randomly placed in the matrix with a density of βr, and the spectral radius was measured to obtain Wrec. Wout: Wout before learning was set to be equal to Win. Wout after learning was set to the value obtained by ridge regression.

[0077] The process from generating the input waveform to deriving the memory capacity and NMSE was repeated 10 times, and the average of the 10 derived memory capacities and NMSE values ​​was used as the final memory capacity and NMSE. The number of nodes in the input layer was 1, the number of nodes in the reservoir layer was 200, and the number of nodes in the output layer was 1. Furthermore, using the Simplex method, which is a method for solving linear programming problems, each parameter of the reservoir computing model was adjusted with the goal of achieving NARMA10. As explained in Non-Patent Document 5, NARMA10 is a task for predicting future values ​​of time-series data using a Nonlinear Autoregressive Moving Average (NARMA) model, and is used as a performance evaluation task for the reservoir computing model.

[0078] The measurement results of the memory capacity and NMSE for the reservoir computing models related to the chaos Boltzmann machine (Comparative Example 1 and Experimental Examples 1 to 4) are shown in Figures 6 and 7, respectively. The measurement results of the memory capacity and NMSE for the models related to the ESN (Comparative Example 2 and Experimental Examples 5 to 8) are shown in Figures 8 and 9, respectively. The Delay on the horizontal axis in Figures 6 to 9 corresponds to T in Equation 1. The measurement results show that Experimental Examples 1 to 8 showed significant improvements in the memory capacity and NMSE values ​​(larger memory capacity and smaller NMSE).

[0079] In the second experiment, a reservoir computing model was prepared in which the reservoir layer had two groups: Group P (a group that switches between awake and sleep states) and Group Q (a group that only performs the task of waking up Group P). Both Group P and Group Q had 200 nodes, and the input layer had a total of four nodes: two nodes that send signals to Group P and two nodes that send signals to Group Q.

[0080] Speech data for 10 numbers from 0 to 9 by six speakers was prepared, with 30 numbers for each number, for a total of 300 pieces of data. Each piece of speech data was preprocessed using the Lyon Auditory Model, and the speech data was converted into a 64-dimensional, 100-step signal, which was used as test data. As a result, 300 pieces of test data were prepared.

[0081] For group Q of the reservoir layer (hereinafter simply referred to as "group Q"), group P of the reservoir layer (hereinafter simply referred to as "group P") was set to be activated only when the test data given to the input layer was a specific digit (for example, the digit 0). Group P was assigned the task of speaker discrimination.

[0082] The number (hereinafter referred to as the "wake word") that activates Group P was changed from 0 to 9, and an experiment was conducted using 300 test data for each wake word. Since the 300 test data included 30 wake words, ideally Group P should only be activated when given 30 test data for the corresponding wake word. The experimental results were as follows:

[0083] Of the 300 times when the test data was the wake word, group Q activated group P 266 times. Of the 2,700 times when the test data was not the wake word, group Q erroneously activated group P 34 times. Of the 300 times when group P was activated, the speaker was correctly identified 283 times.

[0084] While the present invention has been described above by way of example, it is not limited to the above-described embodiments. Modifications to the conditions that do not depart from the spirit of the invention are within the scope of the present invention. For example, the output layer may have multiple nodes. It is also possible to combine the reservoir calculation device according to the second embodiment with the reservoir calculation device according to the third embodiment. Specifically, as shown in FIG. 10 , the input layer may be composed of multiple groups, and the reservoir layer may be composed of multiple groups (large circles centered horizontally in FIG. 10 ) each receiving signals from only specific groups in the input layer. Furthermore, a node C (small circle centered horizontally in FIG. 10 ) may be provided in the reservoir layer, separate from the multiple groups in the reservoir layer.

[0085] The reservoir calculation device and reservoir calculation program according to the present invention can realize a high-performance reservoir calculation model in which the nonlinear transformation characteristics and memory capacity of the reservoir layer are both above a certain level, thereby enabling the compactization of chips and electronic circuits that perform complex identification and prediction processes.

[0086] 10: reservoir calculation device, 11: input layer, 12: reservoir layer, 13: output layer, 14, 14a, 14b, 14c, 14d, 14e, 14f, 15, 16: nodes, 20: reservoir calculation device, 21: input layer, 21a, 21b: input means, 22: reservoir layer, 22a, 22b: reservoir means, 23: output layer, 24, 24a, 24b, 24c, 24d, 24e, 26, 27, 28: nodes, 30: reservoir calculation device, 31: input layer, 32: reservoir layer, 33: output layer, 34, 34a, 35: nodes, 36: reservoir means, 37, 37a, 38: nodes, 40: reservoir computing device, 41: input layer, 41a, 41b: input means, 42: reservoir layer, 42a, 42b: reservoir means, 43: output layer, 43a, 43b: output means, 44, 44a, 44b, 44c, 44d, 44e, 46, 47, 48, 49: nodes, 50: activation means

Claims

1. A reservoir computing device comprising an input layer, a reservoir layer which outputs a signal reflecting a signal provided from the input layer, and an output layer which derives an output value based on the signal output from the reservoir layer, wherein the input layer has a plurality of nodes A, and at least one of the nodes A is provided with a signal reflecting a signal output from another node A in one step, and provides a signal reflecting the provided signal to the reservoir layer in a subsequent step.

2. A reservoir computing device according to claim 1, wherein said input layer and said reservoir layer each comprise a plurality of groups, and each group of said reservoir layer is provided with signals only from a specific group of said input layer.

3. A reservoir computing device according to claim 2, wherein the multiple groups in the reservoir layer include a group P which switches from an active state in which a signal is given from the input layer and the output layer outputs a signal to the output layer under predetermined conditions to a sleep state in which no signal is output even when a signal is given from the input layer, and a group Q which is normally in the active state, and wherein the group P in the sleep state switches to the active state when the output layer, to which a signal is output from group Q, derives a predetermined output value.

4. A reservoir computing device according to any one of claims 1 to 3, characterized in that the reservoir layer comprises a node C which receives a signal directly from the node A in one step and outputs a signal reflecting the received signal to the output layer in a subsequent step, the node C being unconnected to other nodes in the reservoir layer.

5. A reservoir calculation device according to claim 1, characterized in that a multiply-and-accumulate process is performed on the signal output from said reservoir layer.

6. The reservoir computing device according to claim 1, characterized in that said reservoir computing device performs processing based on a chaos Boltzmann machine.

7. A reservoir calculation program for causing an information processing terminal to function as an input means having a plurality of nodes A, a reservoir means for outputting a signal reflecting the signal given from the input means, and an output means for deriving the output value based on the signal output from the reservoir means, in order to derive an output value based on a given signal, and for having the information processing terminal perform processing including a step of at least one of the nodes A being given a signal reflecting a signal output from another of the nodes A, and a step of the node A given a signal giving a signal reflecting the given signal to the reservoir means.

8. A reservoir calculation program as claimed in claim 7, characterized in that there are a plurality of said input means and said reservoir means, and said information processing terminal is caused to execute processing in which each of said plurality of reservoir means receives a signal only from a specific one of said input means.

9. A reservoir calculation program as claimed in claim 8, wherein said plurality of reservoir means include reservoir means p which switches from an active state in which a signal is given from said input means to said output means and outputs a signal to said output means under predetermined conditions to a sleep state in which no signal is output even when a signal is given from said input means, and reservoir means q which is normally in said active state, wherein said information processing terminal is caused to execute a process in which said reservoir means p in said sleep state is switched to said active state by said output means receiving a signal from said reservoir means q and deriving a predetermined output value.

10. A reservoir calculation program as claimed in any one of claims 7 to 9, characterized in that said information processing terminal is caused to function as node C to receive a signal directly from node A, and said information processing terminal is caused to perform a process in which node C outputs a signal reflecting the signal received by node C to said output means, so that the source of signals to node C is limited to node A, and the destination of signals from node C is limited to the output means.

11. A reservoir utilization calculation method for deriving an output value based on an input signal input from an external source, comprising the steps of: a step of the first node providing the Mth input signal input from an external source to a reservoir calculation unit, and the first node outputting the Mth input signal to a second node; a step of the first node providing the M+1th input signal input from an external source to the first node to the reservoir calculation unit, and the first node outputting the M+1th input signal to the second node, and further, a step of the second node providing a signal reflecting the Mth input signal to the reservoir calculation unit.

Citation Information

Patent Citations

  • Random number tester and random number testing method

    JP2022098951A

  • Time series data processing device

    JP2022155980A