Reservoir, storage medium, and information processing device

Abstract

A reservoir according to an embodiment of the present invention comprises an input layer, a hidden layer, and an output layer. The hidden layer is configured to receive input of data from the input layer. The output layer is configured to apply a weight to data from the hidden layer and output the data. The hidden layer has a linear region and a non-linear region. The linear region has linear nodes, an activation function of which is a linear function. The non-linear region has non-linear nodes, an activation function of which is a non-linear function. The linear nodes belonging to the linear region are configured such that said linear nodes cannot be recursively connected.

Description

Reservoirs, storage media, and information processing devices

[0001] This disclosure relates to a reservoir, a storage medium, and an information processing device.

[0002] Neuromorphic devices are components that mimic the human brain using neural networks. They artificially mimic the relationships between neurons and synapses in the human brain.

[0003] A neuromorphic device, for example, has nodes (neurons in the brain) arranged hierarchically and means of communication (synapses in the brain) connecting them. Neuromorphic devices improve the accuracy of answering problems by having the means of communication (synapses) learn. Learning is the process of finding knowledge that may be useful in the future from information, and neuromorphic devices weight the input data.

[0004] One type of neural network is the recurrent neural network. Recurrent neural networks can handle nonlinear time-series data. Nonlinear time-series data is data whose values ​​change over time, and stock prices are one example. Recurrent neural networks can process time-series data by feeding back the processing results of neurons in later layers to neurons in earlier layers.

[0005] Reservoir computing is one method for realizing recurrent neural networks. Reservoir computing performs recursive processing by interacting signals based on internal connections. Reservoir computing is executed on a reservoir computer.

[0006] The performance requirements for a reservoir computer include short-term memory performance and nonlinear performance. Short-term memory performance is a measure of how much past information can be remembered and forgotten. Nonlinear performance is a measure of how much nonlinear transformation is performed on the input data. A reservoir computer with excellent short-term memory performance can output estimated solutions by taking into account past data of time series data. A reservoir computer with excellent nonlinear performance has high expressive power and can handle more complex time series data.

[0007] Generally, in reservoir computers, there is a trade-off between short-term memory performance and nonlinear performance. For example, Patent Document 1 discloses changing the ratio of linear nodes with linear functions to nonlinear nodes with nonlinear functions in order to obtain short-term memory performance and nonlinear performance according to the task.

[0008] International Publication No. 2022 / 024167

[0009] The short-term memory performance of a reservoir computer has an upper limit, even if the number of nodes constituting the reservoir is increased. For example, the information processing device described in Patent Document 1 has a limited upper limit on short-term memory performance, and adjusts the ratio of short-term memory performance to nonlinear performance within that limited upper limit. In the information processing device described in Patent Document 1, it is difficult to improve the short-term memory performance of the reservoir computer itself.

[0010] This disclosure is made in view of the above circumstances and provides a reservoir, a storage medium, and an information processing device that have excellent short-term memory performance.

[0011] A reservoir according to the first embodiment comprises an input layer, a hidden layer, and an output layer. The hidden layer is configured to receive data from the input layer. The output layer is configured to output the data from the hidden layer with added weights. The hidden layer has a linear region and a nonlinear region. The linear region has linear nodes whose activation function is a linear function. The nonlinear region has nonlinear nodes whose activation function is a nonlinear function. The linear nodes belonging to the linear region are configured not to be recursively connected.

[0012] The reservoir, storage medium, and information processing device according to the above embodiment can enhance short-term memory performance.

[0013] This is a conceptual diagram of a reservoir according to the first embodiment. This shows the results of determining the short-term memory performance of the reservoir according to the first embodiment. This shows the results of determining the short-term memory performance of a reservoir according to a comparative example. This is a schematic block diagram showing a specific example of the functional configuration of the information processing device according to the first embodiment. This is a conceptual diagram of a reservoir according to the first modified example. This is a conceptual diagram of a reservoir according to the second modified example. This is a conceptual diagram of an information processing device according to the third modified example.

[0014] The following description of this embodiment will be given in detail with reference to the drawings as appropriate. The drawings used in the following description may be enlarged for convenience to make the features of this disclosure easier to understand, and the specific configuration of each component may differ from the actual one. The configurations etc. exemplified in the following description are examples, and this disclosure is not limited to them, and can be modified as appropriate to achieve the effects of this disclosure.

[0015] "First Embodiment" Figure 1 is a conceptual diagram of a reservoir 10 according to the first embodiment. The reservoir 10 performs nonlinear transformation of the input signal and performs learning and inference processing.

[0016] The reservoir 10 has an input layer 1, a hidden layer 2, and an output layer 3. The input layer 1 and the output layer 3 are connected to the hidden layer 2.

[0017] Input layer 1 receives the input signal S ｉｎ Data based on this is input to the hidden layer 2. The input layer 1 is connected to, for example, a sensor. The input layer 1 receives the input signal S acquired by the sensor. ｉｎ It propagates to the hidden layer 2. The input layer 1 is, for example, a signal propagation circuit. The input layer 1 receives the input signal S ｉｎ Alternatively, the signal itself may be propagated to the hidden layer 2, or the input signal S ｉｎ The signal may be converted and propagated to the hidden layer 2. For example, the input layer 1 receives the input signal S ｉｎ Weighting may be applied to the input signal S ｉｎ The input signal S may be converted into a time-division signal. ｉｎIt may be digitally converted, or these processes may be combined. Input signal S ｉｎ may be either a scalar or a vector. Input signal S ｉｎ If it is a vector, each element representing the component of the vector may be responsible for input signal S ｉｎ For example, all elements representing the components of the vector may be input as input signal S ｉｎ to the reservoir 10, and each element may be input to the hidden layer 2. Also, for example, there may be a plurality of reservoirs 10, and all elements representing the components of the vector may be input as input signal S ｉｎ to each reservoir 10.

[0018] The hidden layer 2 is configured such that data from the input layer 1 is input. The hidden layer 2 stores the data from the input layer 1 and converts it into another signal.

[0019] The hidden layer 2 has a non-linear region 21 and a linear region 22.

[0020] The non-linear region 21 has non-linear nodes n21. The non-linear node n21 is a node whose activation function is a non-linear function. A node corresponds to a neuron in a neural circuit, and the connection between nodes corresponds to a synapse in a neural circuit. The non-linear node n21 non-linearly transforms and outputs the input. There are a plurality of non-linear nodes n21 in the non-linear region 21. The non-linear region 21 consists of a plurality of non-linear nodes n21. The number of non-linear nodes n21 in the non-linear region 21 is not limited.

[0021] Each non-linear node n21 is connected to one or more other non-linear nodes n21. Each non-linear node n21 may be connected to all other non-linear nodes n21 in the non-linear region 21, or may be connected to some of the non-linear nodes n21 in the non-linear region 21. The connections between the non-linear nodes n21 are random.

[0022] The connections between nonlinear nodes n21 include recursive connections. A recursive connection is a connection in which the output returns to the input. For example, a signal output from one nonlinear node n21 at time t may return to the nonlinear node n21 that output the signal at time t+1. This occurs because the signal output from one nonlinear node n21 propagates through other nonlinear nodes n21 and returns to the original nonlinear node n21. Such a connection relationship between nonlinear nodes n21 in which the output from one nonlinear node n21 is input again via another nonlinear node n21 is called a recursive connection.

[0023] Between the nonlinear nodes n21, coupling coefficients indicating coupling weights are set. A signal input to the nonlinear region 21 propagates between the nonlinear nodes n21. The signal that has propagated to a certain nonlinear node n21 is nonlinearly transformed by the activation function of the nonlinear node n21, and then multiplied according to the coupling coefficient before propagating to the next nonlinear node n21.

[0024] The coupling coefficient between the nonlinear nodes n21 can be arbitrarily set within the range of -1.0 to +1.0. The coupling coefficient between the nonlinear nodes n21 can be set, for example, by random numbers. The coupling coefficient between the nonlinear nodes n21 may be set, for example, to maximize the amount of mutual information between the output signal from the reservoir 10 and the expected signal.

[0025] Each of the nonlinear nodes n21 may be fully connected to the input layer 1. Fully connected to the input layer 1 means that each node within the region is connected to the input layer 1. By fully connecting the input layer 1 and each of the nonlinear nodes n21, the signals interact in a complex manner in the hidden layer 2. As a result, the nonlinear performance of the hidden layer 2 can be improved, and the reservoir 10 can handle more complex information.

[0026] Furthermore, each of the nonlinear nodes n21 may be fully connected to the output layer 3. Fully connected to the output layer 3 means that each node within the region is connected to the output layer 3. By fully connecting the output layer 3 and each of the nonlinear nodes n21, more signals are input to the output layer 3. By obtaining an estimated solution using a large amount of data, the accuracy (correctness rate) of the output from the reservoir 10 can be improved.

[0027] The linear region 22 has linear nodes n22. The linear node n22 is a node whose activation function is a linear function. The linear node n22 linearly transforms and outputs the input. There are a plurality of linear nodes n22 in the linear region 22. The linear region 22 consists of a plurality of linear nodes n22. The number of linear nodes n22 in the linear region 22 is not limited.

[0028] Each linear node n22 is coupled to one or more other linear nodes n22. The linear node n22 is configured so as not to be recursively coupled within the linear region 22. That is, the linear region 22 does not have a coupling relationship in which the output from a certain linear node n22 is re-input to the original linear node n22 via another linear node n22.

[0029] For example, the coupling between the linear nodes n22 in the linear region 22 is configured such that the data from the input layer 1 propagates through the linear nodes n22 in one direction. For example, the linear nodes n22 may be linearly connected, and in the linear region 22, the data from the input layer 1 may propagate through the linear nodes n22 in order. For example, the linear nodes n22 may be tree-connected, and in the linear region 22, the data from the input layer 1 may propagate through the linear nodes n22 in a tree shape.

[0030] A coupling coefficient indicating a coupling weight is set between the linear nodes n22. The signal input to the linear region 22 propagates between the linear nodes n22. The signal propagated to a certain linear node n22 is linearly transformed by the activation function of the linear node n22, and further multiplied according to the coupling coefficient, and then propagated to the next linear node n22.

[0031] The coupling coefficient between the linear nodes n22 can be arbitrarily set within the range of -1.0 to +1.0. The coupling coefficient between the linear nodes n22 is, for example, 1.0. By setting the coupling coefficient to 1.0, the amount of change in the data input from the input layer 1 in the linear region 22 can be reduced, and the short-term memory performance can be enhanced.

[0032] Each of the linear nodes n22 may or may not be fully connected to the input layer 1. In the linear region 22, from the perspective of propagating data in one direction, it is preferable that each of the linear nodes n22 and the input layer 1 are not fully connected. For example, the first linear node among the linear nodes n22 may be connected to the input layer 1, and the other linear nodes n22 may be connected to the subsequent stage of the first linear node. The number of the first linear nodes may be one or more.

[0033] Each of the linear nodes n22 may also be fully connected to the output layer 3. By fully connecting each of the output layer 3 and the linear nodes n22, more signals are input to the output layer 3. By obtaining an estimated solution using a large amount of data, the accuracy (correct answer rate) of the output from the reservoir 10 can be increased.

[0034] The hidden layer 2 may have a connection from the linear nodes n22 in the linear region 22 to the non-linear nodes n21 in the non-linear region 21. That is, within the hidden layer 2, the signal that has been propagating within the linear region 22 may propagate to the non-linear region 21. By not completely separating the non-linear region 21 and the linear region 22, the signal propagation within the hidden layer 2 can be complicated, and the reservoir 10 can handle more complex information.

[0035] In contrast, it is preferable that the hidden layer 2 does not have a connection from the non-linear nodes n21 in the non-linear region 21 to the linear nodes n22 in the linear region 22. By not imparting non-linear characteristics to the signal propagating within the linear region 22, the short-term memory performance of the reservoir 10 can be further enhanced.

[0036] The output layer 3 receives a signal from the hidden layer 2 and outputs an output signal S ｏｕｔ based on the signal. The output layer 3 performs learning processing and inference processing.

[0037] In the learning process, the output layer 3 compares the output from the hidden layer 2 and the teacher data D using a comparator C, and adjusts the coupling coefficient (coupling weight) w between the nodes (non-linear nodes n21 and linear nodes n22) of the hidden layer 2 and the nodes n3 of the output layer 3.

[0038] In the inference process, the output layer 3 applies coupling coefficients (coupling weights) w to the data from the hidden layer 2 to produce the output signal S. ｏｕｔ Output as: Output signal S ｏｕｔ This, for example, is an estimated solution to the task. Output signal S ｏｕｔ This can be either a scalar or a vector. Also, the input signal S ｉｎ When is a scalar, the output signal S ｏｕｔ The input signal S can be either a scalar or a vector. ｉｎ When is a vector, the output signal S ｏｕｔ It can be either a scalar or a vector.

[0039] The reservoir 10 may be software, hardware, or a combination of software and hardware. For example, the linear region 22 of the reservoir 10 may be software, and the nonlinear region 21 may be a hardware wafer.

[0040] Software is a program implemented within a computer. Reservoir 10 includes, for example, memory to store this program and a processor to execute this program.

[0041] The hardware is a physical reservoir consisting of a combination of real-world elements or circuits. The physical reservoir consists of multiple circuits or elements corresponding to nodes, which are connected to each other. The physical reservoir realizes the concept of reservoir computing using real-world elements or circuits. Furthermore, if the physical reservoir has an analog interface, the reservoir 10 may have a digital-to-analog converter. Alternatively, the reservoir 10 using digital circuits may be implemented on a PLD (Programmable Logic device) such as an FPGA (Field-Programmable Gate Array). A challenge with physical reservoirs is that they cannot adequately ensure short-term memory performance, making the application of the reservoir 10 according to this embodiment suitable.

[0042] Next, the operation of the reservoir 10 will be explained. The reservoir 10 performs learning and estimation processing.

[0043] First, let's explain the learning process of the reservoir 10. In the learning process, the information to be learned is input signal S ｉｎ This is input to input layer 1.

[0044] Input signal S ｉｎ The data is input to the hidden layer 2. The data is nonlinearly transformed in the nonlinear region 21 and linearly transformed in the linear region 22. The nonlinear transformation occurs when the data propagates between nonlinear nodes n21. The linear transformation occurs when the data propagates between linear nodes n22. The transformed data is multiplied by the coupling coefficient (coupling weight) w and input to the output layer 3.

[0045] In output layer 3, the output from hidden layer 2 is compared with the training data D using comparator C. Output layer 3 adjusts the coupling coefficient (coupling weight) w between the nodes of hidden layer 2 (nonlinear node n21 and linear node n22) and node n3 of output layer 3 according to the comparison result.

[0046] The reservoir 10 repeats the above process until the mutual information between the output and the training data D falls below a predetermined value. When the mutual information between the output and the training data D falls below a predetermined value, the accuracy (correctness) of the estimated solution output from the reservoir 10 during the estimation process becomes greater than or equal to the desired value. During the learning process, the reservoir 10 determines the coupling coefficient (coupling weight) w between the nodes of the hidden layer 2 (nonlinear node n21 and linear node n22) and the node n3 of the output layer 3.

[0047] Next, the estimation process of the reservoir 10 will be explained. In the estimation process, the information to be estimated is the input signal S. ｉｎ This is input to input layer 1.

[0048] Input signal S ｉｎ The data is input to the hidden layer 2. The data is nonlinearly transformed in the nonlinear region 21 and linearly transformed in the linear region 22. The nonlinear transformation occurs when the data propagates between nonlinear nodes n21. The linear transformation occurs when the data propagates between linear nodes n22. The transformed data is multiplied by the coupling coefficient (coupling weight) w and input to the output layer 3. This coupling coefficient (coupling weight) w is determined during the learning process.

[0049] The output layer 3 outputs the estimated solution based on the output from the hidden layer 2 as the output signal S. ｏｕｔ Output as: Output signal S ｏｕｔ This can be the output from hidden layer 2 itself, or the result obtained by substituting the output from hidden layer 2 into the activation function. Reservoir 10 outputs the answer to the task as an estimated solution.

[0050] The reservoir 10 according to this embodiment has a separate nonlinear region 21 responsible for nonlinear transformations and a linear region 22 responsible for linear transformations, thus enabling both nonlinear performance and short-term memory performance. The nonlinear region 21 mainly handles the nonlinear performance of the reservoir 10, while the linear region 22 mainly handles the short-term memory performance of the reservoir 10. Linear transformation linearly transforms the input signal, allowing the output signal to be converted back to the input signal. In other words, in the linear region 22, information from past input signals can be read from the output signal. Thus, by having a linear region 22, the reservoir 10 can refer to past information and has excellent short-term memory performance.

[0051] Figure 2 shows the results of determining the short-term memory performance of the reservoir 10 according to this embodiment. The vertical axis in Figure 2 is the decision coefficient, which represents the accuracy of the input signal reproduction. The horizontal axis in Figure 2 is the delay, which represents how many samples back in time the information can be retrieved.

[0052] In the calculation to obtain Figure 2, the number of nonlinear nodes n21 in the hidden layer 2 was set to 50, and the number of linear nodes n22 was set to 50. Furthermore, connections from linear nodes n22 to nonlinear nodes n21 were allowed, but connections from nonlinear nodes n21 to linear nodes n22 were not allowed.

[0053] As shown in Figure 2, the reservoir 10 according to this embodiment had a decision coefficient of approximately 1.0 up to 49.85 samples and was able to reproduce the input signal. In other words, the reservoir 10 according to this embodiment could retrieve information up to 49.85 samples in the past, and its short-term memory performance (MC) was 49.85.

[0054] Figure 3 shows the results of determining the short-term memory performance of the reservoir in the comparative example. The vertical axis in Figure 3 is the decision coefficient, which represents the accuracy of the input signal reproduction. The horizontal axis in Figure 3 is the delay, which represents how many samples back in time the data can be retrieved.

[0055] In the calculation to obtain Figure 3, the number of nonlinear nodes n21 in the hidden layer 2 was set to 100, and the number of linear nodes n22 was set to 0. In other words, no linear region 22 was provided in the hidden layer 2.

[0056] As shown in Figure 3, the comparative example reservoir had a sampling decision coefficient of approximately 1.0 and was able to reproduce the input signal. The comparative example reservoir could only retrieve information up to 13.47 samples in the past, and its short-term memory performance (MC) was 13.47.

[0057] Thus, the reservoir 10 according to this embodiment can enhance short-term memory performance. Furthermore, the nonlinear performance of the reservoir 10 can be enhanced by making the nonlinear transformation in the nonlinear region 21 more complex. For example, the nonlinear performance of the reservoir 10 can be enhanced by changing the number and coupling state of the nonlinear nodes n21.

[0058] Figure 4 is a schematic block diagram showing a specific example of the functional configuration of the information processing device 100 according to this embodiment. The information processing device 100 is composed of, for example, a personal computer or a server device. The information processing device 100 includes, for example, an input unit 20, an output unit 30, a storage unit 40, and a control unit 50.

[0059] The input unit 20 receives the input signal S ｉｎ This is the part where the input is received. The input unit 20 is an interface that handles connections to, for example, sensors. The input signal S is input to the input unit 20. ｉｎ This information is not limited to passive elements such as sensors, but may also include images, audio, or text information selected by the user.

[0060] The output unit 30 outputs information in a format that the user can recognize. The output unit 30 outputs the estimated solution of the reservoir 10. The output unit 30 may be an image display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The output unit 30 may also be an interface for connecting the image display device to the information processing device 100. In this case, the output unit 30 generates a video signal for displaying image data and outputs the video signal to the image display device connected to it. The output unit 30 may also be a device that outputs sound, such as a speaker. Output 303 may be an interface for connecting an audio output device such as a speaker or headphones to the information processing device 100. In this case, the output unit 30 generates an audio signal for playing back audio data and outputs the audio signal to the audio output device connected to it. The output unit 30 may also be configured as a touch panel integrated with the input unit 20.

[0061] The storage unit 40 is configured using a storage medium such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 40 stores data used by the control unit 50. The storage unit 40 stores data necessary when the control unit 50 performs processing.

[0062] The storage unit 40 includes, for example, a training data storage unit 41 and a coupling coefficient storage unit 42.

[0063] The training data storage unit 41 stores the training data D that has been input in advance. The coupling coefficient storage unit 42 stores the coupling coefficients (coupling weights) w between the nodes of the hidden layer 2 (nonlinear node n21 and linear node n22) and the node n3 of the output layer 3. The coupling coefficient storage unit 42 may update the stored coupling coefficients w after each learning process.

[0064] The control unit 50 is composed of a processor 51 such as a CPU (Central Processing Unit) and a memory 52. ​​The control unit 50 functions when the processor 51 executes a program. Note that all or part of the functions of the control unit 50 may be implemented using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array).

[0065] The program is stored in memory 52. ​​Memory 52 may be a computer-readable recording medium. Computer-readable recording media include, for example, portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and semiconductor memory devices (e.g., SSDs: Solid State Drives), as well as storage devices such as hard disks and semiconductor memory devices built into a computer system. The above program may be transmitted to the information processing device 100 via a telecommunications line.

[0066] Memory 52 stores, for example, the reservoir structure of the first model. The reservoir structure of the first model is the reservoir 10 described above. Memory 52 includes a learning program that performs learning processing and an inference program that performs inference processing in the first model reservoir 10.

[0067] The information processing device 100 according to this embodiment stores the above-mentioned reservoir 10 as a model when operating a learning program or inference program. Therefore, the information processing device 100 according to this embodiment has excellent short-term memory performance.

[0068] Although this embodiment has been described in detail above with reference to the drawings, the configurations and their combinations in this embodiment are merely examples, and additions, omissions, substitutions, and other modifications to the configurations are possible without departing from the spirit of the present invention.

[0069] For example, Figure 5 is a conceptual diagram of a reservoir 11 according to the first modified example. Reservoir 11 is replaceable with the reservoir 10 described above.

[0070] Reservoir 11 differs from reservoir 10 in that the input layer 1 and the linear region 22 of the hidden layer 2 are not connected. Reservoir 11 has a connection from a nonlinear node n21 to a linear node n22, and the linear region 22 receives the input signal S via the nonlinear node n21. ｉｎ The following is entered.

[0071] Since reservoir 11 has a linear region 22 similar to reservoir 10, its short-term memory performance can be improved. On the other hand, reservoir 11 receives the input signal S ｉｎ Since the signal is input to the linear region 22 via the nonlinear node n21, the signal input to the linear region 22 is nonlinear. Therefore, the short-term memory performance of the reservoir 11 is considered to be inferior to that of the reservoir 10.

[0072] For example, Figure 6 is a conceptual diagram of a reservoir 12 according to a second modified example. Reservoir 12 is replaceable with the reservoir 10 described above.

[0073] Reservoir 12 differs from reservoir 10 in that the output layer 3 and the linear region 22 of the hidden layer 2 are not connected. Reservoir 12 has a connection from the linear node n22 to the nonlinear node n21, and the signal from the nonlinear node n21 is input to the output layer 3.

[0074] Since reservoir 12 has a linear region 22, similar to reservoir 10, it can improve short-term memory performance. On the other hand, since reservoir 12 has fewer connections between the nodes of the hidden layer 2 and the output layer 3 than reservoir 10, it is thought to have lower expressive power than reservoir 10.

[0075] Figures 5 and 6 show an example where there is no direct coupling between the input layer 1 or output layer 3 and the linear region 22, but there is a direct coupling between the input layer 1 or output layer 3 and the nonlinear region 21. However, this relationship can be reversed. For example, the reservoir will function even if there is a direct coupling between the input layer 1 or output layer 3 and the linear region 22, but no direct coupling between the input layer 1 or output layer 3 and the nonlinear region 21.

[0076] Figure 7 is a schematic diagram of an information processing device 101 according to a third modified example. The information processing device 101 includes a terminal device 60 and a storage unit 40. The terminal device 60 and the storage unit 40 are communicated with each other via a network 70. The network 70 may be a wireless communication network or a wired communication network. The network 70 may be configured using, for example, the Internet or a local area network (LAN). The network 70 may be configured by combining multiple networks.

[0077] The information processing device 101 differs from the information processing device 100 in that its storage unit 40 is located on an external server. The terminal device 60 has the input unit 20, output unit 30, and control unit 50 described above. The terminal device 60 is configured using information devices such as smartphones, tablets, personal computers, and dedicated devices.

[0078] The terminal device 60 and the storage unit 40 may each have a communication unit. The communication unit may be configured, for example, as a network interface. The communication unit communicates data with other devices via the network 70 in accordance with the control of the control unit 50. The communication unit may be a wireless communication device or a wired communication device.

[0079] The information processing device 101 is simply an externally formed storage unit 40, and achieves the same effects as the information processing device 100.

[0080] 1 Input layer 2 Hidden layer 3 Output layer 10, 11, 12 Reservoir 20 Input unit 21 Nonlinear domain 22 Linear domain 30 Output unit 40 Memory unit 41 Training data storage unit 42 Coupling coefficient storage unit 50 Control unit 51 Processor 52 Memory 60 Terminal device 70 Network 100, 101 Information processing device C Comparator D Training data n3 Node n21 Nonlinear node n22 Linear node S ｉｎ Input signal S ｏｕｔ Output signal w Coupling coefficients (coupling weights)

Claims

1. A reservoir comprising an input layer, a hidden layer, and an output layer, wherein the hidden layer is configured to receive data from the input layer, the output layer is configured to apply weights to the data from the hidden layer, the hidden layer has a linear region and a nonlinear region, the linear region has linear nodes whose activation function is a linear function, the nonlinear region has nonlinear nodes whose activation function is a nonlinear function, and the linear nodes belonging to the linear region are configured not to be recursively connected.

2. The reservoir according to claim 1, having connections from the linear node to the nonlinear node.

3. The reservoir according to claim 1, wherein in the linear region, data from the input layer is configured to propagate in one direction between the linear nodes.

4. The reservoir according to claim 1, wherein the coupling coefficient between the linear nodes in the linear region is 1.

0.

5. The reservoir according to claim 1, which does not have connections from the nonlinear node to the linear node.

6. The reservoir according to claim 1, wherein each of the nonlinear nodes in the nonlinear region is fully coupled to the output layer.

7. The reservoir according to claim 1, wherein each of the linear nodes in the linear region is fully coupled to the output layer.

8. A computer-readable storage medium storing a program that causes a computer to perform a reservoir calculation, wherein the reservoir calculation has a first model of reservoir, the first model comprises an input layer, a hidden layer, and an output layer, the hidden layer is configured to receive data from the input layer, the output layer is configured to apply weights to the data from the hidden layer, the hidden layer has a linear region and a nonlinear region, the linear region has linear nodes whose activation function is a linear function, the nonlinear region has nonlinear nodes whose activation function is a nonlinear function, and the linear nodes belonging to the linear region are configured not to be recursively connected.

9. The storage medium according to claim 8, having connections from the linear node to the nonlinear node.

10. The storage medium according to claim 8, wherein in the linear region, data from the input layer is configured to propagate in one direction between the linear nodes.

11. The storage medium according to claim 8, wherein the coupling coefficient between the linear nodes in the linear region is 1.

0.

12. The storage medium according to claim 8, which does not have connections from the nonlinear node to the linear node.

13. The storage medium according to claim 8, wherein each of the nonlinear nodes in the nonlinear region is fully coupled to the output layer.

14. The storage medium according to claim 8, wherein each of the linear nodes in the linear region is fully coupled to the output layer.

15. An information processing apparatus comprising the storage medium described in claim 8.

Images