Information processing device, information processing method, and information processing program

Abstract

To improve prediction accuracy.SOLUTION: A processor comprises: a first data processing part 211 having a first reservoir computing 20-1 arranged in a forward direction and a second reservoir computing 20-4 arranged in a reverse direction; and a second data processing part 212 having a third reservoir computing 20-2 arranged in a forward direction and a fourth reservoir computing 20-4 arranged in a reverse direction. The first data processing part 211 predicts first time-series data, the second data processing part 212 predicts second time-series data with lower occurrence frequency than that for the first time-series data, and when data that the first data processing part 211 cannot predict is inputted, the second data processing part 212 generates a signal indicating a prediction error of the first data processing part 211 and performs prediction based on the signal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] In order to overcome the limitations of Moore's Law, the entire IT (Information Technology) industry is exploring new computer architectures that are not the conventional von Neumann type. Many non-von Neumann machines have been proposed, ranging from quantum computing, which will likely take many years to develop, to relatively easy-to-get-started technologies such as FPGAs (Field Programmable Gate Arrays) and GPGPUs (General-Purpose computing on Graphics Processing Units). All of these avoid the von Neumann bottleneck and efficiently solve problems in specific fields that require computational resources.

[0003] Machine learning is a problem that can be combined with these. In particular, today's multi-layered deep learning requires a large amount of computational resources, so many companies are researching and developing ways to improve its efficiency. Due to the success of deep learning, new machine learning models are being developed and improved every day at research institutes and companies.

[0004] Recurrent Neural Networks (RNNs) are known as a machine learning technique suitable for handling time-series data. Because RNNs have a structure that includes loops within the network, they can have weighting based on the correlation between past data and current data. RNNs are expected to be used in applications such as dynamic decision-making in video processing and natural language processing.

[0005] However, it is known that updating the interlayer weights due to backpropagation through time takes a very long time, especially in high-performance RNNs such as LSTM (Long Short Term Memory) and GRU (Gated Recurrent Unit), and various methods for improving this have been studied.In recent years, an RNN algorithm called Reservoir Computing (RC), which updates the weights only in the output stage, has attracted attention. RC is a special type of RNN that has a structure called a "reservoir" with random, fixed connections, rather than multi-layer connections like deep learning.

[0006] The presence or absence of connections between nodes in RC, as well as the connection direction and weights, are determined prior to machine learning (training). Typically, when RC is implemented in a program, the presence or absence of connections and the weight values ​​for each connection are determined randomly, and a reservoir is generated using the connection probability between nodes and the weight variance as parameters. Training proceeds while the connections and weights between nodes in the reservoir remain fixed, and only the weights in the output layer are updated by the output weight calculation unit shown in the diagram. When the training process is complete, the output weights are fixed, and output data is obtained in the inference process.

[0007] Because the reservoir is random and has fixed connections, any physical entity can be used as the reservoir, such as a random network circuit with nodes made up of elements with nonlinear properties, a mass of electrically nonlinear materials, or a material with optical nonlinearity. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Publication No. 2019-101635 [Patent Document 2] U.S. Patent No. 10,395,168 Summary of the Invention [Problem to be solved by the invention]

[0009] Machine learning using RC makes it possible to predict time series data. For long-term prediction, it is desirable to be able to predict not only periodic data that occurs frequently, but also events that occur infrequently.

[0010] Analysis using long-term time series data can be used, for example, to predict temperature changes throughout the year or to predict failures from vibrations during machine operation. Long-term temperature changes include both daily temperature changes and seasonal fluctuations, and it is necessary to learn about waveforms that combine two waveforms with significantly different occurrence frequencies. In addition, if unusual vibrations that occasionally lead to failures appear among the vibrations occurring during normal machine operation, the failure prediction system can become more accurate by effectively learning about these special vibrations.

[0011] However, with a single RC, it is difficult to predict realistic data with different occurrence periods, even when the scale is increased. Therefore, there is a need to combine multiple RCs to improve the prediction accuracy of complex data. [Means for solving the problem]

[0012] For this reason, this information processing device: The computer system includes a first reservoir computing, a second reservoir computing, a third reservoir computing, and a fourth reservoir computing, each having an input layer, a reservoir, and an output layer, and the first reservoir computing, the third reservoir computing, the fourth reservoir computing, and the second reservoir computing are connected in series in this order, and First reservoir computing and Record number a first data processing unit having a second reservoir computing; Record number 3Reservoir Computing and Pre- Record number 4. A second data processing unit having a reservoir computing and Preparation, First periodic data is provided to the first reservoir computing as training data, and second periodic data having a longer period than the first periodic data is provided to the third reservoir computing as training data; in the first data processing unit, time series data is input to the first reservoir computing, and the first reservoir computing outputs first output data; the first output data and fourth output data from the fourth reservoir computing are input to the second reservoir computing; in the second data processing unit, a prediction error signal indicating a difference between the first output data and the first periodic data is input to the third reservoir computing; third output data from the third reservoir computing is input to the fourth reservoir computing; the prediction error signal is provided to the fourth reservoir computing as training data; and the fourth reservoir computing outputs the fourth output data based on the third output data and the prediction error signal; In the first data processing unit, The second reservoir computing outputs second output data based on the fourth output data and the first output data. . [Effects of the Invention]

[0013] According to one embodiment, prediction accuracy can be improved. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a computer system according to an embodiment. [Figure 2] FIG. 1 is a conceptual diagram of reservoir computing in a computer system as an example of an embodiment. [Figure 3] 1 is a diagram illustrating reservoir computing in a computer system according to an exemplary embodiment. [Figure 4] FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing device of a computer system according to an embodiment. [Figure 5] 10 is a flowchart illustrating a process of a training phase in a computer system as an example of an embodiment. [Figure 6] 10 is a flowchart illustrating processing of an inference phase in a computer system as an example of an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present information processing device, information processing method, and information processing program will be described with reference to the drawings. However, the embodiments shown below are merely examples, and are not intended to exclude the application of various modifications and techniques not explicitly stated in the embodiments. In other words, the present embodiment can be implemented with various modifications within the scope of its purpose. Furthermore, each figure does not intend to include only the components shown in the figure, but can also include other functions, etc.

[0016] (A) Configuration FIG. 1 is a diagram illustrating a schematic configuration of a computer system 1 according to an embodiment.

[0017] As shown in FIG. 1, the computer system 1 includes reservoir computing (RC) units 20-1 to 20-4, a prediction error calculation circuit 213, a data buffer 2141, and a management unit 101.

[0018] The reservoir computings 20-1 to 20-4 have the same configuration. Hereinafter, when there is no need to distinguish between the reservoir computings 20-1 to 20-4, they will be referred to as reservoir computing 20. Furthermore, reservoir computing may be referred to as RC. FIG. 2 is a conceptual diagram of reservoir computing 20 in a computer system 1 as an example of an embodiment. The RC 20 includes a reservoir 201 , an input layer 202 and an output layer 203 .

[0019] A plurality of nonlinear nodes are randomly fixed and connected in the reservoir 201. The presence or absence of connections between nodes in the reservoir 201, as well as the connection direction and weights, are determined prior to machine learning (training).

[0020] When implementing RC in a program, the presence or absence of connections and the weight values ​​for each connection are randomly determined, and a reservoir is generated using the connection probability between nodes and the weight distribution as parameters.

[0021] Training of the RC 20 is performed while the inter-node connections and weights in the reservoir 201 are fixed. The output layer 203 includes an output weight calculation unit 2031. The output weight calculation unit 2031 updates the output weights Wo during the training process. When the training process is completed, the output weights Wo in the output layer 203 are fixed. During the inference process, output data is obtained from the output layer 203. The output layer 203 of each RC 20 is realized by the information processing device 10 (management unit 101) described later.

[0022] In the reservoir 201, nonlinear nodes are randomly fixed and connected. Therefore, a physical entity may be used as the reservoir 201. That is, the RC 20 may be a physical reservoir computing. The physical reservoir computing may be configured as a circuit network of nonlinear elements.

[0023] To realize RC20 as physical reservoir computing, reservoir 201 may be a random network circuit with nodes made of elements with nonlinear characteristics, a mass of electrically nonlinear material, or a material with optical nonlinearity. FIG. 3 is a schematic diagram showing the RC 20 in the computer system 1 as an example of an embodiment. The RC 20 may be represented by the diagram illustrated in FIG. In the physical reservoir computing 20 illustrated in FIG. 3, the reservoir 201 is configured as a network of physical entities having a plurality of nonlinear nodes.

[0024] The RC 20 may be a physical reservoir computing in which the reservoir 201 is a network of physical entities with multiple nonlinear nodes.

[0025] In the RC 20, input data is input to the input layer 202. The data processed in the input layer 202 is input to the reservoir 201, and after being processed in this reservoir 201, is input to the output layer 203. Hereinafter, in FIG. 1, the direction from the input layer 202 to the output layer 203 via the reservoir 201 in the RC 20-1 will be referred to as the forward direction.

[0026] As shown in FIG. 1, in this computer system 1, a plurality of (four in the example shown in FIG. 1) RCs 20-1 to 20-4 are connected in series.

[0027] A prediction error calculation circuit 213 is disposed downstream of the RC 20-1 in the forward direction. Also, a prediction error calculation circuit 213 is disposed downstream of the RC 20-2 in the forward direction.

[0028] In addition, RC20-3 is disposed downstream of RC20-2, and RC20-4 is disposed downstream of RC20-3.

[0029] 1, the continuum formed by the four RCs 20-1 to 20-4 connected in series is bent 180 degrees between RCs 20-2 and RCs 20-3. As a result, the path direction from RCs 20-3 to RCs 20-4 is opposite to the path direction (forward direction) from RCs 20-1 to RCs 20-2. Hereinafter, the direction opposite to the forward direction will be referred to as the reverse direction.

[0030] Hereinafter, RC20-1 may be referred to as forward RC#1, RC20-2 may be referred to as forward RC#2, RC20-3 may be referred to as backward RC#2, and RC20-4 may be referred to as backward RC#1.

[0031] Time-series data (input data) is input to the forward RC#1, and the forward RC#1 infers (predicts) a time-series periodic signal. That is, the forward RC#1 infers a periodic signal a predetermined time ahead (in the future). The output data (inference result) output from the forward RC#1 is input to the prediction error calculation circuit 213 and the backward RC#1.

[0032] In the training phase, a first periodic signal (sine signal or cosine signal) is input to the forward RC#1 as training data, and the weights of the output layer 203 are updated. The first periodic signal may also be called first periodic data.

[0033] The prediction error calculation circuit 213 calculates the difference (prediction error) between the output data (inference result) of the forward RC#1 and the teacher data (first periodic signal) of the forward RC#1. That is, the prediction error calculation circuit 213 calculates the prediction error of the forward RC#1. The prediction error of the forward RC#1 calculated by the prediction error calculation circuit 213 is input to the data buffer 214 and stored in a predetermined storage area of ​​the data buffer 214. Furthermore, the prediction error in forward RC#1 calculated by the prediction error calculation circuit 213 is also input to forward RC#2. A signal indicating the prediction error in forward RC#1 is input to forward RC#2, and a time-series periodic signal is inferred. The output data (inference result) output from the forward RC#2 is input to the backward RC#2.

[0034] In the training phase, a second periodic signal (sine signal or cosine signal) is input to the forward RC#2, and the weights of the output layer 203 are updated. The second periodic signal may be referred to as second periodic data. The second periodic signal has a period different from that of the first periodic signal, and may be, for example, longer than that of the first periodic signal. Furthermore, it is desirable that the second periodic signal has a waveform that is detected less frequently in the input data than the first periodic signal.

[0035] The backward RC#2 receives the output data of the forward RC#2 and infers the difference (prediction error) between the output signal (inference result) of the forward RC#1 and the teacher data (first periodic signal) of the forward RC#1. That is, the backward RC#2 infers the prediction error of the forward RC#1 for a predetermined time ahead (future).

[0036] In the training phase, the actual values ​​of the prediction errors calculated by the prediction error calculation circuit 213 are read from the data buffer 214 and then input as training data to the backward RC#2, and the weights of the output layer 203 are updated. The output data (inference result) output from the backward RC#2 is input to the backward RC#1.

[0037] The backward RC#1 receives the output data of the backward RC#2 and the output data of the forward RC#1, and infers (time series prediction) the data one step ahead (future) of the input data (time series data) input to the forward RC#1. In the training phase, input data input to the forward RC#1 is input as training data to the backward RC#1, and the weights of the output layer 203 are updated.

[0038] 1, a forward RC#1 and a reverse RC#1 are arranged vertically side by side. The pair of the forward RC#1 and the reverse RC#1 may be referred to as a first data processing unit 211. Data processing in the first data processing unit 211 may also be referred to as first-stage data processing. In the first data processing unit 211, the RC 20-1 is arranged in the forward direction, and the RC 20-4 is arranged in the reverse direction.

[0039] A waveform that occurs frequently and repeatedly is predicted in the first data processing unit 211. When irregular data that cannot be predicted by the first data processing unit 211 is input, the irregular data is passed to the second data processing unit 212 (described later) as a prediction error of the forward direction RC#1. It can be said that irregular data is data that occurs infrequently. The first data processing unit 211 predicts time series data that occurs infrequently.

[0040] 1, a forward RC#2 and a backward RC#2 are arranged vertically. The pair of the forward RC#2 and the backward RC#2 may be referred to as a second data processing unit 212. The second data processing unit 212 includes a prediction error calculation circuit 213 and a data buffer 214. Data processing in the second data processing unit 212 may be referred to as second-stage data processing. In the second data processing unit 212, the RC 20-2 is arranged in the forward direction, and the RC 20-3 is arranged in the reverse direction.

[0041] A signal indicating the prediction error of forward RC#1 generated by the prediction error calculation circuit 213 is input to the second data processing unit 212. Therefore, when a waveform with a low occurrence frequency is input to forward RC#1, a large signal is input to the second data processing unit 212, which processes it. That is, when data that cannot be predicted by the first data processing unit 211 is input, a signal is passed to the second data processing unit 212, and the second data processing unit 212 makes a prediction.

[0042] In the second data processing unit 212, a second periodic signal different from the first periodic signal is input as training data to the forward direction RC#2. Also, a signal indicating the prediction error of the forward direction RC#1 generated by the prediction error calculation circuit 213 is input as training data to the backward direction RC#2. In this way, the second data processing unit 212 is not trained on the frequently occurring (most frequent) data (first time-series data) handled by the first data processing unit 211, but is responsible for prediction only for events that occur less frequently than this first time-series data.

[0043] 1, the second data processing unit 212 is disposed downstream in the forward direction from the first data processing unit 211. As a result, the computer system 1 is configured as a multi-stage reservoir computing system having the second data processing unit 212 downstream from the first data processing unit 211.

[0044] The management unit 101 realizes the output layer 203 of each RC 20. For example, the management unit 101 performs calculations as the output weight calculation unit 2031 in the output layer 203 of each RC 20.

[0045] Furthermore, the management unit 101 initializes various variables in the RC 20 during the training phase of the computer system 1. Furthermore, the management unit 101 controls the input of input data and teacher data to each RC 20. Furthermore, the management unit 101 controls the input of input data to each RC 20 in the inference phase of the computer system 1. FIG. 4 is a diagram illustrating a hardware configuration of the information processing device 10 of the computer system 1 according to an example of the embodiment.

[0046] The information processing device 10 is a computer and includes, as components, a processor 11, a memory 12, a storage device 13, a graphics processing device 14, an input interface 15, an optical drive device 16, a device connection interface 17, and a network interface 18. These components 11 to 18 are configured to be able to communicate with each other via a bus 19.

[0047] The processor (control unit) 11 controls the entire information processing device 10. The processor 11 may be a multiprocessor. The processor 11 may be, for example, any one of a CPU, an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). The processor 11 may also be a combination of two or more types of elements from the CPU, MPU, DSP, ASIC, PLD, and FPGA.

[0048] The processor 11 executes a control program (information processing program, not shown) for the information processing device 10, thereby realizing the function of the management unit 101 illustrated in FIG.

[0049] The information processing device 10 realizes the function of the management unit 101 by executing a program (information processing program, OS program) recorded on a computer-readable non-transitory recording medium, for example.

[0050] The program describing the processing to be executed by the information processing device 10 can be recorded on various recording media. For example, the program to be executed by the information processing device 10 can be stored in the storage device 13. The processor 11 loads at least a part of the program in the storage device 13 into the memory 12 and executes the loaded program.

[0051] The program to be executed by the information processing device 10 (processor 11) may also be recorded on a non-transitory portable recording medium such as an optical disk 16a, a memory device 17a, or a memory card 17c. The program stored on the portable recording medium becomes executable after being installed in the storage device 13, for example, under the control of the processor 11. The processor 11 may also read and execute the program directly from the portable recording medium.

[0052] The memory 12 is a storage memory including a ROM (Read Only Memory) and a RAM (Random Access Memory). The RAM of the memory 12 is used as the main storage device of the information processing device 10. The RAM temporarily stores at least a part of the program to be executed by the processor 11. The memory 12 also stores various data required for processing by the processor 11.

[0053] The storage device 13 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a storage class memory (SCM), and stores various data. The storage device 13 is used as an auxiliary storage device for the information processing device 10. An OS program, a control program, and various data are stored in the storage device 13. The control program includes an information processing program.

[0054] The auxiliary storage device may be a semiconductor storage device such as an SCM or a flash memory. A plurality of storage devices 13 may be used to configure a RAID (Redundant Array of Inexpensive Disks). The storage device 13 may store various data generated by the management unit 101.

[0055] A monitor 14a is connected to the graphics processing device 14. The graphics processing device 14 displays an image on the screen of the monitor 14a in accordance with an instruction from the processor 11. Examples of the monitor 14a include a display device using a CRT (Cathode Ray Tube) and a liquid crystal display device.

[0056] A keyboard 15a and a mouse 15b are connected to the input interface 15. The input interface 15 transmits signals sent from the keyboard 15a and the mouse 15b to the processor 11. The mouse 15b is an example of a pointing device, and other pointing devices can also be used. Examples of other pointing devices include a touch panel, a tablet, a touch pad, and a trackball.

[0057] The optical drive device 16 uses a laser beam or the like to read data recorded on an optical disc 16a. The optical disc 16a is a portable, non-transitory recording medium on which data is recorded so that it can be read by reflected light. Examples of the optical disc 16a include a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc Read Only Memory), and a CD-R (Recordable) / RW (Rewritable).

[0058] The device connection interface 17 is a communication interface for connecting peripheral devices to the information processing device 10. For example, a memory device 17a or a memory reader / writer 17b can be connected to the device connection interface 17. The memory device 17a is a non-transitory recording medium, such as a USB (Universal Serial Bus) memory, that has a function for communicating with the device connection interface 17. The memory reader / writer 17b writes data to or reads data from a memory card 17c. The memory card 17c is a card-type non-transitory recording medium.

[0059] The network interface 18 is connected to a network. The network interface 18 transmits and receives data via the network. Other information processing devices, communication devices, etc. may be connected to the network.

[0060] (B) Operation The processing of the training phase in the computer system 1 configured as above as an example of an embodiment will be described with reference to the flowchart (steps A1 to A7) shown in FIG. In step A1, the management unit 101 initializes the weight of the reservoir 201 of each RC 20. In step A2, the management unit 101 causes the forward direction RC#1 to input, as input data, time series data one step earlier (time t-1) than the latest time series data (time t).

[0061] Furthermore, the management unit 101 inputs the first periodic signal to the forward RC#1 as training data, and causes the output layer 203 (output weight calculation unit 2031) to perform training (machine learning).

[0062] In forward RC#1, the output weight calculation unit 2031 optimizes the weights of the output layer 203 by minimizing a loss function using, for example, the least squares method based on the inference results performed on the input data and the training data.

[0063] The inference result (output data) by the forward RC#1 is input to the prediction error calculation circuit 213 and the backward RC#1. The prediction error calculation circuit 213 calculates the difference (prediction error) between the output data (inference result) of the forward RC#1 and the teacher data (first periodic signal) of the forward RC#1 (step A5). The prediction error of the forward RC#1 calculated by the prediction error calculation circuit 213 is input as input data to the forward RC#2. In addition, the prediction error of the forward RC#1 calculated by the prediction error calculation circuit 213 is input as teacher data to the backward RC#2 via the data buffer 214. Furthermore, the management unit 101 causes the second periodic signal to be input to the forward direction RC#2 as training data.

[0064] In forward RC#2, the output weight calculation unit 2031 optimizes the weights of the output layer 203 by minimizing the loss function using, for example, the least squares method based on the inference results performed on the input data (prediction error) and the training data (step A6).

[0065] By inputting the prediction error of Forward RC#1 to the input layer 202 of Forward RC#2, the data series that can be inferred in the first stage data processing (first data processing unit 211) is not transmitted to the second data processing unit 212, and only the data that cannot be inferred is received by the second data processing unit 212, and training proceeds. The output data of Forward RC#2 is input to Reverse RC#2. Furthermore, the management unit 101 inputs the prediction error of the forward RC#1 calculated by the prediction error calculation circuit 213 to the forward RC#2 via the data buffer 214 as training data.

[0066] In the backward RC#2, the output weight calculation unit 2031 optimizes the weights of the output layer 203 by minimizing a loss function using, for example, the least squares method based on the inference results performed on the input data (output data of the forward RC#2) and the teacher data (step A7). The output data of the backward RC#2 is input to the backward RC#1. The management unit 101 inputs the latest (time t) time series data (input data) to the backward RC#1 as training data.

[0067] In the backward RC#1, the output weight calculation unit 2031 optimizes the weights of the output layer 203 based on the inference results performed on the input data (output data of the backward RC#2) and the teacher data, for example, by minimizing the loss function using the least squares method (step A3).

[0068] In step A4, the management unit 101 checks whether training has been performed for all the planned teacher data in each RC 20. If the result of the check shows that training has not been performed for all the teacher data (see the NO route in step A4), the process returns to step A2. On the other hand, if training has been performed for all the teaching data (see the YES route in step A4), the process ends. Next, the processing of the inference phase in the computer system 1 as an example of an embodiment will be described with reference to the flowchart (steps B1 to B6) shown in FIG. In step B1, the management unit 101 causes the forward direction RC#1 to input the latest time series data (at time t) as input data.

[0069] The forward RC#1, to which the time-series data is input, performs inference. The inference result (output data) is input to the prediction error calculation circuit 213 and the backward RC#1. The prediction error calculation circuit 213 calculates the difference (prediction error) between the output data (inference result) of the forward RC#1 and the teacher data (first periodic signal) of the forward RC#1 (step B2). The prediction error of the forward RC#1 calculated by the prediction error calculation circuit 213 is input as input data to the forward RC#2. In step B3, the forward RC#2 performs inference on the input data (prediction error). The output data of the forward RC#2 is input to the backward RC#2.

[0070] In step B4, the backward RC#2 performs inference on the input data (output data of the forward RC#2). The inference result is input to the backward RC#1.

[0071] In step B5, backward RC#1 performs inference on the input data (output data of backward RC#2) and predicts the time series data one step ahead (future: time t+1) of the latest time series data input in step B1. The inference result is output as output data of the computer system 1. The output data may be stored in a predetermined storage area such as memory 12 or storage device 13.

[0072] In step B6, the management unit 101 checks whether inference has been performed for all input data. If the check result shows that inference has not been performed for all input data (see the NO route in step B6), the process returns to step B1. On the other hand, if inference has been performed for all input data (see the YES route in step B6), the process ends.

[0073] (C) Effects Thus, according to the computer system 1 as an example of the embodiment, the first data processing unit 211, which is equipped with the forward RC#1 and the backward RC#1, infers frequently occurring time series data, and when data that cannot be inferred by the first data processing unit 211 is input, a signal is passed as a prediction error of the forward RC#1 to the second data processing unit 212. Therefore, the second data processing unit 212 does not perform machine learning on the most frequently occurring periodic data, and can be responsible for inference only regarding infrequent events, thereby improving the overall prediction accuracy. Here, the results of a simulation using a physical reservoir with 100 nodes as the reservoir 201 will be shown.

[0074] The reservoir 201 was implemented as a network (a crossbar structure with elements and switches at the intersections) composed of elements (memristors) with nonlinear characteristics. Each reservoir 201 had 10 inputs and 10 outputs, with 100 node elements at the intersections. The connections between the nodes were determined randomly, and the strength of the connections (reservoir 201 connection weights) were also determined randomly. Using four RC20s equipped with reservoirs 201 configured in this way, the computer system 1 shown in Figure 1 was constructed, and training and inference were performed. In addition, a single RC with 400 nodes was created in the program as a comparison source, and the prediction accuracy was compared. The task used was a nonlinear function prediction problem called NARMA10, which is a common method for evaluating RNNs.

[0075] Learning was performed using a data sequence of the NARMA10 function, and the error (NRMSE: normalized root mean square error) was calculated from the correct data sequence output from the NARMA10 function and the numerical sequence predicted by the RC after learning. Learning was performed under the same conditions using a conventional RC and this computer system 1, and the prediction error was evaluated.

[0076] As a result, the prediction error of the conventional single-structure RC was 0.1982, whereas the prediction error of the computer system 1 was 0.1276, which means that the error was reduced and more appropriate function prediction was possible.

[0077] (D) Other The configurations and processes of this embodiment can be selected as needed, or can be combined as appropriate. The disclosed technology is not limited to the above-described embodiment, and can be implemented in various modifications without departing from the spirit of the present embodiment.

[0078] For example, in the above-described embodiment, four RCs 20 are provided and prediction processing is performed in two stages by the first data processing unit 211 and the second data processing unit 212, but this is not limited to this. Six or more RCs 20 may be provided, multiple second data processing units 212 may be provided, and prediction processing may be performed in three or more stages. In this case, the input irregular data may be propagated sequentially to the second data processing units 212 in the second and subsequent stages, and prediction may be performed.

[0079] In the above-described embodiment, each RC 20 may be realized by hardware having a physical reservoir, or the information processing device 10 may realize the functions of each RC 20 by executing software. Furthermore, the above disclosure will enable those skilled in the art to implement and manufacture the present embodiment.

[0080] (E) Supplementary Note The following additional notes are provided regarding the above-described embodiments. (Appendix 1) a first data processing unit having a first reservoir computing arranged in a forward direction and a second reservoir computing arranged in a reverse direction to the first reservoir computing; a second data processing unit having a third reservoir computing arranged in the forward direction and a fourth reservoir computing arranged in the reverse direction; Equipped with In the first data processing unit, first time series data is predicted; In the second data processing unit, second time series data having a lower occurrence frequency than the first time series data is predicted; When data that cannot be predicted by the first data processing unit is input to the first data processing unit, the second data processing unit generates a signal indicating a prediction error of the first data processing unit and performs prediction based on the signal. 1. An information processing device comprising:

[0081] (Appendix 2) In the first data processing unit, an output of the first reservoir computing is input to the second reservoir computing; In the second data processing unit, a signal indicating a prediction error of the first reservoir computing is generated as the signal, the signal is input to the third reservoir computing, and an output of the third reservoir computing is input to the fourth reservoir computing. 2. The information processing device according to claim 1,

[0082] (Appendix 3) a first periodic signal is input to the first reservoir computing as training data; A second periodic signal different from the first periodic signal is input as training data to the third reservoir computing. 3. The information processing device according to claim 1 or 2.

[0083] (Appendix 4) The fourth reservoir computing receives time series data as training data. 4. The information processing device according to any one of claims 1 to 3.

[0084] (Appendix 5) A signal indicating a prediction error of the first reservoir computing is input as training data to the third reservoir computing. 5. The information processing device according to any one of appendices 1 to 4.

[0085] (Appendix 6) At least one of the first reservoir computing, the second reservoir computing, the third reservoir computing, and the fourth reservoir computing includes a circuit network of nonlinear elements as a reservoir. 6. The information processing device according to any one of Supplementary Notes 1 to 5.

[0086] (Appendix 7) a first data processing unit having a first reservoir computing arranged in a forward direction and a second reservoir computing arranged in a reverse direction to the first reservoir computing; a second data processing unit having a third reservoir computing arranged in the forward direction and a fourth reservoir computing arranged in the reverse direction; In an information processing device comprising: A process of predicting first time series data in the first data processing unit; a process of predicting second time series data that occurs less frequently than the first time series data, in the second data processing unit; a process in which, when data that cannot be predicted by the first data processing unit is input to the first data processing unit, the second data processing unit generates a signal indicating a prediction error of the first data processing unit and makes a prediction based on the signal; An information processing method comprising:

[0087] (Appendix 8) a process in the first data processing unit inputting an output of the first reservoir computing into the second reservoir computing; a process of generating, in the second data processing unit, a signal indicating a prediction error of the first reservoir computing as the signal, inputting the signal to the third reservoir computing, and inputting an output of the third reservoir computing to the fourth reservoir computing; 8. The information processing method according to claim 7, comprising:

[0088] (Appendix 9) inputting a first periodic signal as training data into the first reservoir computing; a process of inputting a second periodic signal different from the first periodic signal as training data into the third reservoir computing; 9. The information processing method according to claim 7 or 8, comprising:

[0089] (Appendix 10) A process of inputting time series data as training data into the fourth reservoir computing 10. The information processing method according to any one of Supplementary Notes 7 to 9, comprising:

[0090] (Appendix 11) A signal indicating a prediction error of the first reservoir computing is input as training data to the third reservoir computing. 11. The information processing method according to any one of appendices 7 to 10,

[0091] (Appendix 12) At least one of the first reservoir computing, the second reservoir computing, the third reservoir computing, and the fourth reservoir computing includes a circuit network of nonlinear elements as a reservoir. 12. The information processing method according to any one of Supplementary Notes 7 to 11,

[0092] (Appendix 13) a first data processing unit having a first reservoir computing arranged in a forward direction and a second reservoir computing arranged in a reverse direction to the first reservoir computing; a second data processing unit having a third reservoir computing arranged in the forward direction and a fourth reservoir computing arranged in the reverse direction; In an information processing device comprising: In the first data processing unit, first time series data is predicted; In the second data processing unit, second time series data having a lower occurrence frequency than the first time series data is predicted; When data that cannot be predicted by the first data processing unit is input to the first data processing unit, the second data processing unit generates a signal indicating a prediction error of the first data processing unit and performs prediction based on the signal. An information processing program that causes a processor to execute a process.

[0093] (Appendix 14) In the first data processing unit, an output of the first reservoir computing is input to the second reservoir computing; In the second data processing unit, a signal indicating a prediction error of the first reservoir computing is generated as the signal, the signal is input to the third reservoir computing, and an output of the third reservoir computing is input to the fourth reservoir computing. 14. The information processing program according to claim 13, which causes the processor to execute a process.

[0094] (Appendix 15) inputting a first periodic signal as training data into the first reservoir computing; A second periodic signal different from the first periodic signal is input as training data to the third reservoir computing. 15. The information processing program according to claim 13 or 14, which causes the processor to execute a process.

[0095] (Appendix 16) Input the time series data as training data into the fourth reservoir computing. 16. The information processing program according to any one of appendices 13 to 15, which causes the processor to execute a process.

[0096] (Appendix 17) A signal indicating a prediction error of the first reservoir computing is input as training data to the third reservoir computing. 17. The information processing program according to any one of appendices 13 to 16, which causes the processor to execute a process.

[0097] (Appendix 18) At least one of the first reservoir computing, the second reservoir computing, the third reservoir computing, and the fourth reservoir computing includes a circuit network of nonlinear elements as a reservoir. 18. The information processing program according to any one of appendices 13 to 17, [Explanation of symbols]

[0098] 1. Computer Systems 10. Information processing equipment 11 processors 12 Memory 13 Storage device 14 Graphics Processing Unit 14a Monitor 15 Input Interface 15a keyboard 15b Mouse 16 Optical drive device 16a Optical disc 17 Device connection interface 17a Memory Device 17b Memory reader / writer 17c memory card 18 Network Interface 18a Network 19 Bus 20-1~20-4, 20 Reservoir Computing 101 Management Department 201 Reservoir 202 Input Layer 203 Output Layer 2031 Output weight calculation unit 211 First Data Processing Unit 212 Second Data Processing Section

Claims

1. A system comprising a first reservoir computing, a second reservoir computing, a third reservoir computing, and a fourth reservoir computing, each having an input layer, a reservoir, and an output layer; The first reservoir computing, the third reservoir computing, the fourth reservoir computing, and the second reservoir computing are connected in series in this order, and a first data processing unit having the first reservoir computing and the second reservoir computing; a second data processing unit having the third reservoir computing and the fourth reservoir computing; First periodic data is provided as training data to the first reservoir computing, and second periodic data having a longer period than the first periodic data is provided as training data to the third reservoir computing, In the first data processing unit, time series data is input to the first reservoir computing, the first reservoir computing outputs first output data, and the first output data and fourth output data from the fourth reservoir computing are input to the second reservoir computing; In the second data processing unit, a prediction error signal indicating a difference between the first output data and the first periodic data is input to the third reservoir computing, third output data from the third reservoir computing is input to the fourth reservoir computing, the prediction error signal is provided to the fourth reservoir computing as teacher data, and the fourth reservoir computing outputs the fourth output data based on the third output data and the prediction error signal; In the first data processing unit, the second reservoir computing outputs second output data based on the fourth output data and the first output data.

1. An information processing device comprising:

2. At least one of the first reservoir computing, the second reservoir computing, the third reservoir computing, and the fourth reservoir computing comprises a circuit network of nonlinear elements as a reservoir.

2. The information processing apparatus according to claim 1, wherein:

3. A first reservoir computing, a second reservoir computing, a third reservoir computing, and a fourth reservoir computing, each having an input layer, a reservoir, and an output layer; The first reservoir computing, the third reservoir computing, the fourth reservoir computing, and the second reservoir computing are connected in series in this order, and a first data processing unit having the first reservoir computing and the second reservoir computing; a second data processing unit having the third reservoir computing and the fourth reservoir computing; In an information processing device comprising: First periodic data is provided as training data to the first reservoir computing, and second periodic data having a longer period than the first periodic data is provided as training data to the third reservoir computing, a process in which, in the first data processing unit, time series data is input to the first reservoir computing, the first reservoir computing outputs first output data, and the first output data and fourth output data from the fourth reservoir computing are input to the second reservoir computing; a process in which, in the second data processing unit, a prediction error signal indicating a difference between the first output data and the first periodic data is input to the third reservoir computing, third output data from the third reservoir computing is input to the fourth reservoir computing, the prediction error signal is provided to the fourth reservoir computing as teacher data, and the fourth reservoir computing outputs the fourth output data based on the third output data and the prediction error signal; in the first data processing unit, the second reservoir computing outputs second output data based on the fourth output data and the first output data; An information processing method comprising:

4. A first reservoir computing, a second reservoir computing, a third reservoir computing, and a fourth reservoir computing, each having an input layer, a reservoir, and an output layer; The first reservoir computing, the third reservoir computing, the fourth reservoir computing, and the second reservoir computing are connected in series in this order, and a first data processing unit having the first reservoir computing and the second reservoir computing; a second data processing unit having the third reservoir computing and the fourth reservoir computing; In an information processing device comprising: First periodic data is provided as training data to the first reservoir computing, and second periodic data having a longer period than the first periodic data is provided as training data to the third reservoir computing, In the first data processing unit, time series data is input to the first reservoir computing, the first reservoir computing outputs first output data, and the first output data and fourth output data from the fourth reservoir computing are input to the second reservoir computing; In the second data processing unit, a prediction error signal indicating a difference between the first output data and the first periodic data is input to the third reservoir computing, third output data from the third reservoir computing is input to the fourth reservoir computing, the prediction error signal is provided to the fourth reservoir computing as teacher data, and the fourth reservoir computing outputs the fourth output data based on the third output data and the prediction error signal; In the first data processing unit, the second reservoir computing outputs second output data based on the fourth output data and the first output data. An information processing program that causes a processor to execute a process.

Images