Reservoir Computing Node Weighting and Nonlinearity Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Recurrent neural networks (RNNs) face challenges in achieving high computing performance and signal identification accuracy, particularly when tasks require either linear or nonlinear transformations, as existing reservoir computing systems often excel in one but struggle with the other.
Innovation Solution
A computing system with a reservoir unit where nodes are circularly connected, with weights between closer nodes being larger than those between more distant nodes, and a nonlinearity parameter (g value) that periodically changes, allowing for flexible nonlinearity adjustment in the activation function.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If nonlinear transformation is employed for node computing processing in reservoir computing, then storage capability for past signal information is improved, but signal identification accuracy for tasks requiring linear transformation deteriorates
Solution Approach 1:
The patent applies local quality by assigning different nonlinearity parameters (g values) to different nodes in the reservoir. Specifically, nodes are arranged in a circular pattern and assigned g values that periodically change around the circle, creating local variations in nonlinearity. This allows certain regions to provide nonlinear transformation for storage capability while other regions maintain linearity for accurate signal identification, thereby resolving the trade-off between storage capability and signal identification accuracy.
2Measurement precision
If general RNN weights are learned between all nodes, then identification performance is improved, but computing cost exponentially increases
Solution Approach 1:
The patent extracts the learning process from the reservoir layer by fixing the weights between nodes in the reservoir to predetermined values. Only the output layer weights are learned, which dramatically reduces the number of parameters to be optimized. This extraction of the learning requirement from the main computational body resolves the contradiction between identification performance and computing cost by maintaining performance while eliminating exponential computing cost growth.
3Speed
If reservoir computing with fixed weights is used, then processing speed is improved, but adaptability to different signal conversion characteristics deteriorates
Solution Approach 1:
The patent introduces dynamics by making the nonlinearity parameter (g value) vary periodically around the circular arrangement of nodes. This dynamic variation in nonlinearity across different node locations enables the fixed-weight reservoir to adapt to different signal conversion characteristics. The periodic change in g values allows the system to handle both linear and nonlinear transformation tasks effectively, resolving the contradiction between processing speed and adaptability.
Data Source
AI summary
A technology that can enhance the computing performance of a computing system using reservoir computing (RC), includes a computing system which performs computation using a recurrent neural network (RNN) including an input unit, a reservoir unit, and an output unit. The reservoir unit includes a plurality of nodes circularly connected to each other. The circular connection has a weight matrix for determining a weight between the nodes of the plurality of nodes, in which a weight between the nodes closely arranged on the circle is larger than a weight between the nodes arranged away from each other on the circle. The plurality of nodes each have a g value that is a parameter for designating nonlinearity of an activation function of each of the nodes, and that is set so as to periodically change in a direction on the circle.