Deep echo state network system based on local active memristors
By utilizing a deep echo state network system with local active memristors, time-division multiplexing and linear connections are employed to address the shortcomings of traditional systems in multi-scale time series prediction, achieving efficient deep network design and improving system performance.
Patent Information
- Application Number
- CN202310489920.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-04
- Publication Date
- 2025-12-02
- Estimated Expiration
- 2043-05-04
AI Technical Summary
Traditional echo-state network systems based on local active memristors are not suitable for multi-scale time series prediction tasks, while deep echo-state networks result in too many reservoir nodes and an excessively deep network, which affects system performance.
A deep echo state network system based on local active memristors is adopted. Through the design of input layer, reservoir, training pool and output layer, the nonlinear characteristics and masking mechanism of local active memristors are used to realize time division multiplexing and linear connection, reduce the number of reservoirs and nodes, and the deep network can be realized with only a single local active memristor.
It enables multi-scale time series forecasting tasks, improves system performance, reduces the number of storage pools and nodes, and enhances network efficiency.
Smart Images
Figure CN116502688B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of time series prediction technology, and in particular to a deep echo state network system based on local active memristors. Background Technology
[0002] Echo-state networks (ERNs) have been widely applied in nonlinear system recognition, speech recognition, and time series prediction with good results. Many novel materials and components, including spintronic oscillators and locally active memristors, are being researched for hardware implementation of ERNs to improve system efficiency. Traditional ERNs-based ERNs require a read signal after each input signal to read the device conductance, and single-layer networks are unsuitable for multi-scale time series prediction tasks, limiting their application. Furthermore, deep ERNs can lead to an excessive number of reservoir nodes and an overly deep network for complex time series prediction tasks, negatively impacting system performance.
[0003] The drawbacks of existing technologies are that traditional echo-state network systems based on local active memristors are not suitable for multi-scale time series prediction tasks, while deep echo-state networks can lead to too many reservoir nodes and an excessively deep network for some complex time series prediction tasks, thus affecting system performance. Summary of the Invention
[0004] The main objective of this invention is to provide a deep echo state network system based on a locally active memristor, which enables the echo state network system to perform multi-scale time series prediction tasks, improves system performance, and requires only a single locally active memristor to realize the deep network.
[0005] To achieve the above objectives, the specific technical solution of the present invention is as follows:
[0006] A deep echo state network system based on a local active memristor includes an input layer. The input layer collects vector data sets with time-series attributes and transmits these vector data sets to a first storage pool and a training pool. The output of the storage pool is connected to the training pool, and the training pool outputs the training result W. out The key to the output layer is that the input layer is connected to the first of the n storage pools, and the n storage pools have the same structure, with the first and second pools connected sequentially.
[0007] The reservoir contains a mask and a local active memristor. The mask in the first reservoir is connected to the input layer. The front end of the local active memristor is connected to the mask, and the rear end is connected to ground via a load resistor. The input in the subsequent reservoir is the voltage value between the load resistors of the preceding stage, and the mask in the subsequent reservoir acquires this voltage value. The differences between the different reservoirs are determined by the different masks used.
[0008] Specifically, the input layer acquires vector data sets with time-series attributes and normalizes these data sets to eliminate the adverse effects caused by outlier samples. The input layer then transmits the normalized vector data sets to a first buffer pool. A mask in the buffer pool performs time-division multiplexing on the vector data sets, converting them into time signals. These time signals then pass through a local active memristor. Because the local active memristor itself has non-linear characteristics, passing through it is equivalent to passing through the local active memristor's value w. i The system utilizes the inherent variations of the local active memristor to generate different reservoir states. The current value output by the local active memristor is the reservoir state value. To facilitate the reading of the reservoir state value, a resistor is connected in series after the local active memristor. This resistor is used to convert the output current of the local active memristor into a voltage signal, which is sampled and then used as the reservoir state. The output vector is a linear combination of the reservoir states.
[0009] Furthermore, the locally active memristor model, with its chaotic edges representing the optimal operating points of the ESN, possesses more complex and richer dynamic characteristics, making it more conducive to the implementation of echo state network systems. Meanwhile, the deep structure can obtain a richer reservoir of states, exhibiting good performance in complex time series prediction tasks.
[0010] As a preferred embodiment, the output state expression of the local active memristor in the first storage pool is:
[0011]
[0012] Where, a0 = -1.55, a1 = -1, a2 = 6, b0 = -3, b1 = -1, b2 = 6, b3 = 0.9, b4 =
[0013] 0.076, x is the initial derivative of the local active memristor, x0 = -3.8, v is the input voltage, and i is the output current;
[0014] All the local active memristors in the reservoir are of the same model and all use voltage signals as input. Therefore, this system only needs to use one local active memristor to realize a deep network.
[0015] As a preferred approach, the concept of time multiplexing is introduced, and a masking process is used to generate virtual nodes in the time domain. The state values of the virtual nodes are as follows:
[0016]
[0017] Among them, w iLet τ be the current value obtained through the local active memristor, x be the state value of the virtual node after passing through the mask, N be the total number of virtual nodes in the reservoir, Rt be the resistance value, and X(t) be the state value obtained in the final reservoir.
[0018] Preferably, the state matrix M in the training pool is expressed as:
[0019]
[0020] Where s represents the number of training samples, x1(t)-x N (t) represents the state value of different virtual nodes, which is related to time t;
[0021] The ridge regression method is used to solve for the output connection weight matrix W obtained during training. out The solution process is as follows:
[0022] W out =Y terget M T (MM T +λI) -1
[0023] Among them, Y terget The true values are used for training, λ represents the regularization coefficient, I is an identity matrix, the symbol "T" represents matrix transpose, and the symbol "-1" represents matrix inversion.
[0024] Preferably, the expression for the prediction result y in the output layer is:
[0025] y = W out M.
[0026] As a preferred embodiment, changing the mask time parameter in the reservoir can achieve time-division multiplexing, realize linear connection between reservoirs at each level, and realize differentiation between reservoirs at each level. This can obtain a richer reservoir state, and a deep network can be realized with only a single local active memristor.
[0027] Preferably, the vector data set collected by the input layer is either a vector set of temperature changing over time; or a vector set of voltage changing over time; or a vector set of speed changing over time; or a vector set of current changing over time; or a vector set of pressure changing over time.
[0028] The input layer normalizes the time-varying vector group described above, transforming it into a voltage-time-varying vector group, and then transmits it to the first storage pool.
[0029] The significant advantages of this invention are: the system greatly reduces the number of reservoirs and nodes, improves network performance, replaces the multi-connection node structure of the reservoir with a local active memristor system composed of single nodes affected by delay feedback, is suitable for multi-scale time series prediction tasks, and the implementation of deep networks only requires a single local active memristor. Attached Figure Description
[0030] Figure 1 This is a schematic diagram of the structure of the present invention;
[0031] Figure 2 This is a graph showing the prediction results of sunspot time series in Example 2;
[0032] Figure 3 A graph showing the relationship between mask length ML and number of layers (N) when the size of the reservoir is fixed. Detailed Implementation
[0033] The present invention will be further described in detail below with reference to the accompanying drawings and specific examples:
[0034] Example 1:
[0035] like Figure 1 The diagram illustrates a deep echo state network system based on a local active memristor, comprising an input layer that collects vector data sets with time-series attributes and transmits these data sets to a first storage pool and a training pool. The output of the storage pool is connected to the training pool, which outputs the training result W. out The key to the output layer is that the input layer is connected to the first of the n storage pools, and the n storage pools have the same structure, with the first and second pools connected sequentially.
[0036] The reservoir contains a mask and a local active memristor. The mask in the first reservoir is connected to the input layer. The front end of the local active memristor is connected to the mask, and the rear end is connected to ground via a load resistor. The input to the subsequent reservoir is the voltage value between the load resistors of the preceding reservoir; that is, the state value of the preceding reservoir is the input to the subsequent reservoir. The differences between different reservoirs are determined by different masks.
[0037] Its specific work details are as follows:
[0038] The input layer acquires vector data sets with time-series attributes and normalizes them into vector data sets showing voltage variations over time. This processed vector data set is then transmitted to the first storage pool. Inside the storage pool, the vector data sets undergo time-division multiplexing via a mask and are converted into a time signal. This time signal then passes through a local active memristor to obtain current, and then through a resistor to obtain a voltage value, which represents the storage pool's state value. The voltage value of the preceding storage pool serves as the input to the subsequent storage pool. The subsequent storage pool repeats the above steps, using mask parameters to achieve time-division multiplexing and realize linear connections between the various storage pools.
[0039] The state values output from n reservoirs form a reservoir state value set X(t), which enters the training pool. After forming a state matrix M through simple linear regression training, the output connection weight matrix W obtained through training is solved using ridge regression. out And transmit it to the output layer. The output layer uses the trained W... out The prediction result y can be obtained by multiplying it with the state matrix M.
[0040] Specifically, the output state expression of the local active memristor in the first storage pool is:
[0041]
[0042] Where a0 = -1.55, a1 = -1, a2 = 6, b0 = -3, b1 = -1, b2 = 6, b3 = 0.9, b4 = 0.076, x is the initial derivative of the local active memristor, x0 = -3.8, v is the input voltage, and i is the output current;
[0043] All the local active memristors in the reservoir are of the same model and all use voltage signals as input. Therefore, this system only needs to use one local active memristor to realize a deep network.
[0044] The concept of time multiplexing is introduced, and a masking process is used to generate virtual nodes in the time domain. The state values of the virtual nodes are as follows:
[0045]
[0046] Among them, w i Let τ be the current value obtained through the local active memristor, x be the state value of the virtual node after passing through the mask, N be the total number of virtual nodes in the reservoir, Rt be the resistance value, and X(t) be the state value obtained in the final reservoir.
[0047] The state matrix M in the training pool is expressed as:
[0048]
[0049] Where s represents the number of training samples, x1(t)-x N (t) represents the state value of different virtual nodes, which is related to time t;
[0050] The ridge regression method is used to solve for the output connection weight matrix W obtained during training. out The solution process is as follows:
[0051] W out =Y terget M T (MM T +λI) -1
[0052] Among them, Y terget The true values are used for training, λ represents the regularization coefficient, I is an identity matrix, the symbol "T" represents matrix transpose, and the symbol "-1" represents matrix inversion.
[0053] The expression for the prediction result y in the output layer is:
[0054] y = W out M.
[0055] In the reservoir, changing the mask time parameter achieves time-division multiplexing, enabling linear connection between reservoirs at different levels.
[0056] The vector data set collected by the input layer may be a vector set of temperature changing over time; or a vector set of voltage changing over time; or a vector set of speed changing over time; or a vector set of current changing over time; or a vector set of pressure changing over time.
[0057] The input layer normalizes the vector group of temperature, voltage, speed, current, or pressure that changes over time, transforming it into a vector group of voltage that changes over time, and then transmits it to the first storage pool.
[0058] Example 2:
[0059] Based on Example 1, this example performs time series prediction of sunspots in LAM-D-ESN and Deep-ESN networks with different time steps, respectively. The prediction results are as follows: Figure 2 As shown.
[0060] Figure 2 In this context, Original Sunspot Date refers to the original sunspot date, LAM-D-ESN refers to this system, and Deep-ESN refers to the deep echo state network. Figure 2It can be seen that the LAM-D-ESN deep echo state network system based on local active memristors is significantly better than the Deep-ESN deep echo state network in terms of time series prediction.
[0061] The evaluation metric here is the normalized mean square error (NMSE):
[0062]
[0063] Here, y(t) represents the expected output, o(t) represents the actual output, and σ 2 is the variance of the expected output, and i is the total length of the output.
[0064] Table 1 shows the NMSE of Deep-ESN and LAM-D-ESN at different reservoir levels: where the number of nodes in each reservoir of D-ESN is 500. Figure 3 To establish the relationship between mask length ML and number of levels (N) when the reservoir size is fixed, it can be seen that LAM-D-ESN can achieve good prediction results with a small reservoir size.
[0065] Table 1: NMSE of Deep-ESN and LAM-D-ESN at different numbers of layers
[0066]
Claims
1. A deep echo state network system based on a local active memristor, comprising an input layer, wherein the input layer acquires vector data sets with time-series attributes and transmits the vector data sets to a first storage pool and a training pool, the output of the storage pool is connected to the training pool, and the training pool outputs training results W. out For the output layer, the characteristic is: The input layer is connected to the first of the n reserve pools, and the n reserve pools have the same structure and are connected end to end in sequence; The reservoir contains a mask and a local active memristor. The mask in the first reservoir is connected to the input layer. The front end of the local active memristor is connected to the mask, and the rear end is connected to ground after a load resistor. The input in the subsequent reservoir is the voltage value between the load resistors of the preceding stage.
2. The deep echo state network system based on local active memristors according to claim 1, characterized in that: The output state expression of the local active memristor in the first reservoir is: Where a0 = -1.55, a1 = -1, a2 = 6, b0 = -3, b1 = -1, b2 = 6, b3 = 0.9, b4 = 0.076, x is the initial derivative x0 of the local active memristor, v is the input voltage, and i is the output current; All the local active memristors in the reservoirs are of the same model and all use voltage signals as input.
3. The deep echo state network system based on local active memristors according to claim 2, characterized in that: Virtual nodes are generated in the time domain using a masking process, and the state values of the virtual nodes are as follows: Among them, w i Let τ be the current value obtained through the local active memristor, x be the state value of the virtual node after passing through the mask, N be the total number of virtual nodes in the reservoir, Rt be the resistance value, and X(t) be the state value obtained in the final reservoir.
4. The deep echo state network system based on local active memristors according to claim 1, characterized in that: The state matrix M in the training pool is expressed as: Where s represents the number of training samples, x1(t)-x N (t) represents the state value of different virtual nodes, which is related to time t; The ridge regression method is used to solve for the output connection weight matrix W obtained during training. out The solution process is as follows: W out =Y terget M T (MM T +λI) -1 Among them, Y terget The true values are used for training, λ represents the regularization coefficient, I is an identity matrix, the symbol "T" represents matrix transpose, and the symbol "-1" represents matrix inversion.
5. The deep echo state network system based on local active memristors according to claim 4, characterized in that: The expression for the prediction result y in the output layer is: y=W out M。 6. The deep echo state network system based on local active memristors according to claim 1, characterized in that: In the reservoir, changing the mask time parameter achieves time-division multiplexing, enabling linear connection between reservoirs at different levels.
7. The deep echo state network system based on local active memristors according to claim 1, characterized in that: The vector data set collected by the input layer may be a vector set of temperature changing over time; or a vector set of voltage changing over time; or a vector set of speed changing over time; or a vector set of current changing over time; or a vector set of pressure changing over time. The input layer normalizes the vector group of temperature, voltage, speed, current, or pressure that changes over time, transforming it into a vector group of voltage that changes over time, and then transmits it to the first storage pool.