Machine Learning Systems

Abstract

To provide a machine learning system in which an instrument to be learned and a learning instrument are consistently represented in a seamless nonlinear dynamical system in order to achieve machine learning with a high degree of autonomy.SOLUTION: A machine learning system, which is reservoir computing, includes a reservoir with an echo state neural network configuration, a readout that transforms the internal state of the reservoir, and a learning algorithm. The learning algorithm is utilized as a reservoir by utilizing its high dimensionality and nonlinearity to approximate nonlinear input-output relationships, and the reservoir replaces algorithmic processing of the learning algorithm and readout.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a machine learning system, and more particularly to a machine learning system for reservoir computing (hereinafter referred to as RC). [Background technology]

[0002] In recent years, research and development into machine learning, a core technology in artificial intelligence, has been vigorously pursued, and the technology has made remarkable progress. For limited tasks, computational models that are equivalent to or even superior to humans have been realized, and it is expected that machine learning technology will continue to advance and develop, and its scope of application will continue to expand.

[0003] One of the ultimate goals in the development of artificial intelligence technology is the construction of machine learning systems that achieve a high degree of autonomy, similar to that of biological intelligence. The learning function in biological intelligence is constant, allowing it to operate while maintaining a certain degree of independence from external intervention (self-identity). Furthermore, it is capable of constantly learning through interaction with the environment, thereby enabling it to utilize its learning function to generate adaptive behaviors even in various unanticipated and unknown situations (adaptability). Furthermore, the learning function in biological intelligence includes a self-sufficient structure. In other words, the learning function in biological intelligence includes physical entities (e.g., brains, sensory organs, muscles, etc.), and these structures function independently of the physical entities.

[0004] In this way, the learning function in biological intelligence realizes a high degree of autonomy because it includes a self-sufficient structure that combines identity and adaptability. As a result, it achieves the flexible behavior of living organisms in the real world. However, many current machine learning methods lack autonomy compared to the learning function in biological intelligence, and often fail to achieve the flexible behavior of biological intelligence.

[0005] One of the reasons for this is that in most current machine learning systems, control of learning is left to the designer, and spontaneous learning is not achieved. For example, in most machine learning systems, adjustment of learning parameters is performed only in the "learning phase" where a learning model is constructed, and not in the "evaluation phase" where model performance is evaluated. As a result, most current machine learning systems are unable to achieve spontaneous learning, and their adaptability to constantly learn and self-identity to constantly maintain learning functions are weaker than the learning functions of biological intelligence.

[0006] Another factor is that most machine learning systems require external designers to prepare the training data in advance. As a result, most machine learning systems are unable to select what to learn ("learn on their own") in response to a time-changing environment, and therefore are unable to achieve flexible behavior. In addition, because the designers must prepare the training data in advance, the scope and tasks of learning are limited, resulting in poor versatility.

[0007] Another factor is that most machine learning systems do not incorporate the learning mechanism itself and are unable to achieve learning functions on their own, as is the case with physical entities in biological intelligence. For example, machine learning systems have a separate structure: the learner (optimizer), in which the learning algorithm adjusts the parameters of the learning model, and the learner (optimizer). In other words, the learner mechanism is installed separately from the learning model, and the learner cannot learn on its own. As mentioned above, this is due to the fact that the learning function is not required in the evaluation phase. While this structure is convenient for switching between using and not using the learning algorithm, it can also be said to be a structure that is not self-sufficient. As will be discussed later, this is a more prominent issue in recurrent neural networks (RNNs), which utilize physical systems.

[0008] Thus, compared to the learning functions of biological intelligence, current machine learning has low self-identity, low adaptability, and lacks self-sufficiency, and therefore its learning function lacks autonomy. As a result, flexible behavior like that of living organisms is not achieved, or the achievement of flexible behavior is limited to certain tasks. Therefore, from the perspective of achieving flexible behavior equivalent to or better than that of living organisms, machine learning is required to have an autonomous learning function.

[0009] Considering the above, in order for a machine learning system to achieve a high level of autonomy, it is required that it has the spontaneity to constantly progress in learning through an internal mechanism, the ability to self-determine and compose the learning subject to determine and compose what should be learned, and that the learning mechanism be contained within it.

[0010] The separation of the learner and the learner in such machine learning systems becomes a more prominent issue in the context of physics-based RC (PRC). PRC is a type of RC that utilizes a physical system (such as an electronic system or an optical system) as a reservoir. It generally includes a physical system, a sensor that acquires the input response of the physical system, and a readout that processes and integrates the information from the sensor. PRC with this configuration utilizes the characteristics of the physical system and has advantages such as low power consumption and high speed. However, because the learning unit is processed by a computer separate from the physical system, its performance is ultimately limited by the fundamental constraints of the computer.

[0011] RC is a suitable mathematical model for realizing machine learning mechanisms in physical systems. However, one of its challenges is that RC does not include a built-in learning mechanism, and the learner and the learner are separated. More specifically, RC employs a learning algorithm called recursive least squares (RLS), but the mechanism that implements this learning algorithm is configured independently as an external mechanism, and is not a self-sufficient structure.

[0012] FIG. 1 is a diagram schematically illustrating the configuration of a conventional RC-based machine learning system 10. Here, as an example, the machine learning system 10 includes RC in which an echo state network (hereinafter referred to as ESN) is applied to an RNN model, and the learning algorithm is depicted as being in the form of an RLS algorithm, as described above. As shown in FIG. 1, the machine learning system 10 includes a learner 11 and a learner 12. The learner 11 further includes a lead-in 110 having an ESN configuration that maps input to a high-dimensional space, a reservoir 111 in which the converted information circulates via a recursive internal connection, and a lead-out 112 that converts the internal state of the reservoir 111.

[0013] ESN is generally known as a representative model of RC. It uses an RNN with fixed connection weights as a reservoir to create a state (internal state) in which past information from the time series input is reflected and remains, and then reads out the input characteristics from there.

[0014] In the machine learning system 10 having such a configuration, in the learner 11, an input u(t) at any time t is input to each layer of the reservoir 111 and is recursively processed within the reservoir 111. Then, the internal state x(t) generated in each layer of the reservoir 111 is input to the readout 112, and an output y(t) is generated based on the internal state x(t) and the parameter w(t). Meanwhile, in the learner 12, a learning model is constructed based on the RLS algorithm as described above. This learning process in the learner 12 is implemented separately and independently from the ESN in the reservoir 111 described above.

[0015] A key feature of this system is that the lead-in 110 and reservoir 111 are generated randomly and do not undergo learning. Therefore, within the framework of the PRC described above, this random conversion circuit can be replaced with a physical system. For example, configuring it with an optical circuit enables high-speed, low-power calculations that take advantage of the parallelism of light (see, for example, Non-Patent Document 1). With this configuration, calculations during forward propagation shown by the learner 11 in Figure 1 can be performed extremely quickly, but learning shown by the learner 12 in Figure 1 is still processed by a computer separate from the physical system, and as a result, its performance is limited by the fundamental constraints of the computer.

[0016] If the separation of the learner and the learner in PRC could be replaced with a method that consistently represents it within a seamless nonlinear dynamical system, the physical architecture would be free from the fundamental constraints of computers, and dramatic improvements in performance could be achieved.

[0017] Based on the above, we have developed a highly autonomous machine learning method and system by extending RC to a form capable of autonomous learning, which makes it possible to embed a learning mechanism into the physical system of PRC. [Prior art documents] [Non-patent literature]

[0018] [Non-Patent Document 1] M. Nakajima et al., “Scalable reservoir computing on coherent linear photonic processor”, Commun. Phys., 4, 20, 2021 [Non-patent document 2] Eduardo Izquierdo-Torres and Inman Harvey. “Hebbian learning using fixed weight evolved dynamical neural'networks”. In 2007 IEEE Symposium on Artificial Life, pp. 394-401. IEEE, 2007. [Non-patent document 3] Christian Klos, Yaroslav Felipe Kalle Kossio, Sven Goedeke, Aditya Gilra, and RaoulMartin Memmesheimer. “Dynamical learning of dynamics”. Physical Review Letters, Vol. 125, No. 8, p. 088103, 2020. [Non-patent document 4] Guillaume Bellec, Franz Scherr, Elias Hajek, Darjan Salaj, Robert Legenstein, and Wolfgang Maass. “Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets”. arXiv preprint arXiv:1901.09049, 2019. Summary of the Invention [Problem to be solved by the invention]

[0019] In a conventional RC machine learning system 10, the learner 11 and the learner 12 each have an independent configuration. However, in a highly autonomous learning function, learning must be continuous to ensure its adaptability. To achieve this learning process, a method is required to consistently represent the learner 11 and the learner 12 within a seamless nonlinear dynamical system. This issue is particularly important for constructing a learning machine that takes advantage of the advantages of PRC.

[0020] To date, several techniques have been proposed to realize seamless, coherent machine learning systems by representing the learning process as the dynamics of an RNN. For example, Izquierdo-Torres and Harvey used an evolutionary algorithm to construct a continuous-time RNN with fixed weights that performs Hebb's law behavior (see, for example, Non-Patent Document 2). However, such techniques are difficult to implement using evolutionary algorithms, such as those employed in the context of evolutionary robotics. Klos et al. constructed an RNN that can adapt to tasks without adjusting its internal connections by pre-training a closed-loop system so that the RNN autonomously generates variables used during task learning (see, for example, Non-Patent Document 3). Meanwhile, Bellec et al. constructed a spiking neural network that can adapt to new tasks without adjusting interaction parameters by pre-training its internal connections using BPTT (see, for example, Non-Patent Document 4). However, this method of adjusting the internal connections through pre-training an RNN requires the designer to determine the scope of the task in advance. As a result, generalization is partial, and the RLS algorithm itself is not suitable for designing on a neural network while maintaining its versatility. [Means for solving the problem]

[0021] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide a machine learning system in which the learner and the learner are consistently represented within a seamless nonlinear dynamical system in order to realize machine learning with high autonomy.

[0022] To achieve this objective, the present invention provides a machine learning system that is reservoir computing, which comprises a reservoir having an echo state neural network configuration that uses a recurrent neural network with fixed connection weights to create a state in which past information of a time series input is reflected and remains, and from which the input characteristics are read out, a readout that transforms the internal state of the reservoir, and a learning algorithm, in which the learning algorithm is used to approximate a nonlinear input-output relationship, thereby being utilized as a reservoir, and the processing of the learning algorithm and readout is replaced by the reservoir. [Effects of the Invention]

[0023] By applying the machine learning system of the present invention, spontaneous learning is implemented due to its highly autonomous learning function, realizing highly versatile machine learning. Furthermore, in particular in PRC, an external readout mechanism is not required, enabling calculations that are not rate-limited by the configuration. [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 1 is a diagram schematically illustrating the configuration of a conventional RC machine learning system 10. [Figure 2] FIG. 1 is a diagram illustrating a configuration in which the RLS algorithm is regarded as a reservoir in the construction of a machine learning system according to the present invention. [Figure 3] FIG. 1 is a diagram showing a schematic configuration in which formal processing on the RLS algorithm is replaced with an RNN in the construction of a machine learning system according to the present invention. [Figure 4] FIG. 1 is a diagram illustrating the final configuration of a machine learning system according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0025] Various embodiments of the present invention will be described in detail below with reference to the drawings. The same or similar reference numerals indicate the same or similar elements, and redundant description may be omitted. Materials and numerical values ​​are for illustrative purposes only and are not intended to limit the technical scope of the present invention. The following description is an example, and some configurations may be omitted or modified, or additional configurations may be added, as long as they do not deviate from the gist of one embodiment of the present invention.

[0026] Also, in the following description, the machine learning system according to the present invention will be described in a context in which the learner is an RC and the learning algorithm set in the learner is an RLS algorithm, but please note that this is for illustrative purposes only and is not limiting.

[0027] As described above, the machine learning system according to the present invention is an extended version of RC, in which the learner and the learner are consistently represented within a seamless nonlinear dynamical system.

[0028] The principles of the machine learning system according to the present invention will be described in detail below.

[0029] (principle) In the machine learning system 10 shown in FIG. 1, the RC constituting the learner 11 is a dynamical system expressed by (Equation 1) and (Equation 2).

[0030]

number

[0031] where F ESN is the time evolution map of the ESN in the learner 11.

[0032] Meanwhile, the learner 12 implements learning based on the RLS algorithm, as described above. The RLS algorithm is online learning that updates the parameter w(t) whenever data is added, and is configured so that w(t) that minimizes a specified objective function is updated using a recurrence formula that introduces an auxiliary variable P(t). In this calculation method, the RLS algorithm is classified into two types: P(t), which is updated using only the current state x(t) as a factor, and a difference value Δw(t), which is calculated using the teacher trajectory d(t) as a factor. Therefore, in the machine learning system 10, the calculation implemented by the learner 12 can be formulated as shown in (Equation 3) and (Equation 4).

[0033]

number

[0034] In this way, the entire RC learning can be interpreted as a time-evolving system that takes u(t) and d(t) as arguments. In particular, F ESN , F RLS If a model has the echo state property (ESP), the entire RC learning can be regarded as a dynamical system that can be used as a reservoir. In the machine learning system of this invention, we extend RC so that the functionality of such a learner and a learner can be seamlessly expressed in a single dynamical system.

[0035] A dynamical system that seamlessly expresses the functionality of such a learner and a learner can be described as (Equation 5) and (Equation 6) if its state is assumed to be X(t).

[0036]

number

[0037] where F all , G alland represent the mappings that represent the time evolution of the learning process and the output generation mechanism, respectively. In the machine learning system according to the present invention, we use only the nonlinearity tanh of the ESN and design its internal coupling parameters to achieve F all An RNN that realizes this has been constructed. In such a system, when an input u(t) and a teacher trajectory d(t) are given, learning proceeds "autonomously" on the dynamics of the given system. In this system, structures associated with the learning process, such as modules that are usually set up top-down and explicit role assignments, can be represented as a consistent dynamical system, making it possible to analyze them as dynamic structures that emerge autonomously within uniform dynamics.

[0038] (Architecture of the machine learning system according to the present invention) The machine learning system according to the present invention is realized by the following three components. (1) The learning algorithm of the learner (RLS algorithm) is constructed as a reservoir. (2) Replace the formal processing of the learning algorithm (RLS algorithm) with RNN. (3) Integration of (1) and (2) A detailed explanation of each item is provided below.

[0039] (Building the RLS algorithm as a reservoir) FIG. 2 is a diagram showing a schematic diagram of the configuration of the machine learning system according to the present invention, in which the RLS algorithm is regarded as a reservoir. In the machine learning system according to the present invention, first, it is shown that the RLS algorithm satisfies ESP under certain parameter conditions, and then the RLS algorithm itself is regarded as a reservoir and utilized under those conditions. In other words, the high dimensionality and nonlinearity of the RLS algorithm are focused on and utilized to approximate a certain nonlinear input-output relationship. In particular, the machine learning system according to the present invention focuses on the F ESNThis paper focuses on the RLS algorithm and constructs it as a linear closed loop of variables within the RLS algorithm. This closed loop is achieved by supervised learning using sampled trajectories. In the closed loop system constructed in this way, the ESN of the trainee is replaced with a linear closed loop, leaving only the RLS algorithm, resulting in a consistent system. In this specification, this type of system, in which the RLS algorithm is considered as a reservoir, is called an All-In-One system.

[0040] Next, we will explain the reservoir system in such an all-in-one system that is trained based on the RLS algorithm. The entire system in this case is expressed by (Equation 7) to (Equation 11).

[0041]

number

[0042] Here, μ is a forgetting term in the objective function of the RLS algorithm, and is set to be equal to or greater than 0 and equal to or less than 1.

[0043] In the machine learning system of the present invention, as described above, an all-in-one system is constructed that eliminates the separate structure of the learner and the learner. At this time, the RLS algorithm itself is used as a reservoir to approximate the ESN of the learner. As described above, the time evolution map F ESN is a nonlinear mapping with arguments x(t) and u(t). Therefore, F ESN To approximate F, the trajectory of (Equation 8) can be utilized. In other words, in the machine learning system according to the present invention, (Equation 5) is utilized as a reservoir, and F ESN Configure.

[0044] For a dynamical system to be used as a reservoir, it must have ESP, which is the condition for the system to converge to an asymptotic state at a given input, and roughly speaking, requires that each variable can be described as a function of the past series of inputs.

[0045] On the other hand, P(t) is expressed in explicit form as shown in (Equation 12).

[0046]

number

[0047] When a system is used as a reservoir, a certain steady state is assumed. Therefore, it is assumed to be used in a state where t is sufficiently large and time evolves. Here, if μ<1.0, P(t) converges to the value of (Equation 13).

[0048]

number

[0049] Considering (Equation 8), this means that P(t) is a function of the past time series of x(t) and u(t). In addition, since μ<1.0, the influence of past inputs decreases with time evolution. This is F RLS This suggests that the machine learning system according to the present invention employs the RLS algorithm under the condition μ<1.0.

[0050] Next, we will discuss the independent variables that appear in the RLS algorithm. The RLS algorithm is a variable of P(t) written in one line in the form of equation 10), but the algorithm consists of repeated multiplication and addition of variables. Specifically, it is calculated in the order of (Equation 14) to (Equation 24) below.

[0051]

number

[0052] Here, i and j are subscripts, where i is 1 or more and j is N+1 or less.

[0053] What we should pay attention to now is the performance of the RLS algorithm as a reservoir, in other words, the nonlinear transformation of (Equation 8) that appears in the algorithm. In the above equation, the only independent ones in terms of nonlinearity are a ij , b ij , γ, ΔP ij Therefore, these variables are used as reservoirs in the nonlinear transformation that is always generated as a result of the RLS algorithm calculation. In particular, in the machine learning system according to the present invention, the two cases of (Equation 25) are prepared, and the state θ of the RLS algorithm as a reservoir is RLS Define (t).

[0054]

number

[0055] Here, θ RLS The number of dimensions of (t) is (n 2 + n) / 2, for all variables (5n 2 + 5n + 1) / 2.

[0056] Finally, the system shown in Figure 1 is a linear closed-loop w AIO This is achieved by training the model to satisfy (Equation 26).

[0057]

number

[0058] In addition, this lead-out AIO The learning of is achieved using Ridge regression. By replacing the ESN of the trainee with this linear closed loop, an all-in-one system is finally constructed, where the ESN is represented by utilizing the nonlinearity of the RLS algorithm.

[0059] (Replacing formal processing on RLS algorithm with RNN) Figure 3 is a schematic diagram illustrating a configuration in which the formal processing of the RLS algorithm is replaced with an RNN in the construction of a machine learning system according to the present invention. In the machine learning system according to the present invention, the entire algorithmic processing of the RLS algorithm and readout is replaced with an ESN. In particular, this is constructed by utilizing only the nonlinearity of the original ESN. First, the RLS algorithm is decomposed into nonlinear and linear operations. Each nonlinear operation is then approximated using a single-layer neural network called an Extreme Learning Machine (ELM), and the nonlinear operation is replaced with the trained ELM. In this case, the machine learning system according to the present invention uses tanh as the nonlinear function of the ELM. This ELM is trained using linear regression without using backpropagation. In this specification, this operation is referred to as neuralizing. As a result, in the machine learning system according to the present invention, the RLS algorithm can be replaced with a three-layer neural network. By recursively combining these, an RNN equivalent to the algorithmic processing of the RLS algorithm and readout is obtained.

[0060] Specifically, replacing formal processing in the RLS algorithm with an RNN corresponds to constructing an RNN that performs processing equivalent to (Equation 10) and (Equation 11). The RLS algorithm requires a total of three types of nonlinear operations: the multiplication of two variables in (Equation 14), the multiplication of three variables in (Equation 15) and (Equation 19), and the calculation of the reciprocal in (Equation 18). Hereinafter, these are represented as XY, XYZ, and (1+X)-1, respectively. In the machine learning system according to the present invention, these are expressed using tanh, which is the nonlinearity of the ESN. Specifically, an ELM with tanh as the activation function is used, and three types of ELMs, (Equation 27) to (Equation 29), are constructed corresponding to the three types of nonlinear operations.

[0061]

number

[0062] In the machine learning system according to the present invention, an ELM with 20-, 40-, and 50-dimensional hidden layers was constructed for XY, XYZ, and (1+X)-1, respectively.

[0063] In this way, by replacing the algorithms of (Equation 14), (Equation 15), (Equation 18), (Equation 19), (Equation 21), and (Equation 23) with the ELMs expressed by (Equation 27) to (Equation 29), (Equation 10) and (Equation 11) can be expressed as a three-layer neural network as shown in Figure 3. Then, by recursively connecting the resulting neural networks, a huge RNN in which only the nonlinearity of tanh appears can be constructed.

[0064] This process allows us to construct an RNN with readout equivalent to the RLS algorithm without changing the internal connections. Note that the dimensionality of this Neuralized RLS, i.e., the number of independent nonlinear components, is much larger than that of the RLS algorithm due to the hidden layer of the ELM.

[0065] Also, unlike the RLS algorithm, these elements do not have explicit meanings on their own, and only when combined with other elements can they be associated with variables within the RLS algorithm.

[0066] (Integration of All-In-One system and Neuralized RLS) FIG. 4 is a diagram showing a schematic diagram of the final configuration of the machine learning system according to the present invention. By integrating the above-mentioned All-In-One system configuration and the Neuralized RLS configuration, the machine learning system according to the present invention as shown in FIG. 4 is obtained. Specifically, this integration is performed by subtracting θ RLS (t) is the θ ELM (t), and similarly, the linear closed loop w AIO In other words, the machine learning system according to the present invention has a configuration in which (Equation 26) is replaced with (Equation 30).

[0067]

number

[0068] This eliminates the separate structure of the learner and the learner, and realizes an RNN in which the entire RC using the RLS algorithm is realized on its dynamics with only the nonlinearity of tanh.

[0069] Furthermore, the machine learning system of the present invention is constantly changing its internal state even when no input is given. Furthermore, this teacher trajectory is internal, reflecting the internal state of the system, and in that sense, it is a machine learning system with a spontaneous learning function that constructs its own learning object.

[0070] Therefore, it can be said that the machine learning system according to the present invention is a system with higher autonomy than machine learning systems according to conventional techniques (for example, machine learning system 10). [Explanation of symbols]

[0071] 10 Machine Learning Systems 11 Learned device 110 Lead-in 111 Reservoir 112 Leadout 12 Learning Units

Claims

1. A first neural network layer having a neural network configuration in which a time evolution map in an echo state neural network serving as a reservoir constituting a learner is expressed as a linear closed-loop system, the linear closed-loop system being a system that approximates the nonlinear input-output relationship of a learning algorithm applied to a learner for the learner; a second neural network layer having a configuration in which a nonlinear operation in the learning algorithm is approximated by a first trained recurrent neural network, the first trained recurrent neural network being configured using the nonlinear operation in the echo state neural network; a third neural network layer having a configuration in which a nonlinear operation as a readout for reading out input characteristics from a state in which past information of a time series input is reflected and remains in the echo state neural network is approximated by a second trained recurrent neural network; The machine learning system, wherein the first neural network layer, the second neural network layer, and the third neural network layer are recursively connected.

2. The machine learning system of claim 1 , wherein the learning algorithm is a recursive least squares (RLS) algorithm.

3. The machine learning system of claim 1 , wherein the learning algorithm has a forgetting term less than one.

4. The machine learning system of claim 1 , wherein the nonlinear operation is approximated using an Extreme Learning Machine (ELM).

5. The machine learning system according to claim 4 , wherein tanh is used as the nonlinear function of the ELM.

Images