Method for training neural network

Abstract

The present disclosure relates to computer-implemented methods, systems, apparatuses, and non-transitory computer-readable media for training an artificial neural network (ANN). The method includes defining an energy function for the ANN and the dataset from the quantum object, and using a quantum object modulo to simulate the quantum system to reduce the energy function and obtain a trained ANN. The method may also include refining the reduced energy function using the reduced energy function as an input to a genetic algorithm.

Description

Technical Field

[0001] The present disclosure relates to a quantum-inspired method for training neural networks. Background Art

[0002] Fifth-generation (5G) wireless networks are expected to lay the foundation for intelligent networks by providing some independent artificial intelligence (AI) operations. However, it is foreseeable that networks above 5G will benefit from fully intelligent orchestration and management to ensure network performance and an exponential increase in service types. The increasingly stringent performance requirements of these emerging networks are expected to be met by new technologies, among which quantum machine learning (QML) is seen as a core sixth-generation (6G) enabler. In this article, QML refers broadly to the intersection of two disciplines (i.e., machine learning (ML) and quantum mechanics) to achieve any type of computational benefits, such as algorithm acceleration, reduced memory consumption, better quality solutions, etc.

[0003] Although still relatively unknown, QML is a discipline that has been around for about thirty years, but the field is only now really taking off (its development has been hampered primarily by the complexity of the field itself, both in hardware and software, but certainly not for lack of interest).

[0004] In telecommunications, two specific factors are driving the adoption of QML. On the one hand, 6G networks are expected to make heavy use of data from the network itself, and to exploit this data to obtain intelligent and autonomous networks. But on the other hand, Moore's Law has now reached a plateau, and computing hardware is no longer improving significantly. This has therefore motivated more and more practitioners to explore alternatives, including the possibility of leveraging the power of quantum computing to provide advantages for machine learning algorithms. In addition, it has also become apparent recently that current machine learning model technologies and techniques (e.g., neural networks) are beginning to reach their limitations, and new learning methods are now needed. Summary of the invention

[0005] A computer-implemented method for training an artificial neural network (ANN) is provided. The method includes defining an energy function for the ANN and a data set according to a quantum object, and simulating a quantum system using the quantum object to reduce the energy function and obtain a trained ANN.

[0006] A system for training an artificial neural network (ANN) is provided. The system includes a processing circuit and a memory. The memory contains instructions executable by the processing circuit, whereby the system is operable to define an energy function for the ANN and a data set according to a quantum object, and to simulate a quantum system using the quantum object to reduce the energy function and obtain a trained ANN.

[0007] A non-transitory computer-readable medium having stored thereon instructions for training an artificial neural network (ANN), including defining an energy function for the ANN and a data set according to a quantum object, and simulating a quantum system using the quantum object to reduce the energy function and obtain a trained ANN.

[0008] The methods, systems, and non-transitory computer-readable media provided herein provide a new paradigm for training artificial neural networks (ANNs) and bring improvements to the field of ANN training. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a schematic diagram of an artificial neural network.

[0010] Figure 2 is a schematic diagram of an artificial neuron.

[0011] Figure 3 Schematic diagram of the multidimensional energy valley represented by the error function.

[0012] Figure 4a is a flow chart of a method for training an artificial neural network (ANN).

[0013] Figure 4b is an example pseudocode that defines an artificial neural network (ANN).

[0014] Figure 5 and Figure 6 is the linear ( Figure 5 ) and logarithm ( Figure 6 ) scale.

[0015] Figure 7 and Figure 8 is the linearity obtained by training the ANN using the proposed method in the context of the quadratic function test ( Figure 7 ) and logarithm ( Figure 8 ) scale.

[0016] Fig. 9 and Fig.10 is in the context of quadratic function testing, for gradient descent ( Fig. 9 ) and the method proposed in this paper ( Fig.10 ), a plot of the network hyperparameter values ​​as a function of the number of iterations (in arbitrary units).

[0017] Fig.11Shown in the context of a quadratic function test are the quadratic function to be fitted (represented by a dash-dotted line), the curve found by the method proposed in this article (represented by crosses), and the curve found by the gradient descent method (represented by a diamond line).

[0018] Fig.12 and Fig.13 is the linear ( Fig.12 ) and logarithm ( Fig.13 ) scale.

[0019] Fig.14 and Fig.15 is the linearity obtained by training the ANN using the proposed method in the context of the square root function test ( Fig.14 ) and logarithm ( Fig.15 ) scale.

[0020] Fig.16 and Fig.17 is in the context of the square root function test, for the gradient descent method ( Fig.16 ) and the method proposed in this paper ( Fig.17 ), a plot of the network hyperparameter values ​​as a function of the number of iterations (in arbitrary units).

[0021] Fig.18 Shown in the context of a square root function test are the square root function to be fitted (represented by a dashed line), the curve found by the method proposed in this article (represented by a cross), and the curve found by the gradient descent method (represented by a diamond line).

[0022] Fig.19 is a flow chart of a method for training an artificial neural network (ANN).

[0023] Fig. 20 is a schematic diagram of hardware that can perform the steps and / or methods described herein.

[0024] Fig.21 is a schematic diagram of a virtualization environment in which the various methods and apparatus described herein may be deployed. DETAILED DESCRIPTION

[0025] Various features will now be described with reference to the accompanying drawings to fully convey the scope of the present disclosure to those skilled in the art.

[0026] Sequences of actions or functions may be used within the present disclosure. It will be appreciated that in some contexts some functions or actions may be performed by dedicated circuits, by executing program instructions by one or more processors, or by a combination of both.

[0027] Additionally, the computer readable carrier or carrier wave may contain a suitable set of computer instructions that will cause the processor to perform the techniques described herein.

[0028] Function / action described herein may not occur in the order noted in the action sequence, or may occur simultaneously. In addition, in some diagrams, some boxes, functions or actions may be optional, and may or may not be executed; they are usually shown with dotted lines.

[0029] Although quantum machine learning (QML) is still in its infancy, it has the potential to provide practical and novel solutions in several areas of science and technology. It is well known that quantum mechanical systems produce counterintuitive patterns in data. On the other hand, classical machine learning (ML) models usually have the following characteristics: they can both identify statistical patterns in data and generate data with the same statistical patterns. In other words, neural networks can recognize the patterns they produce. Therefore, the above observations suggest that quantum effects can be used to create a new type of artificial neural network that can recognize patterns that are very difficult to recognize with classical methods. This will enable QML to play a very important role in the development of future telecommunication networks.

[0030] Broadly speaking, the field of QML can be seen as a combination of ML, quantum mechanics, and some aspects of quantum computing. Although no clear and generally accepted definition for QML has yet been provided, it seems that the vast majority of the community is moving in the direction of using a representation using actual quantum computing hardware to gain advantages such as quantum speedup and / or reduced memory consumption in ML algorithms. The problem with this approach, however, is that quantum computing technology is not expected to be realized anytime soon, and for good reasons. If quantum effects must be exploited in some way, a different approach is needed (at least at this stage).

[0031] Some of the QML algorithms proposed in this literature are based on the use of typical quantum effects such as entanglement, coherent transport, tunneling, etc. It is well known that such physical systems are extremely difficult to maintain in the real world. In fact, these systems require specific cryogenic facilities to maintain the temperature at absolute zero (i.e., -273.16 degrees Celsius). Moreover, even if such temperatures exist, it is well known that decoherence will eventually come into play and destroy the "quantumness" of the system, turning it into a classical system (thus losing any ultimate quantum advantage). That is, if one wants to specifically exploit quantum states for training neural networks, a different and new approach must definitely be provided.

[0032] The solution proposed in this article targets the training of neural networks, a new type of learning algorithm with a particular focus on artificial neural networks (but exactly the same approach can be applied to different prediction models).

[0033] The solution proposed in this work is efficient and does not exploit any physical quantum system; instead, it is based on the use of commercially available digital computers, thus offering a radically different paradigm. In fact, whenever an artificial neural network (ANN) is provided to be trained on a given data set, it is always possible to design and simulate a corresponding physical system that can perform the training process, reaching its minimum energy point by a simple time evolution; in other words, the minimum energy point reached by this system after some time represents the solution that provides the final weights and biases of the ANN (it is well known in physics that physical systems always evolve in a way that reduces their internal energy).

[0034] The present disclosure proposes to simulate the behavior of quantum systems. In order to make the simulation faster, while still reliable for training ANNs, an approximation based on density functional theory (DFT) known to computational chemists is introduced. This represents a practical and realistic method for obtaining the quantum state corresponding to the minimum energy of the system. It will therefore be shown that this method is able to train machine learning models in an unprecedented way. This in turn opens the way to practical QML without the use of actual quantum computing devices (it is expected that the simulated and measured quantum states are the same or very similar).

[0035] It should be noted that the tools / systems described in this article do not represent quantum optimizers, quantum annealers or anything of that type. Furthermore, the aim of this work is not to gain any quantum advantage in terms of execution speed or memory usage, but to introduce a new type of learning approach based on simplified simulations of physical quantum systems in order to train neural networks in a fundamentally different quantitative and qualitative way. As a result, this approach allows obtaining neural models of a different quality (compared to traditional machine learning approaches) without going back to any actual quantum physical system.

[0036] This paper introduces three main innovations that will be discussed in the following paragraphs.

[0037] It is well known that quantum solvers provide patterns that are difficult (or even impossible) to obtain by classical methods. So far, these states have been obtained by experimental measurements of real physical systems subject to a large number of different problems (e.g. decoherence). The goal of the system / technique proposed in this paper is to provide a practical and realistic way to obtain these quantum patterns. In practice, the proposed method introduces a way to obtain quantum states using approximate digital simulations, which in turn can be used to train ANNs, obtaining qualitatively different models. This represents a significant departure from the paradigm currently explored in QML.

[0038] The way to digitally simulate (many-body) quantum systems is based on a proposed new type of approximation inspired by density functional theory (DFT) of computational chemistry. In the context of QML, this is the first time that this approximation has been introduced in DFT simulations to implement practical and useful aspects of QML (i.e., the training of ANNs).

[0039] When trained by the method proposed in this paper, a different behavior of the ANN is observed, which cannot be mimicked by classical ML training methods (such as, for example, gradient descent). This is because the quantum tunneling effect is exploited during the training phase, which enables relatively fast finding of the relevant quantum states. This represents a completely different and novel paradigm for training ANNs.

[0040] The method proposed in this paper has important advantages in practical applications.

[0041] Although the aim is to find quantum states to train neural networks, the method is not based on the use of any physical quantum system which is known to be expensive and difficult to maintain (e.g. due to increasing decoherence in this system due to temperature and inherent external noise). Therefore, the proposed method does not suffer from any serious problems faced by the QML community planning to use actual physical systems.

[0042] The quantum states necessary to train a given neural network are calculated by running simplified but still accurate simulations of the many-body quantum system on a digital computer. This allows anyone using commercially accessible (i.e., relatively cheap) computers to train the ANN in a more efficient and different way, thus achieving different behaviors of the network.

[0043] The method, although simulated on a digital computer, exploits a typical quantum effect, namely tunneling. This is very important because it is well known that current (classical) learning methods (e.g., gradient descent) can quickly get stuck in energy valleys of the cost / objective / loss function, limiting the training of the ANN to local minima rather than the optimal solution. The proposed method avoids this problem.

[0044] Since it effectively exploits quantum tunneling, it can also be expected that certain neural networks (e.g., recurrent ANNs) that are difficult to train with current learning methods can be successfully trained by the method proposed here.

[0045] Finally, the approach enables realistic QML functionality right away, while the rest of the community is still waiting for the development of quantum computing devices (in the future). Techniques such as the one described in this paper may become important for some aspects of future telecommunication applications (from 5G and beyond).

[0046] ANN training as an optimization problem.

[0047] A neural network (ANN) is a mathematical abstraction of a biological neural network and can be viewed as a collection of connected computational units or artificial neurons, whose connection strengths are represented by numbers called weights. Therefore, the more connections a network has, the more weights it requires. Figure 1 In this context, the feedforward ANN 100 can be viewed as consisting of layers 101 to 104 of neurons 105 that transmit information from one layer to the next, i.e. from the input layer 101 through one or more hidden layers 102 and 103 to the output layer 104. Figure 2 A single artificial neuron 105 is shown.

[0048] refer to Figure 2 , each neuron 105 in the ANN 100 is characterized by a discriminant function 201 and an activation function 202, the activation function 202 acting on the discriminant. In more detail, if a neuron has a set of inputs and a set of weights , then a common choice of discriminant function is the following quantity

[0049] .

[0050] For simplicity, by forcing , embedding the neuron’s bias into the sum. There are many possible choices for the activation function, usually indicated as a general (nonlinear) function A person skilled in the art is aware of activation functions and is able to select an activation function according to a given set of circumstances.

[0051] Once the topology / architecture of the network 100 is defined (i.e., the number of layers, the number of neurons in each layer and their connections, and the discriminant and activation functions of each neuron), any ANN can be mathematically represented as a function of the following type:

[0052]

[0053] in, Indicates input, represents the set of all weights, is the output value calculated by the network (variable and can be a scalar or a vector, depending on the use case, and in this document, vectors are in bold and scalars are in italics, respectively). Given a given set of circumstances, one skilled in the art would know which type of ANN to choose and how to define the topology of the ANN.

[0054] Thus, given some sample set describing the computational goals to be achieved by the network (often referred to as a dataset), where , the problem of training an ANN consists of minimizing an error function (also called loss function, objective function, or cost function), which formally reads as follows:

[0055]

[0056] and depends on the entire set of weights. In practice, the goal is to find the function that minimizes the error Collection There are many different algorithms to achieve this goal, one of the most popular algorithms is the well-known gradient descent method, which is represented by a combination of gradient descent and back propagation. For example, the error function can be represented by an L2 normal form of a certain shape. Those skilled in the art will understand the error function and can choose the error function according to a given set of circumstances.

[0057] For the sake of clarity and completeness, the main principles of training ANNs using gradient descent are presented below.

[0058] One of the simplest training algorithms is gradient descent, sometimes called steepest descent. In the batch version of gradient descent, the initial weight vector is usually random and is given by Then, the weights are updated iteratively so that at step n, the weights move a small distance in the direction of the maximum error decrease rate (i.e., the negative gradient direction). Department Assessment:

[0059] (1)

[0060] This gradient is re-evaluated at each step. In the sequential version of gradient descent, the error function gradient is evaluated in a similar manner for only one mode at a time. In equation (1), the parameters is called the learning rate, and if its value is small enough, then The value of should decrease at each successive step, eventually leading to The weight vector of .

[0061] One of the limitations of the gradient descent technique is the need to choose a learning rate parameter However, the problem with this method does not end there. For example, in the case of a multidimensional weight space, The curvature of may vary significantly with direction. At most points on the error surface, the local gradient does not point to the minimum. Then, gradient descent needs to take many small steps to reach the minimum, which is obviously a very inefficient process. The method proposed in this paper avoids this type of problem.

[0062] We can now introduce a physical interpretation of the gradient descent technique and its quantum counterpart, which in turn will help understand the method proposed in this paper.

[0063] The update rule (1) presented above is reminiscent of classical physics. In fact, it has a strong similarity to the well-known Newtonian formula, which can be expressed mathematically as:

[0064]

[0065] in, The acceleration is The mass of the particle, and in the presence of an applied force In the case of The gradient of ,in, It's the location.

[0066] In more detail, by using It's location The fact that the second derivative is Integrating the formula, we finally get:

[0067] (2)

[0068] in, (Approximation has been introduced , which is a valid assumption for smaller time horizons). Clearly, formula (2) has the same mathematical shape as formula (1) for updating the weights (and biases) of a neural network. Therefore, in a broad sense, gradient descent can be interpreted as a physical system that evolves over time and thereby reduces its internal energy. For example, it can be interpreted as a "multidimensional ball" that falls into a multidimensional energy valley represented by the error function, or as a collection of N (i.e., the same number as the number of weights to be trained in the ANN) one-dimensional balls, each of which falls into a one-dimensional energy valley (see Figure 3 ).

[0069] The transformation of this classical system into its quantum counterpart is to replace the above time-dependent classical equation (2) with its corresponding Schrödinger equation while keeping the same applied potential energy .get:

[0070] (3)

[0071] in, is the reduced Planck constant, and is the Hamiltonian of the system, which is equal to the sum of the kinetic energy operator and the applied potential. Therefore, instead of solving the problem by gradient descent (which in this case roughly corresponds to the simulation of a classical many-body system), it can be solved by simulating the corresponding quantum many-body system. This ultimately allows the presence of quantum tunneling effects in such simulations, which would never exist in a classical context. The tunneling technique in turn allows (and does) to enable a better search for solutions, since it is not affected by the typical problems that affect gradient descent (e.g. the search for solutions does not get locally stuck in potential valleys representing some non-optimal local minima).

[0072] Therefore, for any given ANN with N hyperparameters to be trained, there will be N corresponding Schrödinger equations to be simulated, just as there are N update equations in the gradient descent method. The specific way these Schrödinger equations are coupled is described below.

[0073] In the following section, the discretization scheme used in this document for numerically solving the Schrödinger equation, namely the finite-difference time-dependent method, is proposed.

[0074] The above Schrödinger equation cannot be treated analytically due to the very complex shape of the applied potential (in this specific case, it is represented by the error function corresponding to a given ANN and dataset). Therefore, one must fall back on numerical methods to extract its time-dependent solution (i.e., the wave function ). There are many methods available, and for the purpose of a complete validation process, the well-known finite difference time domain (FDTD) method was chosen because it is relatively simple and accurate. Of course, this choice does not represent a limitation on the method presented in this disclosure, as other methods can be equally utilized, which will be apparent to those skilled in the art.

[0075] The FDTD method was originally a discrete algorithm in the field of computational electrodynamics, but has recently been used in the simulation of quantum mechanical systems. In classical FDTD, Maxwell's equations are discretized on a grid and explicitly solved using a jump integral in time. Since the FDTD algorithm is explicit and local, it can be implemented and parallelized. Then, applying FDTD to solve the Schrodinger equation can produce a method that can be implemented and has high computational efficiency. Although non-uniform grids have been used for electrodynamic FDTD methods and can be used for the Schrodinger equation, in order to make this description as simple as possible, non-uniform grids are not used here, but they can be used in certain embodiments.

[0076] The main principles of the FDTD method applied to the Schrödinger equation are described below. First, the complex-valued wave function is split into real and imaginary parts:

[0077]

[0078] in, and are the real and imaginary parts of the wave function respectively. Then the Schrödinger equation becomes (in one-dimensional space):

[0079]

[0080]

[0081] The time derivatives are then approximated using centered finite differences according to the FDTD method, providing the (finite difference) evolution equations. Evolution equations are equations that describe the time-dependent evolution of a physical system. For example, a ball falling in a gravitational field can be described by Newton's equations, a black hole by Einstein's equations, and so on. Due to the explicit discretization of the time derivatives, there is an upper limit on the time step to ensure stability, which depends on the mesh cell size and the maximum absolute value of the potential in the computational domain. Finally, the evolution method can be used to solve general time-dependent problems, including time-dependent potentials.

[0082] This is a very robust and accurate method. In fact, it has been used to calculate very complex quantities, such as the spectra of atoms and their wave functions.

[0083] Having introduced the tools and ideas needed to understand the method proposed in this paper, a description of the method itself is provided below. In the following, the method is referred to as the quantum-inspired method.

[0084] Training ANNs by simulating quantum systems.

[0085] As discussed previously, the problem of training an ANN is essentially to minimize the corresponding error function, which is , and among them is called a dataset, where In this context, if the ANN has N hyperparameters, then N corresponding evolution equations must be solved, such as equation (1) introduced previously for gradient descent. Although this may be very different from the classical approach, it is clear that equation (1) can be replaced by a set of N corresponding Schrödinger equations (3): This is the first major difference between the proposed quantum-inspired approach and the well-known standard classical approach. These equations can in turn be solved numerically by the FDTD method described above.

[0086] In practice, the set of N equations will look like this:

[0087]

[0088]

[0089] …

[0090]

[0091] in, .

[0092] The system described by the above equations represents an ensemble of N quantum objects (e.g., electrons) that interact with each other through a given potential, which is given by the error function (To be precise, the potential U should be multiplied by the basic charge constant ).

[0093] However, the exact shape of the coupling between the equations remains to be specified. The specific way to obtain this coupling is provided by approximating (or in other words, tailoring) well-known techniques from DFT to the problem at hand, which is the second main difference between the proposed quantum-inspired approach and the classical DFT approach. In fact, it starts from the main problem solved in DFT, which is expressed by an analogy to the many-body Schrödinger equation, which reads as follows:

[0094] (4)

[0095] Now, the wave function is dependent on the variable of functions. Of course, this is a difficult problem even when solved by numerical techniques. In DFT, these problems are typical difficulties, which are solved by introducing the so-called Kohn-Sham system, which consists of a system of N monomer equations coupled by introducing an exchange-correlation potential. The exact mathematical shape of this potential is generally unknown, and many different approximations have been proposed (each with its own advantages and disadvantages). In this paper, a new coupling potential is proposed in the spirit of DFT, but with a different goal, namely to efficiently obtain the quantum state corresponding to the minimization of the error function (in this case the energy of this system), or equivalently, the quantum state corresponding to the training of the ANN (in other words, the accuracy of the method focuses on training the ANN rather than on computing quantum chemical features).

[0096] In the Kohn-Sham system, the coupling generally has the following mathematical shape (for the ith entity described by the single-body Schrödinger equation):

[0097]

[0098] The main information to be retained is the fact that an exchange-correlation potential is introduced, which depends on only one (electronic) variable at a time, thus completely avoiding the problem of simulating the many-body Schrödinger equation (4). In computational chemistry, this exchange-correlation potential can be any possible function, as long as it represents the system at hand to a certain degree of realism as desired (depending on the problem at hand).

[0099] In this paper, the following potential is introduced for the specific purpose of training ANNs:

[0100]

[0101] Among them, the symbol (in, ) represents the average position of the ith subject according to its wave function, that is, in mathematical terms:

[0102]

[0103] in, is the length of the (one-dimensional) spatial domain. If this potential is applied to the simulation of a real physical (quantum) system, it will be of no use: this is perfectly acceptable, since this is not the purpose of this work (it is also possible that more realistic potentials offer some advantages, but this is outside the scope of this work). The proposed potential introduces an advantage in finding the correct quantum state corresponding to a properly trained ANN. In fact, if and only if each of the N wave functions of the system has an average position corresponding to the solution of the problem at hand ,Right now When , the potential becomes zero.

[0104] Figure 4a The steps of a method 400 for performing training of a given ANN within the context of the tools proposed so far are shown; for each step, comments are added for the sake of clarity and completeness.

[0105] At step 401, an artificial neural network and a data set and its corresponding error function are specified. More specifically, the network is provided by specifying the neurons, layers, connections, etc. of the network (see Figure 4b ), and the dataset is provided as a set of input-output dependencies (e.g., data and labels). The network is provided with N hyperparameters to be calculated to reduce the error (or equivalently, the energy of the system). These N hyperparameters are represented by the N average positions of the N wave functions described in the following steps.

[0106] At step 402, the user passes the length of the one-dimensional space domain specifies the one-dimensional spatial domain. Other physics-related hyperparameters are also defined, more specifically, the number of spatial units , time step , and the maximum number of steps to run Finally, specify the value , which represents the range of values ​​in which to look for ANN weights and biases, i.e. The reason for introducing these physical parameters is not only to define the quantum system to be simulated, but also to specify the solution space in which the search is performed. Specifically, the method proposed in this document uses the wave function The average position To find the energy / error function Therefore, the following transformation will fall within the physical range The average position Convert to Belong to Range The variable r is:

[0107]

[0108] At step 403, at the position equal to the average position Prepare N wave functions under the same initial conditions corresponding to the quantum objects In fact, this is a Gaussian wave function with a mean value of , and the dispersion is The boundary condition of this wave function is when and . This applies during the entire simulation of this quantum object. It should be noted here that the x is not bold because in this case, space is one-dimensional.

[0109] Then, the starting time step is The main loop 405 of the system is executed. In this loop, at 404, several operations are performed, including: a) calculating the current average position of each wave function of the system, b) calculating the applied potential of each wave function of the system, and c) evolving each wave function by the FDTD method. During this time-dependent evolution, the total energy of the system (given by the quantity ) can increase or decrease (because the exploration of the solution space is quantum, or in other words, ergodic). Therefore, at step 406, the best solution found (corresponding to the average position , which minimally reduces the potential (or equivalently, the error function) ) are recorded for future use. It is important to note that during this simulation, tunneling effects will aid in the search for an optimal solution (just as would happen, for example, in a simulated quantum annealing approach).

[0110] At step 407, at the end of the previous evolution cycle, a final step is performed to improve the accuracy of the solution through standard genetic algorithm refinement, in which the best solution found by quantum simulation is improved by using it as the initial condition of the genetic algorithm.

[0111] A genetic algorithm (GA) is an optimization method inspired by evolution and survival of the fittest, used to generate high-quality solutions to optimization and search problems by relying on biologically inspired operators such as mutation, crossover, and selection.

[0112] In the final step of the method, a random search for candidate solutions is performed, which are then randomly evolved to obtain the next generation of solutions. A standard genetic algorithm is applied using the reduced energy function as input. Several iterations are then performed, where offspring are generated according to the reduced energy function by randomly varying the value of the reduced energy function and selecting the best offspring as the starting point for the next iteration. Those skilled in the art will appreciate that other implementations of the genetic algorithm or equivalent algorithms may be used, and those skilled in the art will know how to perform such refinement.

[0113] While the final step of refinement using a standard genetic algorithm does not introduce any quantum effects, it effectively helps improve the accuracy of the solution found via quantum simulation (whose accuracy can be affected by the spatial discretization introduced in the FDTD method).

[0114] The final optimal solution is then used to set the weights and biases of a given ANN, and inference is performed using that ANN.

[0115] Two validation tests have been performed to clearly show that the proposed approach is not only effective, but is also able to provide quantitatively and qualitatively different solutions compared to those obtained by conventional ANN training. These numerical experiments provide clear indications that the proposed approach can be used in real-life scenarios to obtain meaningful and relevant quantum states solely by numerical techniques, namely the training of ANNs. The two tests suggested below in this paper are based on the problem of fitting a curve by a given ANN, which is a typical problem in ML. The curve to be fitted is represented by a quadratic function and a square root function. For each function, three points are provided (where i=1, 2, 3) (In other words, these three points represent the data set for this problem). In more detail, for the quadratic function, we have:

[0116]

[0117] And for the square root function, we have:

[0118]

[0119] In both tests, the ANN used to fit the data consisted of a network with 6 neurons, 5 neurons as input and 1 neuron as output, where the input neurons were all directly connected to the output neuron. The activation function for the input neurons was the hyperbolic tangent, while the output activation function was simply the identity function (i.e., ). The hyperparameters used to simulate the quantum system are (total length of the physical one-dimensional domain), NX=5000 (number of units in spatial discretization), time step Femtoseconds, Quantum Epsilon (QEPS) = 1.e-3 (the simulation stops when the energy falls below this value), and Finally, the hyperparameters used for the last genetic step are NUM_POPULATION=1e6 (number of elements per generation), NUM_GEN=5000 (total number of generations to create), and Genetic Epsilon (GEPS)=1.e-4 (the genetic algorithm stops as long as the energy is less than this value).

[0120] Figures 5 to 11 The results of this test are reported in . In more detail, Figure 5 and Figure 6 The error functions obtained by the gradient descent method are shown respectively. The gradient descent method is linear ( Figure 5 ) and logarithm ( Figure 6 ) scale to solve this fitting problem. Figure 7 and Figure 8 The same diagram is shown, but with the method proposed in this document. It is clear that although they solve the same problem, the two methods behave very differently during the learning process, taking the ANN into different learning paths. This difference lies in the way the solution space is explored, which will take different paths, although it is impossible to predict how different these paths will be.

[0121] This is an important point to note because it clearly shows that the two methods provide qualitatively and quantitatively different solutions to the problem. Specifically, the method taught in this article seems to reach lower energies faster than the gradient descent technique. The figures have different scales, which is common in physics. Here, the different scales are due to the fact that gradient descent and the method proposed here calculate very different quantities during their iterative calculations. For example, gradient descent is calculated over a set of iterations, while the method described in this article is calculated during a set of time steps. Therefore, the two scale tables are not directly comparable.

[0122] Fig. 9 and Fig.10 Represents various values ​​of hyperparameters to be optimized based on the number of iterations of the learning process. Fig. 9shows the value of the network hyperparameters in the gradient descent method as a function of the number of iterations (in arbitrary units), while Fig.10 shows the variation of the values ​​of the network hyperparameters in the method disclosed herein as a function of the number of iterations (in arbitrary units). From these figures, it is even clearer that the two methods follow very different paths during the learning phase. In fact, the curves obtained by the two methods are clearly different both in quantity and quality. In the case of gradient descent, Fig. 9 In the case of quantum-inspired methods, the curve converges smoothly to the final solution (i.e., the local minimum). Fig.10 In the case of , the curves quickly reach a final solution and, eventually, this solution can evolve further due to quantum tunneling effects (represented by jumps in this figure) that occur during the simulation of the quantum system. Furthermore, the solutions to which the two methods converge are different, providing different qualities / behaviors to the neural network. It is expected that quantum-inspired solutions will have advantages over solutions found by classical methods. Finally, Fig.11 The curve to be fitted and the fitted curves provided by the two learning methods are shown.

[0123] Now let's look at the results for the case where we want to fit a square root function. These results are Figures 12 to 18 In detail, Fig.12 and Fig.13 The loss / error / cost function obtained by gradient descent on both linear and algorithmic scales is shown. Fig.14 and Fig.15 The same is shown, but with a quantum-inspired approach. It can be seen that the same phenomena that can be observed in the case of quadratic function fitting are present here. In fact, it is clearly shown that the two methods take very different paths during the learning phase, and in particular, the quantum-inspired approach seems to reach lower errors at a faster rate. Fig.16 and Fig.17 and Fig. 9 and Fig.10 is similar, but for the square root case. Clearly, the proposed method reaches the plateau faster than gradient descent, and furthermore, the quantum tunneling jumps are even more pronounced in this case. Finally, Fig.18 The curves to be fitted are shown, as well as the solutions provided by these methods.

[0124] As recognized by those skilled in the art, the quantum-inspired method presented herein can of course be optimized in many different ways. It should be noted that what is presented herein is a proof-of-concept. Therefore, the specific implementation presented here should not be seen as a limitation of the method itself, but rather as one possible implementation among many (which is likely to be one of the simplest, intended to focus on the main principles of the method rather than specific (and most likely irrelevant) details).

[0125] So possible improvements / alternatives to what has already been proposed include:

[0126] - Introducing local density approximation (LDA) type exchange correlation terms to improve the level of realism,

[0127] - Introducing more realistic activation functions based on measurements of biological neurons, and

[0128] -Introduction of adaptive mesh refinement in FDTD method.

[0129] Again, it is emphasized that the choices made in this disclosure are not limitations of the presented methods, but they are chosen only for the purpose of simplifying the discussion.

[0130] refer to Fig.19 , a computer-implemented method 1900 for training an artificial neural network (ANN) is provided. The method includes: at step 1902, defining an energy function for the ANN and a data set according to a quantum object. This means that an energy function for the ANN and for a specific data set to be used with the ANN is defined according to the quantum object. The method includes: at 1904, simulating a quantum system using the quantum object to reduce the energy function and obtain a trained ANN.

[0131] The energy function may depend on the topology of the ANN and the data set, and may be adapted to simulate the quantum system according to the following error function:

[0132]

[0133] in, The ANN is represented as a function, where the data set is given by It indicates that, .

[0134] The energy function can be adjusted by transforming it into an exchange-correlation potential suitable for density functional theory (DFT) simulations. This transformation can be obtained by using the average position of the quantum objects that make up the quantum system.

[0135] The quantum object may include an object for each of a plurality of hyperparameters to be trained. The plurality of hyperparameters to be trained may include a hyperparameter for each of a plurality of weights and biases of the ANN. The quantum object may include a quantum object for each of the following: the number of layers of the ANN, the number of neurons in each layer, the connections between neurons, the discriminant, and at least one activation function of the neuron. The quantum object may include an object for each of a plurality of variables of the quantum system, the plurality of variables including: the length Lx of the spatial domain, which defines the finite length within which all quantum objects are confined; the number of spatial cells NX, which divide the finite length into parts; the time step to be used for the simulation ; Maximum number of steps , is defined as the maximum number of iterations to be performed during the simulation of a quantum system; and the maximum value range , defining the solution space for each quantum object. NX and It can be determined based on available computing resources. For example, if the method is performed on a single computer, the number of NX can be hundreds, or if the method is performed using a computer cluster or in a cloud implementation, the number of NX can be thousands or more.

[0136] The energy function can be expressed as:

[0137]

[0138] in, is based on the actual position of the i-th entity of the corresponding wave function, and each symbol represents the average position of the i-th subject, where , the average position can be expressed as:

[0139]

[0140] in, is the length of the one-dimensional space domain, and is the ith wave function.

[0141] A quantum object can be described as a set of N individual Schrödinger equations defined as:

[0142]

[0143]

[0144] …

[0145]

[0146] in, is the reduced Planck constant, and is the mass of the electron.

[0147] Simulating a quantum system may include iteratively solving the set of N individual Schrödinger equations until an energy function is minimized under a quantum epsilon (QEPS) threshold, or until a maximum number of steps is reached. until.

[0148] Iteratively solving the set of N single-unit Schrödinger equations may include: calculating a current average position of each wave function of the system; calculating an applied potential for each wave function of the system; and evolving each wave function via a finite-difference time-domain (FDTD) method.

[0149] Each corresponding average position of the reduced energy function can be obtained using the following equation Extract the weights and bias r of the trained ANN:

[0150]

[0151] in, defines the maximum numerical range of the solution space, and Defines the length of the spatial domain.

[0152] The method may also include: at step 1906, using the reduced energy function as an input to the genetic algorithm to refine the reduced energy function. The refinement may refine the average position of the energy function .

[0153] The genetic algorithm can iterate and use the reduced energy function for the next iteration, or if the reduced energy function cannot be obtained in the iteration, use the previous reduced energy function for the next iteration, until the reduced energy function is minimized under a genetic epsilon (GEPS) threshold or until a maximum number of iterations is reached.

[0154] The method may further comprise any of the steps described herein.

[0155] It should be noted that the methods and steps described herein are typically computer-implemented methods and steps.The term "computer" may be interpreted to have different meanings, such as those explained below.

[0156] refer to Fig. 20 , provides an apparatus, computer, server or device (HW) 2001, in which the functions and steps described herein for training an ANN can be implemented.

[0157] The apparatus 2001 may be a server, a network node, a radio base station, or other computing device, which may be part of a cloud computing system, an edge computing system, or may be a standalone device. The apparatus may be operable to perform selected steps or all steps of the method for training an ANN described herein. It may also be operable to store the ANN, use the ANN, and send / receive the ANN. In some embodiments, the apparatus may be part of a system comprising multiple apparatuses, in which it may be advantageous to perform the method in a distributed manner.

[0158] The apparatus 2001 includes a processing circuit 2003 and a memory 2005. The memory 2005 may contain instructions executable by the processing circuit 2003, whereby the functions and steps described herein may be performed to provide any relevant features and benefits disclosed herein.

[0159] The apparatus 2001 may also include a non-transitory, non-transitory machine-readable storage medium 2007 having stored therein software and / or instructions 2009 executable by the processing circuit 2003 to perform the functions and steps described herein. The apparatus may also include a network interface and a power source.

[0160] Instructions 2009 may include a computer program for configuring processing circuit 2003. The computer program may be stored in a physical memory local to the device, which may be removable, or the computer program may be alternatively or partially stored in the cloud. The computer program may also be embodied in a carrier such as an electrical signal, an optical signal, a radio signal, or a computer-readable storage medium.

[0161] refer to Fig.21 , provides a virtualized environment 2100 in which the functions and steps described in this article for training ANN can be implemented.

[0162] Virtualized environments 2100 (which may exceed Fig.21 The scope shown) may include systems, networks, servers, nodes, computers, devices, etc. that communicate with each other by wired or wireless means (e.g., through a network interface component (NIC) including a physical network interface (NI)). Some or all of the functions and steps described herein may be implemented as one or more virtual components executed on one or more physical devices in one or more networks, systems, environments, etc. (e.g., via one or more applications, components, functions, virtual machines, containers, etc.).

[0163] The virtualized environment provides hardware 2101 including a processing circuit 2103 and a memory 2105. The memory 2105 may contain instructions executable by the processing circuit 2103, whereby the functions and steps described herein may be performed to provide any of the relevant features and benefits disclosed herein.

[0164] The hardware 2101 may also include a non-transitory, non-transitory machine-readable storage medium 2107 having stored therein software and / or instructions 2109 executable by the processing circuit 2103 to perform the functions and steps described herein.

[0165] Instructions 2109 may include a computer program for configuring processing circuit 2103. The computer program may be stored in a removable memory such as a portable optical disc, a portable digital video disc or other removable media. The computer program may also be embodied in a carrier such as an electrical signal, an optical signal, a radio signal or a computer-readable storage medium.

[0166] refer to Fig. 20 and Fig.21 , a system 2001, 2100, 2101 for training an artificial neural network (ANN) is provided, which can be composed of a single computing device, a cluster of computing devices, or a cloud of computing devices. The system includes a processing circuit 2003, 2103 and a memory 2005, 2105, the memory containing instructions executable by the processing circuit, whereby the system is operable to define an energy function for an ANN and a data set according to a quantum object, and use the quantum object to simulate a quantum system to reduce the energy function and obtain a trained ANN.

[0167] The energy function may depend on the topology of the ANN and the data set, and may be adapted to simulate the quantum system according to the following error function:

[0168]

[0169] in, The ANN is represented as a function, where the data set is given by It indicates that, .

[0170] The energy function can be adjusted by transforming it into an exchange-correlation potential suitable for density functional theory (DFT) simulations. This transformation can be obtained by using the average position of the quantum objects that make up the quantum system.

[0171] The quantum object may include an object for each of a plurality of hyperparameters to be trained. The plurality of hyperparameters to be trained may include a hyperparameter for each of a plurality of weights and biases of the ANN. The quantum object may include a quantum object for each of the following: the number of layers of the ANN, the number of neurons in each layer, the connections between neurons, the discriminant, and at least one activation function of the neuron. The quantum object may include an object for each of a plurality of variables of the quantum system, the plurality of variables including: the length Lx of the spatial domain, which defines the finite length within which all quantum objects are confined; the number of spatial cells NX, which divide the finite length into parts; the time step to be used for the simulation ; Maximum number of steps , is defined as the maximum number of iterations to be performed during the simulation of a quantum system; and the maximum value range , defining the solution space for each quantum object. NX and This can be determined based on available computing resources.

[0172] The energy function can be expressed as:

[0173]

[0174] in, is based on the actual position of the i-th entity of the corresponding wave function, and each symbol represents the average position of the i-th subject, where , the average position can be expressed as:

[0175]

[0176] in, is the length of the one-dimensional space domain, and is the ith wave function.

[0177] A quantum object can be described as a set of N individual Schrödinger equations defined as:

[0178]

[0179]

[0180] …

[0181]

[0182] in, is the reduced Planck constant, and is the mass of the electron.

[0183] The system may also be operable to simulate a quantum system by iteratively solving the set of N single-unit Schrödinger equations until the energy function is minimized under a quantum epsilon (QEPS) threshold, or until a maximum number of steps is reached. until.

[0184] The system may be further operable to iteratively solve the set of N single-unit Schrödinger equations by: calculating a current mean position of each wave function of the system; calculating an applied potential for each wave function of the system; and evolving each wave function by a finite-difference time-domain (FDTD) method.

[0185] Each corresponding average position of the reduced energy function can be obtained using the following equation Extract the weights and bias r of the trained ANN:

[0186]

[0187] in, defines the maximum numerical range of the solution space, and Lx defines the length of the spatial domain.

[0188] The system may be further operable to refine the reduced energy function using the reduced energy function as an input to a genetic algorithm. The genetic algorithm may iterate and use the reduced energy function for the next iteration, or if the reduced energy function cannot be obtained in an iteration, use the previous reduced energy function for the next iteration, until the reduced energy function is minimized under a genetic epsilon (GEPS) threshold or until a maximum number of iterations is reached.

[0189] The system may also be operable to perform any of the steps described herein.

[0190] Still reference Fig. 20 and Fig.21 , a non-transitory computer-readable medium 2007, 2107 is provided, on which instructions 2009, 2109 for training an artificial neural network (ANN) are stored. The instructions include: defining an energy function for the ANN and a data set according to a quantum object, and using the quantum object to simulate a quantum system to reduce the energy function and obtain a trained ANN.

[0191] The non-transitory computer-readable medium may also include instructions for performing any of the steps described herein.

[0192] With the benefit of the teachings presented in the foregoing description and the associated drawings, those skilled in the art will recall various modifications. Therefore, it should be understood that modifications such as specific forms other than those described above are intended to be included within the scope of the present disclosure. The previous description is merely illustrative and should not be considered as limiting in any way. The scope sought is given by the appended claims rather than the foregoing description, and all variations and equivalents falling within the scope of the claims are intended to be included therein. Although specific terms may be used herein, they are used only in a general and descriptive sense and are not intended to be limiting.

Claims

1. A computer-implemented method for training an artificial neural network (ANN), comprising: - defining an energy function for the ANN and the data set based on the quantum object; as well as - Using the quantum object to simulate a quantum system to reduce the energy function and obtain a trained ANN.

2. The method according to claim 1, wherein: The energy function depends on the topology of the ANN and the data set and is suitable for simulating the quantum system according to the following error function: in, The ANN is represented as a function, and wherein the data set is given by It indicates that, .

3. The method according to claim 2, wherein: The energy function is adjusted by converting it into an exchange-correlation potential suitable for density functional theory (DFT) simulations.

4. The method according to claim 3, wherein: The transformation is obtained by using the average position of the quantum objects constituting the quantum system.

5. The method according to claim 1, wherein: The quantum object includes an object for each of a plurality of hyperparameters to be trained.

6. The method according to claim 5, wherein: The plurality of hyperparameters to be trained includes a hyperparameter for each of a plurality of weights and biases of the ANN.

7. The method according to claim 1, 5 or 6, wherein: The quantum object includes a quantum object for each of the following items: the number of layers of the ANN, the number of neurons in each layer, the connections between neurons, a discriminant, and at least one activation function of the neuron.

8. The method according to claim 1, 5, 6 or 7, wherein: The quantum object comprises, for each variable of a plurality of variables of the quantum system, an object, the plurality of variables comprising: a length Lx of a spatial domain, the spatial domain defining a finite length within which all of the quantum objects are confined; a number of spatial cells NX, the spatial cells NX dividing the finite length into parts; a time step to be used for simulation ; Maximum number of steps , is defined as the maximum number of iterations to be performed during the simulation of the quantum system; and the maximum value range , defining the solution space for each quantum object.

9. The method according to claim 8, wherein: NX and Determined based on available computing resources.

10. The method according to claim 1, wherein: The energy function is expressed as: in, is based on the actual position of the i-th entity of the corresponding wave function, and each symbol represents the average position of the i-th subject, where , the average position can be expressed as: in, is the length of the one-dimensional space domain, and is the ith wave function.

11. The method according to claim 1, wherein: The quantum object is described as a set of N individual Schrödinger equations defined as: … in, is the reduced Planck constant, and is the mass of the electron.

12. The method according to claim 11, wherein: Simulating the quantum system comprises iteratively solving the set of N single-unit Schrödinger equations until the energy function is minimized under a quantum Epsilon QEPS threshold, or until a maximum number of steps is reached. until.

13. The method according to claim 12, wherein: Iteratively solving the set of N single-unit Schrödinger equations includes: - calculating the current mean position of each wave function of the system; - calculating the applied potential for each wave function of the system; and -Evolve each wave function by the finite-difference time-domain FDTD method.

14. The method according to claim 12, wherein: The reduced energy function is then converted to each corresponding average position using the following equation Extract the weights and bias r of the trained ANN: in, defines the maximum numerical range of the solution space, and Lx defines the length of the spatial domain.

15. The method according to claim 1, further comprising: The reduced energy function is refined using the reduced energy function as an input to a genetic algorithm.

16. The method according to claim 15, wherein: The genetic algorithm iterates and uses the reduced energy function for the next iteration, or if the reduced energy function cannot be obtained in the iteration, the previous reduced energy function is used for the next iteration, until the reduced energy function is minimized under the genetic Epsilon GEPS threshold or until the maximum number of iterations is reached.

17. A system for training an artificial neural network (ANN), the system comprising a processing circuit and a memory, the memory containing instructions executable by the processing circuit, whereby the system is operable to: - defining an energy function for the ANN and the data set based on the quantum object; and - Using the quantum object to simulate a quantum system to reduce the energy function and obtain a trained ANN.

18. The system of claim 17, wherein: The energy function depends on the topology of the ANN and the data set and is suitable for simulating the quantum system according to the following error function: in, The ANN is represented as a function, and wherein the data set is given by It indicates that, .

19. The system of claim 18, wherein: The energy function is adjusted by converting it into an exchange-correlation potential suitable for density functional theory (DFT) simulations.

20. The system of claim 19, wherein: The transformation is obtained by using the average position of the quantum objects constituting the quantum system.

21. The system of claim 17, wherein: The quantum object includes an object for each of a plurality of hyperparameters to be trained.

22. The system of claim 21, wherein: The plurality of hyperparameters to be trained includes a hyperparameter for each of a plurality of weights and biases of the ANN.

23. The system of claim 17, 21 or 22, wherein: The quantum object includes a quantum object for each of the following items: the number of layers of the ANN, the number of neurons in each layer, the connections between neurons, a discriminant, and at least one activation function of the neuron.

24. The system of claim 17, 21, 22 or 23, wherein: The quantum object comprises, for each variable of a plurality of variables of the quantum system, an object, the plurality of variables comprising: a length Lx of a spatial domain, the spatial domain defining a finite length within which all of the quantum objects are confined; a number of spatial cells NX, the spatial cells NX dividing the finite length into parts; a time step to be used for simulation ; Maximum number of steps , is defined as the maximum number of iterations to be performed during the simulation of the quantum system; and the maximum value range , defining the solution space for each quantum object.

25. The system of claim 24, wherein: NX and Determined based on available computing resources.

26. The system of claim 17, wherein: The energy function is expressed as: in, is based on the actual position of the i-th entity of the corresponding wave function, and each symbol represents the average position of the i-th subject, where , the average position can be expressed as: in, is the length of the one-dimensional space domain, and is the ith wave function.

27. The system of claim 17, wherein: The quantum object is described as a set of N individual Schrödinger equations defined as: … in, is the reduced Planck constant, and is the mass of the electron.

28. The system of claim 27, further operable to simulate the quantum system by iteratively solving the set of N individual Schrödinger equations until the energy function is minimized under a quantum epsilon threshold (QEPS), or until a maximum number of steps is reached. until.

29. The system of claim 28, further operable to iteratively solve the set of N individual Schrödinger equations by: - calculating the current mean position of each wave function of the system; - calculating the applied potential for each wave function of the system; and -Evolve each wave function by the finite-difference time-domain FDTD method.

30. The system of claim 28, wherein: The reduced energy function is then converted to each corresponding average position using the following equation Extract the weights and bias r of the trained ANN: in, defines the maximum numerical range of the solution space, and Lx defines the length of the spatial domain.

31. The system of claim 17, further operable to refine the reduced energy function using the reduced energy function as an input to a genetic algorithm.

32. The system of claim 31, wherein: The genetic algorithm iterates and uses the reduced energy function for the next iteration, or if the reduced energy function cannot be obtained in the iteration, the previous reduced energy function is used for the next iteration, until the reduced energy function is minimized under the genetic Epsilon GEPS threshold or until the maximum number of iterations is reached.

33. A non-transitory computer readable medium having stored thereon instructions for training an artificial neural network (ANN), the instructions comprising: - defining an energy function for the ANN and the data set based on the quantum object; as well as - Using the quantum object to simulate a quantum system to reduce the energy function and obtain a trained ANN.

34. The non-transitory computer readable medium of claim 33, further comprising the steps of any one of claims 2 to 16.

Images