An ising machine optimization-based recurrent neural network training method and system
By transforming the weight training of recurrent neural networks into the Ising model and utilizing the physical evolution of the Ising machine, the gradient vanishing and exploding problems in long sequence training of traditional recurrent neural networks are solved, achieving more efficient and stable training and prediction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-03-06
- Publication Date
- 2026-06-26
AI Technical Summary
Traditional recurrent neural network training methods face the problems of vanishing and exploding gradients when dealing with long sequences, making it difficult to learn long-distance dependencies, resulting in an unstable training process. Furthermore, they rely on the traditional von Neumann computing architecture, which suffers from long computation time and sensitivity to initial values.
The weight training task of the recurrent neural network is transformed into a quadratic unconstrained binary optimization problem and mapped to the Ising model. The spin configuration is obtained by using the physical evolution or annealing process of the Ising machine to determine the weights of the recurrent neural network.
It effectively avoids gradient vanishing and gradient exploding, improves training stability and long-term prediction ability, enhances model expressiveness and prediction accuracy, and significantly shortens computation time by utilizing the physical acceleration capability of the Ising machine, achieving millisecond-level training time.
Smart Images
Figure CN122287704A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of neural network training technology, and specifically to a recurrent neural network training method and system based on Ising machine optimization. Background Technology
[0002] Time series forecasting has significant application value in many fields such as financial markets, weather forecasting, and industrial monitoring; Recurrent Neural Networks (RNNs), due to their unique internal feedback structure, can effectively handle the time dependence in sequence data, making them a key technology for time series forecasting.
[0003] However, traditional RNN training heavily relies on gradient-based optimization algorithms, especially Backpropagation Through Time (BPTT). The core idea of BPTT is to unfold the RNN into a deep feedforward network in the time dimension, backpropagate the error signal over time, calculate the gradient of the loss function with respect to each network weight, and finally update the weights using gradient descent and its variants.
[0004] When dealing with long sequences, BPTT faces two core challenges: vanishing gradients and exploding gradients. These problems make it difficult for RNNs to learn long-range dependencies, resulting in unstable training processes and severely limiting their long-term predictive ability on complex dynamical systems (such as chaotic systems). Furthermore, BPTT also suffers from long training times, sensitivity to initial values, and a tendency to get trapped in local optima.
[0005] To address these issues, researchers have proposed Reservoir Computing (RC) methods, such as Echo State Networks. In this framework, the weights of the input and hidden layers (reservoir) are randomly fixed, and only the output layer weights are trained, typically using a simple linear regression method. However, RC methods suffer from the following drawbacks: the reservoir parameters require extensive manual tuning, the linear readout layer has limited expressive power, long-term prediction stability is poor, and it still relies on the traditional von Neumann computing architecture.
[0006] Therefore, there is an urgent need for an RNN training method that does not rely on gradient calculation and can fully utilize the advantages of new computing architectures. Summary of the Invention
[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a recurrent neural network training method and system based on Ising machine optimization.
[0008] To achieve the above objectives, the present invention provides the following technical solution:
[0009] This application provides a recurrent neural network training method based on Ising machine optimization, including the following steps:
[0010] The weight training task of the recurrent neural network is transformed into a quadratic unconstrained binary optimization problem;
[0011] The quadratic unconstrained binary optimization problem is mapped to the Ising model;
[0012] The Ising model is loaded into the Ising machine, and the spin configuration corresponding to the minimum energy state of the Ising model is obtained through the physical evolution or annealing process of the Ising machine.
[0013] The weights of the recurrent neural network are determined based on the spin configuration and loaded into the recurrent neural network to complete the training.
[0014] Optionally, the transformation of the recurrent neural network weight training task into a quadratic unconstrained binary optimization problem specifically includes:
[0015] The recurrent neural network is reconstructed into a dynamic system, and its closed-loop dynamics and loss function are transformed into a global quadratic energy function.
[0016] The reconstructed loss function is encoded as a quadratic unconstrained binary optimization model.
[0017] Optionally, encoding the reconstructed loss function into a quadratic unconstrained binary optimization model includes performing binary discrete encoding on the continuous weights to be optimized:
[0018] Each weight Approximately represented by K binary variables Linear combination:
[0019]
[0020] in, Indicates the first Each weight has a value range of 1. , K represents the total number of weights to be optimized; K is the number of binary variables used for each weight. Preset precision vector The There are 1 element used to control the weight precision when combining binary variables; by substituting the binary representations of all weight variables into the loss function, we obtain the quadratic unconstrained binary optimization form.
[0021] Optionally, substituting the binary representations of all weight variables into the loss function to obtain the quadratic unconstrained binary optimization form specifically includes:
[0022] Constructing the precision matrix This makes the weight matrix ,in It is a long vector consisting of all binary variables;
[0023] Will Substituting the mean squared error loss function, we obtain a quadratic unconstrained binary optimization model:
[0024]
[0025] in, The input data matrix has each row corresponding to an input vector at a time step. This is the target output matrix, where each row corresponds to the target output value at the corresponding time step. This indicates the matrix transpose.
[0026] Optionally, the binary discrete encoding includes introducing an offset matrix C to support the representation of positive and negative weights, the weight matrix being represented as:
[0027]
[0028] in, To and A constant offset matrix of the same dimension is used to set the lower bound of the weight values;
[0029] The corresponding quadratic unconstrained binary optimization model is:
[0030] .
[0031] Optionally, mapping the quadratic unconstrained binary optimization problem to the Ising model includes converting the binary variables into Ising spins.
[0032]
[0033] in, It is a binary variable. For the corresponding Ising spin;
[0034] Substituting the transformed spin variable into the quadratic unconstrained binary optimization model, we obtain the energy function of the Ising model:
[0035]
[0036] For all Isin spin The long vector formed; For the input data matrix Target output matrix And precision matrix The derived coupling matrix; This is a constant term that is independent of the spin variable.
[0037] Optionally, the Ising machine includes at least one of the following types:
[0038] Optical Ising machine, quantum annealing machine, optoelectronic Ising machine, electronic Ising machine based on FPGA or CMOS circuits, and Ising machine based on superconductors or atomic systems.
[0039] Optionally, the recurrent neural network includes at least one of leaky integral neurons (RNN), long short-term memory networks (LSM), or gated recurrent units.
[0040] Secondly, this application provides a recurrent neural network training system based on Ising machine optimization, comprising:
[0041] The QUBO mapping module is used to transform the weight training task of a recurrent neural network into a quadratic unconstrained binary optimization problem and map it to the Ising model;
[0042] The Ising machine solver module is used to load the Ising model into the Ising machine and obtain the spin configuration corresponding to the energy minimum state of the Ising model through the physical evolution or annealing process of the Ising machine.
[0043] The weight loading module is used to determine the weights of the recurrent neural network according to the spin configuration and load them into the recurrent neural network to complete the training.
[0044] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the above-described method for training a recurrent neural network based on Ising machine optimization.
[0045] Compared with the prior art, the present invention has the following beneficial effects:
[0046] 1. This scheme transforms the training process from gradient-based iterative optimization to physics-evolution-based combinatorial optimization, fundamentally avoiding the vanishing and exploding gradient problems. This enables RNNs to effectively learn complex dependencies in long sequences, significantly improving training stability and long-term prediction capabilities for complex dynamics such as chaotic systems; for example... Figure 4 As shown: Experimental results show that in the Logistic chaotic mapping prediction task, compared with the traditional RC algorithm, the proposed method can extend the effective prediction step size from 25 steps to 30 steps (a 20% improvement) while maintaining a lower error.
[0047] 2. This approach does not simply train a simple linear readout layer, but uses QUBO to globally optimize the core weights of the RNN, enabling it to learn more complex and accurate nonlinear mappings, thereby improving the model's expressive power and prediction accuracy. At the same time, this method also avoids the tedious and non-optimal manual parameter tuning process in the RC method.
[0048] 3. Utilizing the physical acceleration capabilities of non-von Neumann hardware such as the optical Ising machine, the core optimization solution process (physical annealing) can be completed in milliseconds. Compared to the traditional BPTT or RC methods, which require computation times in the range of seconds to minutes on CPUs / GPUs, the RC algorithm has a computation time of less than 1 second. The classic RNN algorithm takes 2 seconds to 3 minutes (depending on the parameters). Based on advanced Ising machine computing equipment, this solution is expected to achieve millisecond-level computation, realizing a performance leap of several orders of magnitude and providing the possibility for real-time or large-scale RNN training.
[0049] 4. This invention validates a novel architecture that deeply integrates neural network training with physical computation. As the scale and precision of the Ising machine hardware continue to improve, the performance and applicability of this solution will continue to be enhanced, providing a clear technical path for building the next generation of efficient, stable, and low-power AI systems. Attached Figure Description
[0050] Figure 1 This is a flowchart of training a recurrent neural network based on the backpropagation algorithm;
[0051] Figure 2 The flowchart shows the training method for recurrent neural networks based on reservoir computation.
[0052] Figure 3 This is a flowchart of the recurrent neural network training method based on Ising machine optimization according to the present invention.
[0053] Figure 4 The image shows the RNN prediction results of the 25-step RC algorithm. Detailed Implementation
[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0055] Furthermore, in this invention, an element referred to as fixed to or disposed on another element may be directly disposed on the other element, or there may be an intermediate element. When an element is considered to be connected to another element, it may be directly connected to the other element, or there may be an intermediate element present simultaneously. The terms vertical, horizontal, left, right, and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementation.
[0056] Explanation of technical terms in this application:
[0057] RNN (Recurrent Neural Network): An artificial neural network with feedback connections, specifically designed for processing and predicting sequential data.
[0058] Ising Machine: A non-von Neumann computing paradigm inspired by statistical physics that solves combinatorial optimization problems by simulating the natural evolution of a physical system to its energy minimum state (ground state).
[0059] Optical Ising Machine: An Ising machine implemented using optical or quantum optical systems (such as coherent light pulses, parametric oscillators, etc.) has the advantages of fast computation speed, high solution quality and low power consumption.
[0060] QUBO (Quadratic Unconstrained Binary Optimization): A combinatorial optimization problem model whose objective is to minimize a quadratic function consisting of binary variables. The Ising model and the QUBO model are interchangeable.
[0061] BPTT (Backpropagation over Time): A standard algorithm for training recurrent neural networks, which calculates gradients by expanding the network over time and backpropagating the error.
[0062] Vanishing / exploding gradients: When training deep neural networks (especially RNNs), the gradients during backpropagation decay or grow exponentially with the number of layers, making it difficult for the network to learn long-distance dependencies or causing training instability.
[0063] Reservoir Computation (RC): A recurrent neural network training framework in which the weights of the input layer and hidden layer (reservoir) are randomly fixed, and only the weights of the output layer are trained.
[0064] Example 1
[0065] This application provides a recurrent neural network training method based on Ising machine optimization, including the following steps:
[0066] The weight training task of the recurrent neural network is transformed into a quadratic unconstrained binary optimization problem;
[0067] The quadratic unconstrained binary optimization problem is mapped to the Ising model;
[0068] The Ising model is loaded into the Ising machine, and the spin configuration corresponding to the minimum energy state of the Ising model is obtained through the physical evolution or annealing process of the Ising machine.
[0069] The weights of the recurrent neural network are determined based on the spin configuration and loaded into the recurrent neural network to complete the training.
[0070] In this embodiment, the present invention first transforms the RNN weight training task into a quadratic unconstrained binary optimization (QUBO) problem, thereby completely avoiding gradient calculation; then, the QUBO problem is mapped to the Ising model through variable substitution; next, the coefficients of the Ising model are loaded into a physical Ising machine (such as an optical Ising machine) to initiate the physical evolution process; for example, in a coherent Ising machine based on a degenerate optical parametric oscillator, the system undergoes collective symmetry breaking under the action of pump light, and the phase of each light pulse (representing the spin state) naturally evolves to the state that minimizes the loss of the entire network, which corresponds to the spin configuration of the energy minimum state of the Ising model; finally, the final state configuration of all spins is read from the Ising machine, and the optimized continuous weight values of the RNN are restored according to the encoding rules and loaded into the RNN to complete the training.
[0071] In one specific implementation, transforming the weight training task of the recurrent neural network into a quadratic unconstrained binary optimization problem specifically includes:
[0072] The recurrent neural network is reconstructed into a dynamic system, and its closed-loop dynamics and loss function are transformed into a global quadratic energy function.
[0073] The reconstructed loss function is encoded as a quadratic unconstrained binary optimization model.
[0074] To map the RNN training problem to the Ising model, we first reconstruct the dynamic equations of the classic leaky integral neuron RNN; then, we define a new weight matrix: The original RNN's closed-loop dynamics and loss function are transformed into a global quadratic energy function with respect to the new variables, along with the transformed state variables. This reconstruction process preserves the nonlinear expressive power of the RNN and allows the training loss to be represented in a format suitable for mapping to the Ising Hamiltonian. Subsequently, the reconstructed loss function is encoded into a QUBO model, laying the foundation for subsequent QUBO mapping.
[0075] In one specific implementation, encoding the reconstructed loss function into a quadratic unconstrained binary optimization model includes performing binary discrete encoding on the continuous weights to be optimized:
[0076] Each weight Approximately represented by K binary variables Linear combination:
[0077]
[0078] in, Indicates the first Each weight has a value range of 1. , K represents the total number of weights to be optimized; K is the number of binary variables used for each weight. Preset precision vector The There are 1 element used to control the weight precision when combining binary variables; by substituting the binary representations of all weight variables into the loss function, we obtain the quadratic unconstrained binary optimization form.
[0079] Each weight in the continuous weight matrix to be optimized Using K binary variables A linear combination is used to approximate the precision vector. Each element It is a preset constant, such as a power of 2 or a uniform interval value; in this way, the continuous weight optimization problem is transformed into a combinatorial optimization problem of binary variables, and the QUBO form can be obtained by substituting it into the loss function.
[0080] In one specific implementation, substituting the binary representations of all weight variables into the loss function to obtain a quadratic unconstrained binary optimization form specifically includes:
[0081] Constructing the precision matrix This makes the weight matrix ,in It is a long vector consisting of all binary variables;
[0082] Will Substituting the mean squared error loss function, we obtain a quadratic unconstrained binary optimization model:
[0083]
[0084] in, The input data matrix has each row corresponding to an input vector at a time step. This is the target output matrix, where each row corresponds to the target output value at the corresponding time step. This indicates the matrix transpose.
[0085] By constructing the precision matrix The weight matrix can be Represented as Substituting this relationship into the expansion of the mean squared error loss function, for example... This yields the standard form of the QUBO model described above, where all terms are about binary variables. The quadratic form.
[0086] In one specific implementation, the binary discrete encoding includes introducing an offset matrix C to support the representation of positive and negative weights, the weight matrix being represented as:
[0087]
[0088] in, To and A constant offset matrix of the same dimension is used to set the lower bound of the weight values;
[0089] The corresponding quadratic unconstrained binary optimization model is:
[0090] .
[0091] In order for the model to capture both positive and negative parameters, an offset matrix is introduced. This represents the lower bound of the weight values, thus allowing the weight matrix to be represented as... Substituting the values into the loss function, we obtain the QUBO model that includes the offset term.
[0092] In one specific implementation, mapping the quadratic unconstrained binary optimization problem to the Ising model includes converting the binary variables into Ising spins.
[0093]
[0094] in, It is a binary variable. For the corresponding Ising spin;
[0095] Substituting the transformed spin variable into the quadratic unconstrained binary optimization model, we obtain the energy function of the Ising model:
[0096]
[0097] For all Isin spin The long vector formed; For the input data matrix Target output matrix And precision matrix The derived coupling matrix; This is a constant term that is independent of the spin variable.
[0098] Through transformation =( +1) / 2 converts the binary variables of the QUBO model into Ising spins. Substituting ∈{-1,1} into the QUBO model yields the energy function of the Ising model. = The Q matrix is determined by the input data, the target output, and the accuracy matrix. Thus, the training problem of RNN is transformed into finding the Ising model that minimizes energy.
[0099] In one specific embodiment, the ising machine includes at least one of the following types:
[0100] Optical Ising machine, quantum annealing machine, optoelectronic Ising machine, electronic Ising machine based on FPGA or CMOS circuits, and Ising machine based on superconductors or atomic systems.
[0101] In addition to the optical coherent Ising machine, the solution module of this invention can be replaced by other physically implemented Ising machines, including quantum annealing machines, optoelectronic Ising machines, electronic / FPGA Ising machines, and Ising machines based on other physical systems. These different types of Ising machines can all be used to perform physical evolution or annealing processes.
[0102] In one specific embodiment, the recurrent neural network includes at least one of leaky integral neurons (RNN), long short-term memory networks (LSM), or gated recurrent units.
[0103] The core idea of this invention (converting training into QUBO and solving it with an Ising machine) can also be applied to other types of recurrent neural networks, such as Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs), simply by adjusting the reconstruction method of their dynamic systems accordingly.
[0104] Secondly, this application provides a recurrent neural network training system based on Ising machine optimization, comprising:
[0105] The QUBO mapping module is used to transform the weight training task of the recurrent neural network into a quadratic unconstrained binary optimization problem and map it to the Ising model. Specifically, this module includes functions such as dynamic system reconstruction, binary discrete encoding, QUBO model construction, and Ising model conversion. Its operation process is consistent with the steps in Example 1.
[0106] The Ising machine solver module is used to load the Ising model into the Ising machine and obtain the spin configuration corresponding to the energy minimum state of the Ising model through the physical evolution or annealing process of the Ising machine.
[0107] The weight loading module is used to determine the weights of the recurrent neural network according to the spin configuration and load them into the recurrent neural network to complete the training.
[0108] Thirdly, this application provides a computer-readable storage medium storing a computer program thereon, characterized in that, when executed by a processor, the program implements the recurrent neural network training method based on Ising machine optimization as described above. The storage medium can be any medium capable of storing program code, such as ROM, RAM, magnetic disk, optical disk, or USB flash drive. When the program is loaded into a computer and run, all the method steps described in this invention can be executed.
[0109] Example 2
[0110] This embodiment provides a recurrent neural network training method based on Ising machine optimization, using the Logistic chaotic mapping prediction task as an example for illustration.
[0111] Step 1: Reconstructing the RNN dynamics system:
[0112] To map the RNN training problem to the Ising model, the dynamic equations of the classic leaky integral neuron RNN (RC framework) are first reconstructed; this is achieved by defining a new weight matrix W=W in W out +W r and the transformed state variable z t The closed-loop dynamics and loss function of the original RNN are transformed into a function with respect to new variables (such as W and W'). o The global quadratic energy function form of the RNN is obtained; this reconstruction process preserves the nonlinear expressive power of the RNN and allows the training loss to be represented in a format suitable for mapping to the Ising Hamiltonian. The final loss function form (as shown in Equation 21 of the paper) lays the foundation for subsequent QUBO mapping.
[0113] Step 2: QUBO mapping and discrete coding of the loss function.
[0114] The mean squared error (MSE) loss function (or simplified loss function) of the reconstructed RNN is encoded into a QUBO model. The specific method is as follows:
[0115] Variable binarization: Binarizing each weight in the continuous weight matrix (e.g., W1) to be optimized. Approximately represented by K binary variables Linear combination: and a fixed precision vector To represent. Then It can be approximated as:
[0116] (1)
[0117] in It is the kth element of the preset precision vector.
[0118] Constructing a QUBO matrix: Arrange all binary variables into a long vector:
[0119] And construct the accuracy matrix (in (where D×D is the identity matrix) such that .Will Substituting into the expansion of the MSE loss function (e.g.) ,in For input data, (With the objective as the target), we can obtain the form of a standard QUBO problem:
[0120] Through the above steps, the problem of minimizing the RNN loss function is successfully transformed into a QUBO problem. Based on this method, the decimal weights of the recurrent neural network loss function after dynamic reconstruction are binary discretely encoded, and the weight matrix is binary discrete according to the following encoding method:
[0121] (2)
[0122] Where C represents the offset matrix indicating the lower bound of the weight values, this allows the model to capture both positive and negative parameters. This yields the QUBO model generated by the RNN mapping.
[0123] (3)
[0124] The binary variables of the QUBO model can be converted into Ising spins, and then substituted into equation (3) to obtain the Ising model.
[0125] (4)
[0126] (5)
[0127] Equation (5) represents the Ising model derived from the loss function trained on the RNN, s For Isin spin, among which =-1 corresponds to =0, and =1 corresponds to =1; the Q matrix in equation (5) is determined by the input data. ,Target Together with the precision matrix P, c is a constant term; the model can then be uploaded to a physical Ising machine, or solved using a simulated Ising machine algorithm and heuristic algorithms such as classical simulated annealing. The solver will return a set of optimal (or near-optimal) binary variable configurations, which can then be decoded back to the original weights. I represents a column vector of all 1s with the same dimension as S; this vector is introduced to represent the constant bias term in vector form during the spin transformation from binary variables to Ising as defined in equation (5). Thus, the training problem of the RNN is transformed into finding an Ising model that minimizes energy.
[0128] Step 3: Solve the QUBO problem using the Ising machine.
[0129] The coefficients of the Ising model transformed from the QUBO model constructed in step 2 (i.e., The matrix is transformed into the coupling matrix J and bias field h of the physical Ising machine and loaded into the physical Ising machine (such as the optical Ising machine) or the simulated Ising machine.
[0130] Physical evolution / annealing: Initiating the physical evolution process of the Ising machine; for example, in a coherent Ising machine based on a degenerate optical parametric oscillator, the system undergoes collective symmetry breaking under the action of pump light, and the phase of each optical pulse (representing the spin state) will naturally evolve to the state that minimizes the loss of the entire network (corresponding to the energy of the Ising model).
[0131] Result sampling: After physical annealing, the final state configuration of all spins is read from the Ising machine; this configuration corresponds to an optimal or near-optimal solution to the original QUBO problem.
[0132] Step 4: Decoding and Weight Restoration
[0133] The spin state configuration of the Ising machine obtained in step 3 is decoded in reverse according to the encoding rules in step 2 to restore the optimized continuous weight values of the RNN. These weight values are then reloaded into the RNN model, thus completing the entire training process.
[0134] It should be noted that there are multiple alternative implementations of the technical solution of this invention:
[0135] Different QUBO mapping methods: Different dynamic system reconstruction methods or other mathematical techniques that map the MSE loss function to a QUBO problem can be used, such as using auxiliary variables to reduce a higher-order problem to a second-order problem. The precision vector p can also be encoded using other methods such as uniform interval values.
[0136] Different types of Ising machines: In addition to optical coherent Ising machines, quantum annealing machines (such as D-Wave systems), optoelectronic Ising machines, electronic Ising machines based on FPGA or CMOS circuits, and Ising machines based on superconductors or atomic systems can also be used.
[0137] Different optimization solvers: During the software simulation and verification phase, heuristic algorithms such as simulated annealing, tabu search, and genetic algorithms can be used as alternatives to solve the QUBO problem.
[0138] Different RNN variants: The core idea of this invention is also applicable to other types of recurrent neural networks such as Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRU), only requiring adjustment of the reconstruction method of their dynamic system.
[0139] The recurrent neural network training method and system based on Ising machine optimization provided by this invention can be widely applied to various time series forecasting tasks, including but not limited to financial market forecasting, weather forecasting, industrial monitoring, and chaotic system modeling. Utilizing the physical acceleration advantage of the Ising machine, training time can be significantly shortened, and prediction accuracy and stability can be improved, demonstrating promising prospects for industrial applications.
[0140] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
[0141] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A training method for recurrent neural networks based on Ising machine optimization, characterized in that, Includes the following steps: The weight training task of the recurrent neural network is transformed into a quadratic unconstrained binary optimization problem; The quadratic unconstrained binary optimization problem is mapped to the Ising model; The Ising model is loaded into the Ising machine, and the spin configuration corresponding to the minimum energy state of the Ising model is obtained through the physical evolution or annealing process of the Ising machine. The weights of the recurrent neural network are determined based on the spin configuration and loaded into the recurrent neural network to complete the training.
2. The method according to claim 1, characterized in that, The transformation of the recurrent neural network weight training task into a quadratic unconstrained binary optimization problem specifically includes: The recurrent neural network is reconstructed into a dynamic system, and its closed-loop dynamics and loss function are transformed into a global quadratic energy function. The reconstructed loss function is encoded as a quadratic unconstrained binary optimization model.
3. The method according to claim 2, characterized in that, Encoding the reconstructed loss function into a quadratic unconstrained binary optimization model includes performing binary discrete encoding on the continuous weights to be optimized: Each weight Approximately represented by K binary variables Linear combination: ； in, Indicates the first Each weight has a value range of 1. , K represents the total number of weights to be optimized; K is the number of binary variables used for each weight. Preset precision vector The There are 1 element used to control the weight precision when combining binary variables; by substituting the binary representations of all weight variables into the loss function, we obtain the quadratic unconstrained binary optimization form.
4. The method according to claim 3, characterized in that, The step of substituting the binary representations of all weight variables into the loss function to obtain the quadratic unconstrained binary optimization form specifically includes: Constructing the precision matrix This makes the weight matrix ,in It is a long vector consisting of all binary variables; Will Substituting the mean squared error loss function, we obtain a quadratic unconstrained binary optimization model: ； in, The input data matrix has each row corresponding to an input vector at a time step. This is the target output matrix, where each row corresponds to the target output value at the corresponding time step. This indicates the matrix transpose.
5. The method according to claim 4, characterized in that, The binary discrete encoding includes introducing an offset matrix C to support the representation of positive and negative weights, the weight matrix being expressed as: ； in, To and A constant offset matrix of the same dimension is used to set the lower bound of the weight values; The corresponding quadratic unconstrained binary optimization model is: 。 6. The method according to claim 1, characterized in that, The process of mapping a quadratic unconstrained binary optimization problem to the Ising model includes converting binary variables into Ising spins. : ； in, It is a binary variable. For the corresponding Ising spin; Substituting the transformed spin variable into the quadratic unconstrained binary optimization model, we obtain the energy function of the Ising model: ； For all Isin spin The long vector formed; For the input data matrix Target output matrix And precision matrix The derived coupling matrix; This is a constant term that is independent of the spin variable.
7. The method according to claim 1, characterized in that, The Ising machine includes at least one of the following types: Optical Ising machine, quantum annealing machine, optoelectronic Ising machine, electronic Ising machine based on FPGA or CMOS circuits, and Ising machine based on superconductors or atomic systems.
8. The method according to claim 1, characterized in that, The recurrent neural network includes at least one of leaky integral neurons (RNN), long short-term memory networks (LSM), or gated recurrent units.
9. A recurrent neural network training system based on Ising machine optimization, characterized in that, include: The QUBO mapping module is used to transform the weight training task of a recurrent neural network into a quadratic unconstrained binary optimization problem and map it to the Ising model; The Ising machine solver module is used to load the Ising model into the Ising machine and obtain the spin configuration corresponding to the energy minimum state of the Ising model through the physical evolution or annealing process of the Ising machine. The weight loading module is used to determine the weights of the recurrent neural network according to the spin configuration and load them into the recurrent neural network to complete the training.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the recurrent neural network training method based on Ising machine optimization as described in any one of claims 1 to 8.