Resistive Processing Unit Preconditioner for Linear Equation Solvers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for solving linear equations, such as Gaussian elimination, are inefficient in terms of execution complexity and memory requirements, especially when dealing with sparse matrices, and iterative solutions using approximate inverse preconditioners can be computationally intensive.
Innovation Solution
The implementation of hardware-accelerated application of preconditioners using analog resistive processing units (RPUs) for matrix-vector multiplication operations, which reduces energy consumption and computation time by storing and applying an approximate inverse preconditioning matrix in RPU arrays for iterative solutions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional methods like Gaussian elimination are used to solve linear equations, then computational accuracy is maintained, but execution complexity and memory requirements increase significantly
Solution Approach 1:
The patent replaces conventional digital computational methods (Gaussian elimination) with an analog physical system consisting of resistive processing units and memristive devices. The mechanical/digital computation process is substituted by physical electrical phenomena (Ohm's law, Kirchhoff's laws) occurring naturally in the resistive network, thereby reducing computational complexity while maintaining solution accuracy.
Solution Approach 2:
The patent segments the computational task into two distinct phases: (1) preprocessing phase where the inverse preconditioning matrix is computed using conventional digital methods and stored in memristive devices, and (2) execution phase where the analog resistive network performs matrix-vector multiplication naturally. This segmentation allows complex operations to be performed efficiently in the analog domain while keeping the system design manageable.
2Quantity of substance
If iterative solutions with approximate inverse preconditioners are used, then memory requirements are reduced, but computation time and energy consumption increase
Solution Approach 1:
The patent applies preliminary action by pre-computing the inverse preconditioning matrix using conventional digital methods and storing it in memristive devices before the actual solution process. This preliminary preparation enables subsequent iterative solutions to proceed much faster, as the analog resistive network can perform matrix-vector multiplications in parallel without the overhead of repeated digital computations.
Solution Approach 2:
The patent substitutes digital iterative computation with an analog physical system where the resistive network naturally performs the mathematical operations. This substitution dramatically reduces computation time for each iteration while maintaining the memory efficiency of iterative methods, as the analog system processes information physically rather than through sequential digital operations.
3Use of energy by moving object
If digital methods are used for matrix-vector multiplication, then precision is maintained, but energy consumption and computation time increase
Solution Approach 1:
The patent replaces energy-intensive digital matrix-vector multiplication operations with an analog resistive network that performs the same mathematical operations through physical electrical phenomena. According to Ohm's law and Kirchhoff's laws, the current flowing through the resistive network naturally computes the matrix-vector product, eliminating the need for sequential digital computations and significantly reducing energy consumption while increasing computation speed.
Solution Approach 2:
The analog resistive network performs matrix-vector multiplication autonomously through physical laws without requiring active control or intervention. The system self-organizes the computation through the natural flow of electrical current through the resistive elements, eliminating the need for complex control logic and reducing overall system energy consumption while maintaining high computational throughput.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach significantly reduces the number of iterations required to solve linear equations, improving computational efficiency and reducing energy consumption compared to digital methods, while handling sparse matrices effectively.
Implementation Method 1
The synaptic weights can be implemented using an array of resistive processing unit (RPU) cells having tunable resistive memory devices (e.g., tunable conductance), wherein conductance states of the RPU cells are encoded or otherwise mapped to the synaptic weights.
Implementation Method 2
performing analog matrix-vector multiplication operations on the preconditioning matrix and respective ones of the plurality of residual vectors to generate a plurality of output vectors
Implementation Method 3
The resistive processing unit comprises an array of cells which respectively comprise resistive devices
Data Source
AI summary
Techniques are provided to implement hardware accelerated application of preconditioners to solve linear equations. For example, a system includes a processor, and a resistive processing unit coupled to the processor. The resistive processing unit includes an array of cells which include respective resistive devices, wherein at least a portion of the resistive devices are tunable to encode entries of a preconditioning matrix which is storable in the array of cells. When the preconditioning matrix is stored in the array of cells, the processor is configured to apply the preconditioning matrix to a plurality of residual vectors by executing a process which includes performing analog matrix-vector multiplication operations on the preconditioning matrix and respective ones of the plurality of residual vectors to generate a plurality of output vectors used in one or more subsequent operations.