Analog Learning Engine for Neural Network Weight Tuning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital machine learning is cumbersome due to the large number of neurons in neural networks, making calculations of derivatives and error contours time-consuming and requiring significant mathematical computing power.
Innovation Solution
An analog neural network error contour generation mechanism that perturbs weights and biases associated with analog neurons to measure errors, using a backpropagation mechanism with mini-batches of training samples to modify weights based on error functions and local activation derivatives, and implementing weight tuning circuits with switched charge reservoirs for accurate adjustments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If digital machine learning is used to train neural networks, then learning accuracy can be achieved, but the calculation process becomes cumbersome and time-consuming due to the large number of neurons and derivatives required
Solution Approach 1:
The patent replaces digital computational systems with an analog neural network system that uses continuous physical signals (voltages and currents) to perform learning operations. The analog neurons and synapses physically embody the neural network architecture, allowing parallel computation of weight adjustments without requiring sequential digital calculations of derivatives and error contours.
Solution Approach 2:
The patent changes the domain of computation from discrete digital values to continuous analog parameters (voltages and currents). This allows the system to represent and manipulate weight values, activation levels, and error signals as continuous physical quantities, enabling more efficient gradient computation and weight updates through analog circuit operations rather than digital arithmetic.
2Measurement precision
If digital machine learning is used to train neural networks, then learning accuracy can be achieved, but significant mathematical computing power is required
Solution Approach 1:
The patent replaces digital computational systems with an analog neural network system that uses continuous physical signals (voltages and currents) to perform learning operations. The analog neurons and synapses physically embody the neural network architecture, allowing parallel computation of weight adjustments without requiring sequential digital calculations of derivatives and error contours.
Solution Approach 2:
The patent merges the functions of computation, memory storage, and weight adjustment into a single integrated analog neural network structure. The physical connections between analog neurons inherently store weight values while simultaneously enabling computation, eliminating the need for separate digital processors to handle mathematical operations.
3Measurement precision
If the number of neurons in a neural network is increased to improve learning capability, then learning accuracy improves, but the complexity of calculations increases
Solution Approach 1:
The patent merges the functions of computation, memory storage, and weight adjustment into a single integrated analog neural network structure. The physical connections between analog neurons inherently store weight values while simultaneously enabling computation, eliminating the need for separate digital processors to handle mathematical operations.
Solution Approach 2:
The patent divides the neural network into modular analog neuron units and synapse elements that can be independently configured and connected. Each analog neuron operates as an independent computational unit with local weight adjustments, allowing complex networks to be built from simple, reusable building blocks without proportionally increasing overall system complexity.
Data Source
AI summary
A neural network learning mechanism has a device which perturbs analog neurons to measure an error which results from perturbations at different points within the neural network and modifies weights and biases to converge to a target.