MOSFET Neural Network Layers With Local Weight and Bias Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning networks, particularly those employing Adaline and Madaline structures, suffer from high computational complexity (O(N^3) scalability issues and high power, computing, and memory resource consumption due to extensive data movement and weight/bias term transitions.
Innovation Solution
Implementing MOSFETs as layers in machine learning networks to simulate activation functions, allowing weight and bias terms to be stored locally, reducing data movement and power consumption by using MOSFETs in parallel to perform operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If traditional machine learning networks with extensive data movement are used, then computational capability is achieved, but power consumption and resource usage increase significantly
Solution Approach 1:
The patent combines multiple computational functions (activation functions, weight storage, bias storage) into a single integrated circuit device. The neural network layer integrates weight memory, bias memory, and computation units with activation function circuits, eliminating the need for separate data movement between distinct computational and memory components. This merging reduces power consumption while maintaining computational capability.
Solution Approach 2:
The patent introduces local weight memory and bias memory as intermediary storage between the computation units and external memory systems. These local memories serve as buffers that reduce the frequency and volume of data transfers across the network, thereby reducing power consumption associated with data movement while preserving the required computational operations.
2Device complexity
If data movement and weight/bias term transitions are extensive, then machine learning operations are performed, but computational complexity increases to O(N^3)
Solution Approach 1:
The patent segments the neural network into multiple independent layer units, each capable of performing computations locally. By dividing the network into discrete layers with local weight and bias storage, the system avoids the need for global data redistribution that causes O(N^3) complexity. Each layer operates semi-independently, reducing overall computational complexity while maintaining learning efficiency.
Solution Approach 2:
The patent implements local weight memory and bias memory at each computation unit or layer, rather than relying on centralized storage. This local quality allows each unit to access its parameters without requiring data movement across the entire network, thereby reducing computational complexity and improving operation efficiency simultaneously.
3Use of energy by stationary object
If weight and bias terms are stored remotely, then memory capacity is sufficient, but data movement increases power consumption
Solution Approach 1:
The patent implements a nested memory hierarchy where local weight memory and bias memory are embedded within or directly coupled to each computation unit, which is itself part of the larger neural network system. This nested structure allows frequently accessed weight and bias terms to be stored locally, reducing the need for repeated data movement from remote memory while maintaining overall memory capacity through the hierarchical arrangement.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Reduces computational complexity and power consumption by minimizing data movement and resource usage, enabling efficient machine learning operations with lower power requirements.
Implementation Method 1
a channel electrically connecting the source terminal to the drain terminal; a gate proximate to the channel, wherein the gate is configured to control electrical current flowing from the source terminal to the drain terminal via the channel based on a voltage-from-gate-to-source being applied at the gate
Implementation Method 2
a gate control component proximate to the gate, wherein the gate control component is configured to modify a transconductance of the semiconductor device assembly to modulate a weight of the computation of the machine learning network
Implementation Method 3
using the MOSFET to implement an activation function of the computation
Data Source
AI summary
In some implementations, a machine learning device may perform, using a metal-oxide-semiconductor field-effect transistor (MOSFET), a computation of a machine learning network, wherein performing the computation of the machine learning network includes: using the MOSFET to implement an activation function of the computation, and performing at least one of: adjusting a transconductance of the MOSFET to modulate a weight of the computation, or adjusting a threshold voltage of the MOSFET to modulate a bias of the computation.