Transistor TRNG for Neural Network Dropout and Initialization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current random number generators used for dropout and weight initialization in neural networks suffer from correlation and unexpected periods, leading to inefficient learning due to the need for batch normalization, which increases computational time by about 30%.
Innovation Solution
A true-random number signal generator utilizing a transistor with trapping layers to generate random noise signals, where the probability distribution is controlled by the magnitude of the gate voltage, minimizing correlation and period issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pseudo random numbers are used for dropout and weight initialization, then the neural network can perform basic random operations, but the correlation and unexpected period of the random numbers make learning inefficient
Solution Approach 1:
The patent replaces algorithmic pseudo-random number generation with a physical random number generator based on transistor noise. The physical system uses inherent electrical noise from transistor operation, which is fundamentally unpredictable and free from the periodicity and correlation issues of algorithmic approaches. This substitution of physical phenomenon for computational algorithm directly resolves the reliability-productivity contradiction.
Solution Approach 2:
The patent changes the fundamental parameter of randomness generation from computational (algorithmic) to physical (electrical noise). By utilizing the inherent noise characteristics of transistor operation and capturing it through trapping layers, the system achieves truly random numbers with no correlation or period, thereby improving both reliability and learning efficiency simultaneously.
2Reliability
If batch normalization is applied to correct random number correlation, then the random number quality improves, but computational time increases by about 30%
Solution Approach 1:
The patent performs preliminary action by generating high-quality true random numbers at the source (transistor level) before they are needed for neural network operations. The random number generator produces uncorrelated, properly distributed random values in advance, eliminating the need for subsequent batch normalization processing. This preliminary generation of correct random numbers prevents the time loss that would otherwise be required for correction.
Solution Approach 2:
The patent extracts random number generation from the computational pipeline (removing it from software/algorithms) and places it in the physical hardware layer. By extracting the random number source to the physical domain using transistor noise, the system obtains inherently high-quality random numbers without requiring computational correction through batch normalization, thus avoiding the 30% computational time penalty.
3Reliability
If a true random number generator using transistor noise is implemented, then correlation and period issues are eliminated, but device complexity increases due to multiple trapping layers
Solution Approach 1:
The patent applies nesting by placing multiple trapping layers within the gate structure of a single transistor. The trapping layers are nested between the gate electrode and gate insulator, creating a compact hierarchical structure where simpler elements (trapping layers) are contained within a more complex structure (transistor gate). This nesting achieves true random number generation with controlled probability distribution while maintaining a relatively compact device footprint.
Solution Approach 2:
The transistor structure serves multiple functions: it generates random noise through channel conduction, stores random states through carrier trapping in the trapping layers, and controls the output probability distribution through gate voltage modulation. This multi-functionality consolidates what would otherwise require separate components into a single device, reducing overall system complexity despite the enhanced internal structure.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution provides true-random number signals with reduced correlation and period, enhancing the efficiency of neural network learning by eliminating the need for batch normalization, thus reducing computational overhead.
Implementation Method 1
Each of the first and second trapping layers includes one or more traps. The one or more traps of the first and second trapping layers are configured to capture or release one or more carriers flowing through the channel region. In response to an applied signal, the random noise signal is generated as a function as the one or more carriers being captured or released by the one or more traps.
Data Source
AI summary
A random number signal generator used for performing dropout or weight initialization for a node in a neural network. The random number signal generator includes a transistor which generates a random noise signal. The transistor includes a substrate, source and drain regions formed in the substrate, a first insulating layer formed over a channel of the transistor, a first trapping layer formed over the first insulating layer, a second insulating layer formed over the first trapping layer, and a second trapping layer formed over the second insulating layer. One or more traps in the first and second trapping layers are configured to capture or release one or more carriers flowing through the channel region. The random noise signal is generated as a function of one or more carrier being captured or released by the one or more traps.