Mixed-Signal Neural Network Training via Channel Equalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional digital circuitry for implementing artificial intelligence models, such as neural networks, requires significant resources in terms of circuitry area, energy, and compute power, leading to large, bulky devices with high latency and energy inefficiency, especially when deployed in edge devices for real-time inference.
Innovation Solution
The development of advanced mixed-signal integrated circuits that enhance the inferential accuracy of artificial neural networks through channel equalization and dynamic composite scaling factors, allowing for efficient training and deployment of AI models in edge devices without the need for large memory and energy-intensive processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional digital circuitry is used for weighted sum calculations in neural networks, then the model can be trained and deployed, but the circuitry area and memory requirements become excessively large
Solution Approach 1:
The patent replaces traditional digital circuitry with analog circuitry to perform weighted sum calculations. Analog circuits use continuous voltage or current signals to represent data, eliminating the need for digital memory storage of weights and enabling computations to be performed through physical circuit operations rather than digital processing, thereby dramatically reducing circuitry area.
Solution Approach 2:
The patent changes the fundamental parameter representation from digital (discrete binary values) to analog (continuous voltage/current levels). This parameter change allows weights and inputs to be represented by analog signals that can be multiplied and summed through passive circuit elements, reducing the need for large digital memory arrays and complex digital logic circuits.
2Measurement precision
If traditional digital circuitry with large memory is used to store weights, then neural network accuracy is maintained, but energy consumption increases significantly
Solution Approach 1:
The patent substitutes energy-intensive digital memory access and processing with energy-efficient analog circuit operations. Analog circuits perform weighted sum calculations through passive electrical operations (multiplication via conductance, summation via current addition) that consume minimal power compared to active digital memory access and processing, while maintaining inference accuracy through proper analog signal management.
Solution Approach 2:
The patent extracts the weight storage function from active digital memory and embeds weights directly into the analog circuit topology through conductance values of passive elements. This extraction eliminates the need for separate memory structures and their associated energy consumption for data retrieval and processing.
3Power
If remote computing systems are used for neural network processing, then computational power is sufficient, but latency increases due to network transmission delays
Solution Approach 1:
The patent segments the neural network processing functionality and embeds it directly into the edge device through integrated analog circuitry. This segmentation allows the device to perform inference operations locally without relying on remote computing systems, eliminating network transmission delays while maintaining sufficient computational power through specialized analog processing circuits.
Solution Approach 2:
The patent introduces an intermediate analog processing layer at the edge device that serves as a mediator between the physical sensor inputs and the remote computing system. This intermediary performs preliminary weighted sum calculations and signal conditioning locally, reducing the data transmission burden and eliminating inference latency for time-critical operations.
4Adaptability or versatility
If more digital memory circuitry is added to accommodate neural network weights, then model capacity increases, but the device becomes larger and more energy-intensive
Solution Approach 1:
The patent replaces energy-intensive digital memory and processing systems with analog circuit implementations that achieve the same functional capacity. Analog circuits use physical properties (conductance, capacitance) to encode and process neural network parameters, providing model capacity through circuit topology rather than digital memory, thereby reducing energy intensity while maintaining or enhancing adaptability.
Data Source
AI summary
A system and method for enhancing inferential accuracy of an artificial neural network during training includes during a simulated training of an artificial neural network identifying channel feedback values of a plurality of distinct channels of a layer of the artificial neural network based on an input of a training batch; if the channel feedback values do not satisfy a channel signal range threshold, computing a channel equalization factor based on the channel feedback values; identifying a layer feedback value based on the input of the training batch; and if the layer feedback value does not satisfy a layer signal range threshold, identifying a composite scaling factor based on the layer feedback values; during a non-simulated training of the artificial neural network, providing training inputs of: the training batch; the composite scaling factor; the channel equalization factor; and training the artificial neural network based on the training inputs.