Token-Based Integrated-Circuit Architecture for Low-Latency Edge AI
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional AI processing systems are bulky, energy-inefficient, and suffer from latency issues when deployed in edge devices, requiring significant compute resources and continuous access to remote computing systems, which is not feasible in many applications due to infrastructure limitations.
Innovation Solution
An integrated circuit architecture with a token-based governance module and network-on-chip system enables efficient, asynchronous data processing and reduced memory requirements, allowing local AI deployment in edge devices with real-time inference capabilities and energy efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If traditional digital circuitry is used for weighted sum calculations in neural network applications, then the system can perform computations, but the circuitry area and energy consumption increase significantly due to the need for large digital memory to store weights
Solution Approach 1:
The patent replaces traditional digital memory-based weight storage with analog crossbar array memory that uses conductance values of physical resistors to store weights. This physical representation eliminates the need for digital memory circuits and associated readout logic, dramatically reducing circuitry area and energy consumption while maintaining computational capability.
Solution Approach 2:
The patent changes the representation parameter of weights from digital values stored in memory to analog conductance values of physical resistors. This parameter change enables direct physical multiplication through Ohm's law and Kirchhoff's current law, eliminating the need for digital multiplication circuits and reducing energy consumption.
2Power
If traditional digital circuitry is used for weighted sum calculations, then computations can be performed, but computing time increases due to the need to access weights from digital memory
Solution Approach 1:
The patent replaces digital memory access operations with direct physical signal propagation through the crossbar array. Weights are stored as static conductance values in resistors, and computations occur through passive electrical signal flow, eliminating the time required for digital memory readout and data transfer operations.
Solution Approach 2:
The patent pre-sets the conductance values of resistors in the crossbar array to represent the neural network weights before computation begins. This preliminary configuration allows the system to perform computations using only signal propagation, without needing to retrieve weight values during the computation process.
3Power
If remote computing systems are used for AI processing, then compute resources are available, but latency increases due to network transmission delays
Solution Approach 1:
The patent segments the AI processing system into local edge devices equipped with neuromorphic computing circuits. This segmentation enables inference operations to be performed locally at the edge, eliminating the need for continuous network communication with remote cloud systems and reducing latency for real-time applications.
4Ease of operation
If traditional computing systems are deployed in edge devices, then local processing is enabled, but device size and energy consumption become prohibitive
Solution Approach 1:
The patent replaces traditional digital computing circuits with neuromorphic computing circuits that use physical resistor networks for weight storage and analog signal propagation for computation. This substitution dramatically reduces the physical footprint required for neural network processing, enabling local AI capabilities in compact edge devices.
Solution Approach 2:
The crossbar array structure serves multiple functions simultaneously: it stores weights as conductance values, performs multiplication through signal propagation, and sums results through current aggregation. This multi-functionality eliminates the need for separate memory and computation circuits, reducing overall device size.
Data Source
AI summary
A system and method for automated data propagation and automated data processing within an integrated circuit includes an intelligence processing integrated circuit comprising at least one intelligence processing pipeline, wherein the at least one intelligence processing pipeline includes: a main data buffer that stores input data; a plurality of distinct intelligence processing tiles, wherein each distinct intelligence processing tile includes a computing circuit and a local data buffer; a token-based governance module, the token-based governance module implementing: a first token-based control data structure; a second token-based control data structure, wherein the first token-based control data structure and the second-token based control data operate in cooperation to control an automated flow of the input data and/or an automated processing of the input data through the at least one intelligence processing pipeline.