On-Chip Sparse Coding Neural Network Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware designs for sparse coding algorithms face challenges in implementing them on a single chip due to high power consumption and the need for costly and slow off-chip memory and processing.
Innovation Solution
A hardware architecture is proposed that integrates a sparse coding neural network on a single chip, utilizing a scalable multi-layer architecture with neuron clusters connected in a bus structure and clusters connected in a ring structure, optimizing memory usage and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by stationary object
If conventional hardware designs are used to implement sparse coding algorithms on a single chip, then the algorithm can be implemented, but power consumption is high and processing speed is slow due to reliance on off-chip memory and processing
Solution Approach 1:
The patent combines memory and processing units into a unified on-chip architecture, eliminating the separation between off-chip memory and processing components. This integration allows data to be processed directly where it is stored, reducing power consumption from memory access operations and eliminating the speed bottlenecks associated with off-chip communication.
Solution Approach 2:
The architecture segments the neural network into multiple neuron clusters that can operate in parallel on the chip. Each cluster processes specific portions of the sparse coding algorithm independently, enabling simultaneous execution of multiple computational tasks and thereby increasing overall processing throughput while maintaining low power consumption per unit.
2Productivity
If parallel operations of hundreds of model neurons are implemented, then feature extraction capability is improved, but the number of on-chip interconnects and memory bandwidth requirements increase
Solution Approach 1:
The neural network is divided into multiple neuron clusters, each handling a subset of neurons. This segmentation reduces the interconnect complexity within each cluster while maintaining the overall parallel processing capability across all clusters. The modular structure allows hundreds of neurons to operate in parallel without requiring a proportional increase in interconnect complexity.
Solution Approach 2:
The patent organizes neuron clusters in a two-dimensional array structure rather than a flat one-dimensional arrangement. This spatial organization in multiple dimensions reduces the number of direct interconnects needed by allowing local communication within clusters and hierarchical communication between clusters, thereby supporting hundreds of parallel neurons with manageable interconnect complexity.
3Measurement precision
If a complete family of features is developed for natural images, then classification accuracy is improved, but the computation required for classification increases
Solution Approach 1:
The architecture implements local competitive algorithms within each neuron cluster where only locally relevant features compete for activation. This local processing approach allows the system to develop a complete family of features for accurate classification while reducing the computational burden by limiting interactions to local neighborhoods rather than requiring global computation across all features simultaneously.
Data Source
AI summary
A sparse coding system having a neural network including a plurality of neurons each having a respective feature associated therewith and each being configured to be electrically connected to every other neuron in the network and to a portion of an input dataset. The plurality of neurons are arranged in a plurality of neuron clusters each having a respective subset of the plurality of neurons, and the neurons in each cluster are electrically connected to one another in a bus structure, and the plurality of clusters are electrically connected together in a ring structure. Also provided is a sparse coding system that includes an inference module configured to extract features from an input image containing an object, wherein the inference module has an implementation of a sparse coding algorithm, and a classifier configured to classify the object in the input image based on the extracted features.