Cluster-Connected Neural Network for Sparse Deep Learning Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
State-of-the-art neural networks require specialized hardware for training and prediction due to their high memory and processing demands, making it impractical to run deep learning models on endpoint devices with limited resources.
Innovation Solution
Implementing a cluster-connected neural network architecture that is locally fully connected within clusters and globally sparsely connected between clusters, reducing the number of inter-cluster weights and enhancing computational efficiency and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a traditional fully-connected neural network is used, then the network can achieve high accuracy in deep learning tasks, but the memory requirements and processing demands become excessively high, making it impractical to run on endpoint devices with limited resources
Solution Approach 1:
The neural network is divided into multiple clusters of neurons, where each cluster is fully connected internally but sparsely connected to other clusters. This segmentation reduces the total number of connections and weights while maintaining the essential computational functionality, thereby reducing memory requirements and enabling deployment on resource-constrained devices
Solution Approach 2:
Different regions of the neural network have different connection densities: intra-cluster connections are dense (fully connected) to preserve local computational power and accuracy, while inter-cluster connections are sparse to reduce overall memory and processing demands. This local quality differentiation allows the network to maintain accuracy where needed while reducing resource consumption globally
2Measurement precision
If a traditional fully-connected neural network is used, then the network can process information comprehensively, but the computational speed during runtime prediction becomes too slow for real-time applications on devices with limited processing power
Solution Approach 1:
By segmenting the network into clusters with selective connectivity, the computational graph is simplified and more paths can be processed in parallel during runtime prediction. This reduces the computational burden on individual devices and enables faster inference speeds suitable for real-time applications
Solution Approach 2:
The network performs partial computations by eliminating unnecessary inter-cluster connections that would provide excessive computational paths. This selective computation approach maintains sufficient prediction accuracy while significantly improving computational speed by focusing resources only on essential calculations
3Productivity
If specialized hardware like GPUs is used for training and prediction, then the processing capacity and speed are sufficient for deep learning tasks, but the device complexity and cost increase, making it unavailable for most endpoint devices
Solution Approach 1:
The invention extracts the essential computational functionality from complex specialized hardware by designing a neural network architecture that can run efficiently on general-purpose processors. By removing the need for GPU-accelerated computing through architectural simplification, the system becomes deployable on standard endpoint devices without requiring expensive specialized hardware
Solution Approach 2:
The network architecture changes key parameters such as connection density and weight distribution to optimize performance for execution on resource-constrained devices. These parameter modifications enable the network to achieve acceptable processing capacity and speed using only standard hardware resources, eliminating the need for specialized accelerators
4Quantity of substance
If the number of weights in the neural network is reduced to decrease memory usage, then the network becomes more efficient and can run on limited devices, but the network capacity and accuracy may be compromised
Solution Approach 1:
The weight matrix is segmented into block-diagonal form corresponding to cluster assignments, where most inter-cluster weights are set to zero. This segmentation preserves the essential information flow within clusters while eliminating redundant inter-cluster connections, maintaining network accuracy with significantly fewer weights
Solution Approach 2:
The network employs local quality differentiation where intra-cluster connections maintain full weight capacity for accurate local processing, while inter-cluster connections use reduced or zero weights. This selective weight allocation preserves necessary computational accuracy while dramatically reducing the total number of weights and memory requirements
Data Source
AI summary
A device, system, and method is provided for training or prediction using a cluster-connected neural network. The cluster-connected neural network may be divided into a plurality of clusters of artificial neurons connected by weights or convolutional channels connected by convolutional filters. Within each cluster is a locally dense sub-network of intra-cluster weights or filters with a majority of pairs of neurons or channels connected by intra-cluster weights or filters that are co-activated together as an activation block during training or prediction. Outside each cluster is a globally sparse network of inter-cluster weights or filters with a minority of pairs of neurons or channels separated by a cluster border across different clusters connected by inter-cluster weights or filters. Training or predicting is performed using the cluster-connected neural network.