Neural Network Inference Device Edge Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional hierarchical neural networks face challenges with increasing calculation and memory requirements due to the number of edges between layers, leading to resource shortages and decreased inference accuracy.
Innovation Solution
The proposed solution involves an inference device and method that reduces the number of edges between layers by implementing a neural network configuration where each node in one layer is connected only to a subset of nodes in the adjacent layers, using a specific indexing and weighting system to calculate activity levels efficiently, thereby reducing the calculation and memory demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of nodes in intermediate layers is increased to improve inference accuracy, then the classification accuracy is improved, but the calculation amount and memory amount increase in proportion to the square of the input layer nodes
Solution Approach 1:
The patent segments the neural network into multiple layers (input layer, multiple intermediate layers, output layer) and establishes selective connection patterns between them. Each intermediate layer is further segmented into multiple groups, where nodes in one group connect only to specific nodes in adjacent layers, rather than all-to-all connections. This segmentation reduces the total number of edges while maintaining sufficient computational capacity for accurate inference.
2Measurement precision
If full connections between all nodes in adjacent layers are implemented to maintain high inference accuracy, then the classification accuracy is maintained, but the calculation amount and memory amount drastically increase
Solution Approach 1:
The patent applies local quality by making different parts of the neural network have different connection densities. Specifically, nodes are divided into groups, and connections are established selectively between groups rather than uniformly across all nodes. This creates a non-uniform connection pattern where each local region (group of nodes) has optimized connections to its neighbors, reducing overall complexity while maintaining local processing capability for accurate inference.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
A first intermediate layer activity level calculator refers to an index stored in a first intermediate layer storage, acquires, from among activity levels of respective nodes in an input layer calculated by an input layer activity level calculator and weight for respective edges and bias values stored in the first intermediate layer storage, an activity level of each node in the input layer that is connected with each node in the first intermediate layer and weight for each of edges and a bias value thereof, and calculates an activity level of each node in the first intermediate layer using the activity level of each node in the first input layer and the weight for each of the edges and the bias value having been acquired. This is capable of reducing the calculation amount and the memory amount upon performing inference, and also obtaining higher inference accuracy.