Inter-group Indexing for Neural Network Weight Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As artificial neural networks grow in size, the increasing number of synaptic connections leads to excessive memory requirements for weight values, often necessitating the use of slower virtual memory, which significantly slows down performance, and existing solutions like weight clustering result in a dramatic loss of accuracy.
Innovation Solution
The method involves arranging weights in ascending order and dividing them into groups based on linear patterns, designating one group as a base group and others as dependent groups, where only the base group's values are stored in memory, and dependent groups are indexed to the base group, allowing for accurate synthesis without storing actual dependent group values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the number of weights in the artificial neural network is increased to improve network size and capability, then the network can perform more complex cognitive tasks, but the memory requirement for weight values increases excessively, leading to use of slower virtual memory and dramatically reduced performance
Solution Approach 1:
The patent segments the weight values into distinct groups (base group and dependent groups) based on their magnitude ranges. This segmentation allows the system to store only the base group weights in memory while representing dependent group weights through indices, thereby reducing the total memory requirement while maintaining the full functionality of the neural network with increased size and capability
2Quantity of substance
If weight clustering is used to reduce memory requirements, then memory usage is reduced, but accuracy is dramatically lost
Solution Approach 1:
The patent introduces an intermediary indexing mechanism that maps dependent group weights to base group weights through index values. This intermediary approach allows the system to reduce memory usage by not storing all weight values directly, while maintaining accuracy through the precise index-based representation that preserves the relative magnitude relationships between weights, avoiding the accuracy loss associated with direct weight clustering
Data Source
AI summary
A computer-implemented method for representation of weight values in an artificial neural network using inter-group indexing may include, in an artificial neural network that includes neurons and synaptic connections between the neurons with each of the synaptic connections including a weight, arranging the weights in ascending order. The method may also include dividing the arranged weights into groups based on approximately linear patterns of the weights. The method may further include designating a base group and the other groups as dependent groups. The method may also include storing in memory values of the weights in the base group. The method may further include storing in the memory a group index for each of the dependent groups. The method may also include storing in the memory an index for each of the weights in the dependent groups corresponding to one of the weights in the base group without storing in the memory values of the weights in the dependent groups.