Neural Network Weight Storage with Dual ECC Protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage devices for neural networks lack efficient methods to differentiate and protect varying types of data, such as weights, input files, and results, which are accessed and modified frequently, leading to suboptimal access times and storage efficiency.
Innovation Solution
Implementing a dual-error correction code system in memory devices, where a stronger error correction code is used for essential weights and a weaker code for prunable weights, and utilizing different memory types and configurations based on data importance and access frequency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single strong error correction code is used for all neural network data, then data reliability is improved, but access time and storage efficiency deteriorate
Solution Approach 1:
The patent applies different error correction code strengths to different portions of neural network data based on their importance. Critical data (weights and results) receive strong ECC protection, while less critical data (input files) receive weaker or no ECC protection. This local differentiation resolves the contradiction by providing strong protection only where needed, rather than uniformly across all data.
Solution Approach 2:
The patent segments neural network data into distinct categories (input files, weights, results) and stores each category in separate memory portions with appropriately tailored error correction codes. This segmentation allows the system to optimize protection levels for each data type, improving overall access efficiency while maintaining reliability for critical data.
2Productivity
If all neural network data is stored with equal protection levels, then system simplicity is maintained, but access efficiency and storage optimization deteriorate
Solution Approach 1:
The patent implements different storage characteristics for different data portions based on their access patterns and importance. Weights are stored with high protection and optimized for reliability, input files are stored with minimal protection for fast access, and results are stored with appropriate protection levels. This local quality approach improves access efficiency for each data type without requiring complete system redesign.
3Reliability
If strong error correction is applied to all data, then data integrity is improved, but storage space and processing overhead increase
Solution Approach 1:
The patent applies error correction codes with varying strengths to different data portions. Critical data such as weights and results receive strong ECC protection to ensure integrity, while less critical data such as input files receive weaker or no ECC protection. This selective approach reduces overall storage space requirements and processing overhead while maintaining data integrity for the most important data elements.
Data Source
AI summary
In some implementations, the present disclosure relates to a method. The method includes obtaining a set of weights for a neural network comprising a plurality of nodes and a plurality of connections between the plurality of nodes. The method also includes identifying a first subset of weights and a second subset of weights based on the set of weights. The first subset of weights comprises weights that used by the neural network. The second subset of weights comprises weights that are prunable. The method further includes storing the first subset of weights in a first portion of a memory. A first error correction code is used for the first portion of the memory. The method further includes storing the second subset of weights in a second portion of the memory. A second error correction code is used for the second portion of the memory. The second error correction code is weaker than the first error correction code.