Binarized Neural Network Weight Storage with Parity Error Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage devices face challenges in efficiently storing and processing neural networks due to the large space requirements of weights, which are exacerbated by the use of error correction codes that increase storage space and computational resources.
Innovation Solution
Implementing a binarized neural network with a parity check mechanism to determine errors in weights, allowing the system to selectively refrain from using erroneous weights and reduce storage and computational demands by using parity data instead of error correction codes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If error correction codes are used to protect neural network weights, then reliability is improved, but storage space and computational resources increase
Solution Approach 1:
The patent extracts only the essential error detection function from full error correction codes, implementing a simplified parity check mechanism that uses minimal additional storage (single parity bit per filter) while providing sufficient error detection capability for neural network weights
Solution Approach 2:
The patent changes the parameter of error protection from comprehensive error correction to basic parity-based error detection, reducing the complexity and storage requirements while maintaining adequate reliability for the application
2Measurement precision
If full precision weights are used in neural networks, then processing accuracy is improved, but storage space and computational resources increase
Solution Approach 1:
The patent changes the precision parameter of weights from full precision (32-bit or 64-bit floating point) to binarized values (0 or 1), dramatically reducing storage space and computational complexity while maintaining sufficient processing accuracy through the use of multiple filters
Solution Approach 2:
The patent segments the weight representation into multiple binarized filters, where each filter provides a coarse approximation and multiple filters together achieve the desired processing accuracy, trading representation complexity for computational efficiency
3Quantity of substance
If binarized neural networks are used, then storage space is reduced, but error detection capability must be simplified
Solution Approach 1:
The patent extracts only the essential error detection function from complex error correction codes, implementing a simplified parity check mechanism that uses minimal additional storage (single parity bit per filter) while providing sufficient error detection capability
Solution Approach 2:
The patent uses inexpensive parity bits as disposable error detection markers that can be easily computed and discarded, providing adequate error detection without the overhead of sophisticated error correction mechanisms
Data Source
AI summary
Systems and methods are disclosed for storing neural networks and weights for neural networks. In some implementations, a method is provided. The method includes storing a plurality of weights of a neural network comprising a plurality of nodes and a plurality of connections between the plurality of nodes. Each weight of the plurality of weights is associated with a connection of the plurality of connections. The neural network comprises a binarized neural network. The method also includes receiving input data to be processed by the neural network. The method further includes determining whether a set of weights of the plurality of weights comprises one or more errors. The method further includes refraining from using the set of weights to process the input data using the neural network in response to determining that the set of weights comprises the one or more errors.