Neural Network Data Compression and Validity Gating for Lower Power
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network training methods face challenges in improving learning performance while reducing power consumption, particularly due to issues like overfitting and inefficient data processing.
Innovation Solution
The proposed solution involves a control apparatus that uses an encoding circuit to compress data sequences and a decoding circuit to manage validity determination sequences, thereby optimizing data input and output operations in neural networks. This approach includes techniques like dropout and zero gating to improve learning performance and reduce power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional neural network training methods are used, then learning performance can be improved, but power consumption increases
Solution Approach 1:
The patent extracts and processes only valid data points from data sequences, separating them from invalid or redundant data. The validity determination sequence identifies which data points should be processed, allowing the system to extract only necessary computations and omit unnecessary operations, thereby reducing power consumption while maintaining learning performance.
Solution Approach 2:
The patent implements partial action by processing only a subset of data points that are determined to be valid. Instead of processing all data sequences completely, the system performs computations only on valid data points identified through the validity determination sequence, reducing overall computational load and power consumption while maintaining adequate learning performance.
2Reliability
If data processing operations are increased to improve learning performance, then overfitting may be reduced, but computational complexity increases
Solution Approach 1:
The patent extracts only valid data points from data sequences using a validity determination sequence. This extraction mechanism separates necessary computations from unnecessary ones, reducing computational complexity by eliminating operations on invalid data while maintaining the processing of sufficient valid data to prevent overfitting and improve learning performance.
Solution Approach 2:
The patent segments data processing into two distinct phases: validity determination and actual computation. The validity determination sequence divides data points into valid and invalid categories, allowing the system to segment computational efforts and focus resources only on valid data, thereby reducing overall computational complexity while maintaining learning performance.
3Productivity
If all data sequences are processed completely, then data utilization is maximized, but power consumption increases
Solution Approach 1:
The patent extracts only valid data points from complete data sequences using the validity determination sequence. This allows the system to maximize data utilization by identifying and processing all valid data points while simultaneously reducing power consumption by omitting processing of invalid data points, achieving both goals without contradiction.
Solution Approach 2:
The patent implements partial action by processing only the necessary subset of data points that are determined to be valid. Instead of uniformly processing all data sequences completely, the system performs computations selectively on valid data points identified through the validity determination mechanism, optimizing the balance between data utilization and power consumption.
Data Source
Figure 1A
Figure 1B~2
Figure 3A
AI summary
A neural network deep learning data control apparatus includes: a memory; an encoding circuit configured to receive a data sequence, generate a compressed data sequence in which consecutive invalid bits in a bit string of the data sequence are compressed into a single bit of the compressed data sequence, generate a validity determination sequence indicating a valid bit and an invalid bit in a bit string of the compressed data sequence, and write the compressed data sequence and the validity determination sequence to the memory; and a decoding circuit configured to read the compressed data sequence and the validity determination sequence from the memory, and determine a bit in the bit string of the compressed data sequence set for transmission to a neural network circuit, based on the validity determination sequence, such that the neural network circuit omits an operation with respect to non-consecutive invalid bits.