Dynamic Neural Network Node Masking for Computational Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial neural networks face inefficiencies due to unnecessary calculations and energy consumption from performing operations involving zero or near-zero weight multiplications, which are not effectively addressed by existing pruning methods that require static assumptions and apply uniformly across all inputs.
Innovation Solution
Implementing a dynamic approach using mask bits to identify and bypass 'weak' nodes with values below a threshold, reducing the number of calculations by condensing matrix equations and focusing on 'strong' nodes, thereby enhancing computational efficiency during neural network execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional dense neural networks with many nodes are used, then classification accuracy is maintained, but computational complexity and energy consumption increase significantly
Solution Approach 1:
The patent extracts and removes unnecessary computations from the neural network by identifying and eliminating zero or near-zero weight multiplications. This is achieved through dynamic masking that selectively disables irrelevant nodes and connections, keeping only the essential computational paths that contribute to accurate classification.
Solution Approach 2:
The patent applies dynamic computation by adjusting the network structure at runtime based on input characteristics. The masking mechanism dynamically enables or disables computational paths depending on the specific input data, allowing the network to adapt its complexity to the actual task requirements rather than maintaining fixed dense connectivity.
2Productivity
If static pruning methods are applied to reduce network size, then computational requirements decrease, but accuracy is compromised due to uniform application across all inputs
Solution Approach 1:
The patent applies local quality by treating different parts of the neural network differently based on their specific contribution to the classification task. Instead of uniform pruning, the dynamic masking mechanism selectively applies computation reduction only to specific nodes and connections that have zero or near-zero weights for the current input, preserving accuracy while improving efficiency.
Solution Approach 2:
The patent transitions from static to dynamic pruning by evaluating weight values at runtime and adjusting the computational graph accordingly. The masking mechanism is applied dynamically based on the specific input data and current network state, allowing the system to maintain accuracy for relevant computations while eliminating unnecessary operations.
3Loss of information
If all nodes are processed in dense layers, then complete feature extraction is achieved, but energy consumption increases due to unnecessary zero multiplications
Solution Approach 1:
The patent extracts only the meaningful computational operations by identifying and removing zero or near-zero weight multiplications. The dynamic masking mechanism selectively disables irrelevant computational paths, ensuring that energy is consumed only for computations that actually contribute to feature extraction and classification.
Solution Approach 2:
The patent applies partial action by performing computations only where necessary. Instead of processing all nodes uniformly, the dynamic masking mechanism enables computation only for nodes with significant weight values, performing exactly the amount of computation needed to maintain feature extraction completeness without excessive energy consumption.
Data Source
AI summary
A system for executing an artificial neural network having a plurality of interconnected nodes, the system includes a memory storing weight values of the neural network. The memory can be configured to a store node value and a mask bit value for each of the plurality of nodes of the neural network. Further the system can include multiply and accumulate (MAC) units to perform operations for determining node values. The system includes a control unit circuitry that, during execution of the neural network, dynamically controls operations of the MAC units to cause a reduction in a number of calculations to be performed by the MAC units. The control unit circuitry causes the MAC units to perform operations involving a subset of the plurality of nodes to avoid performing operations involving nodes of the plurality nodes that are outside of the subset.