Neural Network Kernel Configuration via Zero-Value Parameter Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network models, especially large ones, face challenges in resource-constrained devices due to high memory and processing demands, leading to increased processing time, storage requirements, power consumption, and complexity, particularly when dealing with static weight parameters that include zeros, which can result in inefficient resource utilization.
Innovation Solution
A device driver analyzes neural network parameters to identify and separate static weight parameters with zero values, reducing kernel sizes by eliminating these parameters and configuring hardware to efficiently map and process the remaining non-zero parameters, thereby optimizing resource utilization and reducing unnecessary computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural network models with static weight parameters are processed using conventional methods, then processing can be performed, but resource utilization is inefficient due to unnecessary multiplications with zero values
Solution Approach 1:
The patent extracts and removes zero-value weight parameters from the neural network kernel before processing. By identifying and eliminating these redundant parameters, the system avoids unnecessary multiplication operations, thereby improving processing efficiency and reducing power consumption without affecting the neural network's computational accuracy.
Solution Approach 2:
The patent changes the parameter representation by separating static weight parameters into non-zero and zero-value components. This parameter transformation allows the system to optimize the kernel size dynamically, removing zero-value parameters from active processing and thereby reducing the computational burden and energy consumption.
2Power
If large neural network models are deployed in resource-constrained devices, then more powerful processing is available, but memory and storage demands increase significantly
Solution Approach 1:
The patent extracts redundant zero-value parameters from the neural network model, thereby reducing the effective kernel size. This extraction reduces the memory and storage requirements for storing weight parameters while maintaining the essential processing capabilities, making large neural network models more suitable for resource-constrained devices.
Solution Approach 2:
The patent applies local optimization by treating different weight parameters differently based on their values. Non-zero parameters are retained and processed, while zero-value parameters are removed or marked for skipping. This local differentiation optimizes memory usage by storing only necessary parameters while maintaining processing power for relevant computations.
3Reliability
If static weight parameters including zeros are processed, then complete neural network computation is performed, but processing time increases due to unnecessary operations
Solution Approach 1:
The patent extracts and removes zero-value weight parameters from the computation pipeline. By eliminating these parameters beforehand, the system avoids unnecessary multiplication and addition operations during inference, significantly reducing processing time while maintaining complete and accurate neural network computation through the retained non-zero parameters.
Solution Approach 2:
The patent performs preliminary analysis and removal of zero-value parameters before the actual neural network inference. This preliminary action prepares an optimized kernel that excludes redundant parameters, allowing the main processing stage to focus only on necessary computations, thereby reducing overall processing time without compromising computation completeness.
4Ease of operation
If conventional neural network processing is used, then standard operations are performed, but device complexity increases due to resource demands
Solution Approach 1:
The patent extracts and removes redundant zero-value parameters, thereby simplifying the neural network kernel structure. This reduction in parameter count decreases the complexity of the processing system required to handle the network, making it more suitable for resource-constrained devices while maintaining ease of operation through automated parameter management.
Solution Approach 2:
The patent transforms the parameter representation by separating and eliminating zero-value weights, changing the effective dimensionality of the processing task. This parameter change reduces the complexity of hardware or software systems needed to execute the neural network, as fewer parameters require storage, transmission, and computation.
Data Source
AI summary
Subject matter disclosed herein may relate to storage and/or processing of signals and/or states representative of neural network parameters in a computing device, and may relate more particularly to configuring circuitry in a computing device to process signals and/or states representative of neural network parameters.