Neural Network Sub-system Segmentation for Parameter Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for developing and testing neural networks face inefficiencies in parameter search execution time, leading to impractical evaluation times due to combinatorial explosion, especially when dealing with multiple parameters and their combinations.
Innovation Solution
The method involves serializing and parallelizing sub-systems of the neural network by determining one-way dependencies and independencies, and pruning input parameters that do not affect the sub-systems, thereby reducing the number of parameters to be evaluated.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If comprehensive parameter combinations are evaluated to ensure thorough testing, then measurement precision is improved, but loss of time increases due to combinatorial explosion
Solution Approach 1:
The patent segments the neural network into separable sub-systems with one-way dependencies, allowing independent evaluation of each sub-system. This division reduces the combinatorial explosion by evaluating sub-systems separately rather than testing all parameter combinations across the entire network simultaneously, thereby maintaining measurement precision while significantly reducing evaluation time.
Solution Approach 2:
The patent applies partial action by pruning input parameters that do not affect each sub-system. Instead of evaluating all possible parameters for every sub-system, the method identifies and removes irrelevant parameters, performing only the necessary evaluations required for accurate parameter search without wasting time on excessive or redundant parameter combinations.
2Measurement precision
If all input parameters are evaluated for each sub-system to ensure complete analysis, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent extracts and removes irrelevant input parameters from each sub-system through a pruning process. By identifying parameters that do not affect the sub-system's output and removing them, the method reduces computation and storage requirements while maintaining complete analysis of the relevant parameters, thus improving the ratio of analytical completeness to system complexity.
3Ease of operation
If serial processing of sub-systems is used to simplify dependency management, then ease of operation is improved, but productivity decreases
Solution Approach 1:
The patent segments the neural network into sub-systems with one-way dependencies, creating a hierarchical structure where upstream sub-systems feed into downstream sub-systems. This segmentation allows for simplified dependency management through serial processing of the hierarchy while enabling parallel processing within each level, thus maintaining ease of operation while improving overall productivity.
Solution Approach 2:
The patent introduces a hierarchical dimension to the processing structure, organizing sub-systems into multiple levels based on their dependency relationships. This dimensional organization allows serial processing at the hierarchical level (simplifying dependency management) while enabling parallel processing within each hierarchical level (improving productivity), effectively resolving the contradiction through multi-level processing.
4Productivity
If parallel processing of independent sub-systems is implemented to increase productivity, then productivity is improved, but device complexity increases
Solution Approach 1:
The patent segments the neural network into independent sub-systems that can be processed in parallel, directly enabling productivity improvement through concurrent evaluation. The segmentation creates naturally parallelizable units while maintaining manageable complexity through clear boundary definitions and one-way dependency relationships between sub-systems.
Solution Approach 2:
The patent implements a dynamic processing architecture where the degree of parallelization can be adjusted based on available computational resources and dependency relationships. This dynamic approach allows the system to maximize productivity when resources permit while gracefully degrading to serial processing when complexity constraints arise, providing flexibility in managing the productivity-complexity trade-off.
Data Source
AI summary
An artificial neural network may be configured to test the impact of certain input parameters. To improve testing efficiency and to avoid test runs that may not alter system performance, the effect of input parameters on neurons or groups of neurons may be determined to classify the neurons into groups based on the impact of certain parameters on those groups. Groups may be ordered serially and/or in parallel based on the interconnected nature of the groups and whether the output of neurons in one group may affect the operation of another. Parameters not affecting group performance may be pruned as inputs to that particular group prior to running system tests, thereby conserving processing resources during testing.