Neural Network Optimization via Parallel Workflow Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The optimization of neural networks is a serial process that depends on the interdependence of machine-learning experts and engineers, leading to prolonged training times and idle resources, as the process requires iterative design and tuning stages that are mutually dependent.
Innovation Solution
The optimization process is bifurcated into independent workflows, allowing machine-learning engineers to evaluate hardware constraints and experts to focus on accuracy, enabling parallel processing and dynamic architecture evolution based on evaluation results, with multiple model variants being tested and optimized simultaneously for latency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single neural network model is optimized sequentially for accuracy first and then for hardware constraints, then the optimization process ensures thorough evaluation of each aspect, but the total optimization time increases significantly and resources remain idle during waiting periods
Solution Approach 1:
The patent divides the neural network optimization process into two independent workflows: an accuracy evaluation workflow and a hardware constraint evaluation workflow. These workflows operate in parallel rather than sequentially, allowing accuracy assessment and hardware constraint assessment to proceed simultaneously, thereby reducing total optimization time while maintaining thorough evaluation of both aspects
Solution Approach 2:
The patent performs preliminary hardware constraint evaluation on multiple model variants before full accuracy training is completed. By evaluating hardware constraints on untrained or partially trained models, the system can identify which models are likely to meet hardware requirements early in the process, avoiding wasted training time on models that would fail hardware constraints anyway
2Productivity
If multiple model variants are evaluated and trained simultaneously, then the system can identify the best model more efficiently, but the computational resources and complexity increase
Solution Approach 1:
The patent segments the evaluation process into independent workflows that can be distributed across multiple hardware devices. Each device can evaluate different model variants independently, and the results are aggregated to determine the best model. This segmentation allows parallel processing without requiring complex inter-device coordination, managing system complexity while improving productivity
Solution Approach 2:
The patent introduces a workflow management system that acts as an intermediary between multiple evaluation processes and the final model selection. This intermediary coordinates the parallel evaluation workflows, manages resource allocation, and aggregates results, thereby enabling efficient multi-model evaluation without proportionally increasing system complexity
Data Source
AI summary
Systems, tools and methods are provided for optimizing neural networks (NNs) to run efficiently on target hardware such as central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), etc. The provided software tools are implemented as part of a machine-learning operations (MLOps) workflow for building a neural network, and include optimization algorithms (e.g., for quantization and/or pruning) and compiler processes that reduce memory requirements and processing latency.