Hierarchical Machine Learning Architecture for Edge Inference Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional machine learning model training requires a large set of labeled data and extensive time, often resulting in inaccurate models due to human error and limited data exposure for edge machine learning engines, which hampers efficiency and accuracy.
Innovation Solution
A hierarchical machine learning architecture comprising a master machine learning engine supported by real-time lightweight edge engines that interact with human experts to train models using minimal examples, aggregating features, algorithms, and parameters to optimize the model and improve performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional machine learning training is used with large labeled datasets, then model accuracy can be improved, but training time and computational resources increase significantly
Solution Approach 1:
The patent divides the machine learning system into two segments: a master ML engine that performs comprehensive training with large datasets, and edge ML engines that perform lightweight real-time inference. This segmentation allows the system to achieve high accuracy through the master engine while maintaining fast response times at the edge, resolving the contradiction between accuracy and training time.
Solution Approach 2:
The master ML engine performs preliminary training actions by pre-training models with extensive labeled datasets before deploying them to edge engines. This preliminary action enables the edge engines to make accurate predictions without requiring extensive real-time training, thus achieving high accuracy without sacrificing inference speed.
2Reliability
If extensive labeled training data is used, then model robustness improves, but data labeling errors and human effort increase
Solution Approach 1:
The system employs self-service mechanisms where the master ML engine automatically generates synthetic labeled data and performs self-supervised learning. This reduces reliance on manual human labeling, minimizing labeling errors and efforts while still achieving robust models through automated data generation and validation processes.
Solution Approach 2:
The master ML engine acts as an intermediary between raw data and edge inference engines. It processes and validates training data, generating high-quality labeled datasets that reduce errors propagating to edge engines. This intermediary role ensures data quality without requiring extensive manual labeling at the edge.
3Measurement precision
If full-featured ML models are deployed at edge, then accuracy improves, but computing resource consumption increases
Solution Approach 1:
The patent applies local quality by deploying different model versions to different locations: comprehensive models reside at the master engine with full computational resources, while lightweight distilled models are deployed to edge engines with limited resources. This ensures each location has the appropriate model complexity for its capabilities, maintaining accuracy where possible while conserving edge resources.
Solution Approach 2:
The system creates simplified copies of the master model for deployment at edge engines. These copied models are distilled versions that retain essential predictive capabilities while consuming fewer resources. The copying process enables edge devices to perform inference with acceptable accuracy without requiring the full computational power of the original comprehensive model.
4Adaptability or versatility
If real-time training is performed at edge, then adaptability improves, but model quality suffers due to limited data exposure
Solution Approach 1:
The system implements feedback loops where edge ML engines send inference results and performance metrics back to the master ML engine. The master engine uses this feedback to retrain and update models, which are then propagated back to edge engines. This feedback mechanism enables continuous adaptation at the edge while maintaining model quality through centralized retraining with comprehensive data.
Solution Approach 2:
The system adopts a dynamic architecture where the division of labor between master and edge engines can change over time. Edge engines can dynamically request model updates from the master engine based on performance degradation or new data patterns. This dynamic interaction allows the system to adapt to changing conditions while maintaining model quality through coordinated updates.
Data Source
AI summary
A system and method relate to a processing device implementing a master artificial intelligence (AI) engine to receive, from each of one or more real-time AI engines, a machine learning algorithm, parameters associated with the machine learning algorithm, and features employed to train the parameters, receive labeled data used to train the parameters associated with the machine learning algorithm, and construct, based on a combination rule, a master machine learning model using the features, the machine learning algorithm, and the parameters associated with the machine learning algorithm received from each of the one or more real-time AI engines.