Isolation Forest Node Traversal for Ultra-Low RAM Edge Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing ML solutions for edge devices face challenges with limited memory resources, requiring significant RAM for isolation forest models, which hinders deployment and maintenance due to increasing memory needs with model updates and concept drifts.
Innovation Solution
A method that minimizes working memory requirements by traversing decision trees one node at a time, using a unique tree traversal algorithm that extracts only the necessary nodes from the ML model, reducing memory usage to approximately 32 bytes, a fraction of the typical 100 MB-200 MB, and maintaining a constant memory footprint throughout the AI/ML model life cycle.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire isolation forest model is loaded into working memory for execution, then the model can be executed accurately, but the memory consumption increases to 100 MB-200 MB which exceeds the limited RAM budget at edge devices
Solution Approach 1:
The patent segments the isolation forest model into individual decision trees stored in non-volatile memory, and further segments each tree into nodes that are loaded into working memory one at a time during traversal. This segmentation allows the system to execute the model accurately while consuming only minimal working memory (approximately 32 bytes) instead of loading the entire model (100 MB-200 MB).
Solution Approach 2:
The patent extracts only the necessary portions of the model (individual nodes) from non-volatile memory and loads them into working memory as needed during tree traversal. This extraction approach ensures that the model executes accurately by accessing the complete tree structure from storage while maintaining minimal working memory footprint by not keeping the entire model in RAM simultaneously.
2Adaptability or versatility
If model updates are performed to counter concept drifts, then the model adapts to changing conditions, but the RAM requirements increase over the model's life cycle making deployment difficult
Solution Approach 1:
The patent implements a dynamic memory management approach where the working memory footprint remains constant at approximately 32 bytes regardless of model updates. The system dynamically loads only the necessary nodes from non-volatile memory during tree traversal, ensuring that model adaptability to concept drifts is maintained while RAM requirements do not increase with model updates.
Solution Approach 2:
The patent uses a persistent storage mechanism to store the complete isolation forest model, allowing model updates to be applied to the stored model without increasing working memory requirements. The system copies only the necessary model nodes into working memory during execution, enabling continuous model adaptation while maintaining a constant, minimal RAM footprint.
3Productivity
If a traditional ML solution is deployed at the edge, then the model can perform inference, but the limited compute and storage resources constrain the deployment and maintenance operations
Solution Approach 1:
The patent segments the model execution into a persistent storage phase (where the complete isolation forest is stored) and an execution phase (where only necessary nodes are loaded into working memory). This segmentation enables inference capability at the edge while significantly reducing the computational and memory resources required compared to traditional approaches that load the entire model into memory.
Solution Approach 2:
The patent replaces the traditional mechanical approach of loading the entire model into RAM with a storage-based approach where the model is persisted in non-volatile memory and nodes are extracted on-demand during tree traversal. This substitution of the memory access mechanism enables efficient inference at edge devices with limited resources by eliminating the need to keep the complete model in working memory.
Data Source
AI summary
A method and system for generating a prediction in a low resource device using a decision tree based machine learning model includes receiving input for a prediction request, selecting a first tree from the machine learning model, selecting and loading a first node from the first tree into working memory, accumulating a result from the first node, releasing the first node from working memory, and selecting and loading a second node from the first tree into working memory.