AI Model Node Partitioning for Memory Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial intelligence learning models with a large number of nodes exceed the memory capacity of electronic devices, making normal compilation or interpretation difficult due to hardware limitations.
Innovation Solution
An electronic device separates nodes into multiple partitions instead of a single partition, allowing for normal compilation or interpretation of artificial intelligence learning models by identifying supported operation functions on a target processor and forming groups based on error occurrence, thereby managing memory constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If artificial intelligence learning models with a large number of nodes are compiled or interpreted on electronic devices, then the model complexity and node count increase, but the memory capacity of electronic devices becomes insufficient
Solution Approach 1:
The patent divides the artificial intelligence learning model into multiple partitions based on node indices. Each partition contains a subset of nodes from the original model, allowing the model to be processed in smaller chunks that fit within device memory. The system identifies partition boundaries by detecting where memory constraints occur during compilation or interpretation, then creates separate partition files that can be loaded and executed independently.
2Quantity of substance
If nodes are separated into multiple partitions, then memory constraints are managed effectively, but the system complexity increases due to partition management
Solution Approach 1:
The system automatically identifies partition boundaries by monitoring memory usage during the compilation or interpretation process. It detects where memory constraints occur and autonomously creates partition divisions without requiring manual intervention. The system also automatically manages loading and unloading of partition files based on execution needs, reducing the burden on users while effectively managing memory constraints.
3Ease of manufacture
If partitioning is implemented to fit memory constraints, then compilation or interpretation becomes feasible, but processing time increases due to multiple partitions
Solution Approach 1:
The system performs partition identification and division during the initial compilation or interpretation phase, preparing partition files in advance. By pre-processing the model into manageable partitions and storing them as separate files, the system enables faster subsequent executions. The partition structure is established beforehand, allowing efficient loading and execution without repeated processing overhead.
Data Source
AI summary
An electronic device is provided. The electronic device includes memory storing artificial intelligence models and one or more programs including instructions, and one or more processor, wherein the one or more programs including instructions, when executed by the one or more processors, cause the electronic device to load the artificial intelligence models stored in memory and execute a runtime engine of a framework, identify whether an operation function is supported on a target processor, identify whether a first node for executing an inference on the artificial intelligence models operate without errors based on supporting the operation function on the target processor, repeat the identification until a last node by adding one more nodes in case that the first node operates without errors, form a first group by creating a partition from the first node to an identified N−1st node based on the identification that an error occurred on an Nth node, and form a second group by creating a partition for the Nth node on which the error occurred.