Embedded AI Runtime Preloading for Faster Application Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI applications in embedded computing systems face inefficiencies in resource utilization, leading to increased response times, sluggish performance, and excessive battery consumption due to redundant resource loading and unloading during transitions between applications.
Innovation Solution
A Neural Lookahead Framework (NLF) optimizes resource utilization by predicting resource requirements using runtime profiles and configuring the runtime environment accordingly, managing resource allocation and de-allocation efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If AI applications are executed on embedded computing systems with extensive computations, then application accuracy and personalized user experience are improved, but resource consumption (memory, battery power, computing power) increases
Solution Approach 1:
The system performs preliminary actions by detecting application launch events and pre-configuring the runtime environment with required resources before the AI application actually executes. This includes pre-loading model files, pre-waking computing units, and pre-allocating memory resources based on the runtime profile, thereby avoiding resource contention and reducing actual execution time and energy consumption during runtime
2Adaptability or versatility
If multiple neural network models are loaded into shared memory space for AI applications, then application functionality is improved, but application launch time and switching response time increase
Solution Approach 1:
The system detects application launch events and proactively pre-loads required neural network models into shared memory space before the application actually needs them. By predicting resource requirements from runtime profiles and configuring the runtime environment in advance, the system eliminates the time penalty associated with loading models during application launch, thereby reducing launch time while maintaining full functionality
3Adaptability or versatility
If hardware and software resources are frequently loaded and unloaded during switching between AI applications, then application switching capability is improved, but battery consumption increases and memory usage becomes inefficient
Solution Approach 1:
The system performs preliminary configuration of the runtime environment by detecting application launch events and pre-allocating resources based on runtime profiles. This includes pre-waking computing units and pre-allocating memory resources, thereby avoiding the energy-consuming cycles of frequent resource loading and unloading during application switching, while maintaining the ability to switch between applications
Solution Approach 2:
The system maintains continuous availability of AI resources by keeping computing units awake and models loaded in memory during the application lifecycle. By configuring the runtime environment in advance and maintaining resource availability, the system eliminates the stop-start cycles of resource allocation and de-allocation, reducing energy consumption from frequent loading/unloading operations while preserving application switching capability
Data Source
AI summary
Disclosed herein is a method and an optimization unit for optimizing and/or improving efficiency of resource utilization in an embedded computing system executing Artificial Intelligence (AI) applications. The method includes: detecting, by an optimization unit comprising processing circuitry and/or executable program instructions configured in the embedded computing system, a launch of an AI application on the embedded computing system; retrieving a runtime profile corresponding to the AI application, the runtime profile indicating resource requirements for executing the AI application; and configuring a runtime environment of the embedded computing system for the AI application based on the runtime profile corresponding to the AI application.