Selective GPU Function Loading API
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Running applications on a GPU involves loading large numbers of functions, leading to full loading of all functions into memory even if only a few are used, resulting in performance slowdowns and resource inefficiencies due to unnecessary memory usage.
Innovation Solution
Implementing an API to selectively load libraries and GPU functions by performing partial loading at application startup and full loading only when specific functions are used, based on state information generated during partial loading.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If all functions are fully loaded into GPU memory at application startup, then the application can access any function immediately, but memory usage increases and performance slows down due to unnecessary loading of unused functions
Solution Approach 1:
The patent segments the library functions into multiple groups and divides the loading process into phases. Only the first group of functions is fully loaded into GPU memory at startup, while subsequent groups are loaded on-demand when needed. This segmentation allows the system to maintain fast access to frequently used functions while avoiding the memory overhead of loading all functions simultaneously.
Solution Approach 2:
The patent performs preliminary partial loading of the first group of functions at application startup, preparing the most commonly accessed functions in advance. This preliminary action ensures that critical functions are ready for immediate execution while deferring the loading of less frequently used functions to later times when memory resources are better utilized.
2Reliability
If all functions are fully loaded into GPU memory, then no function calls will fail due to missing code, but other resources are tied up due to unnecessary memory footprint
Solution Approach 1:
The patent implements a dynamic loading strategy where the set of loaded functions changes over time based on actual usage patterns. The system monitors which functions are called during execution and dynamically loads additional function groups as needed. This dynamic approach maintains reliability by ensuring that functions are loaded before they are needed, while optimizing memory footprint by unloading or not loading functions that are never used.
Solution Approach 2:
The system employs a self-service mechanism where the application automatically requests loading of additional function groups when it encounters functions that are not currently loaded. The runtime system monitors function calls and triggers loading of the next group when a function from a subsequent group is invoked, eliminating the need for manual configuration while maintaining both reliability and memory efficiency.
3Quantity of substance
If partial loading is performed at application startup, then memory usage is reduced, but additional API calls are needed to perform full loading when functions are used
Solution Approach 1:
The patent implements a self-service loading mechanism where the system automatically manages the loading of function groups without requiring explicit user intervention. When an application encounters a function that is not currently loaded, the runtime system automatically triggers the loading of the appropriate function group. This self-service approach reduces the perceived complexity for users while maintaining efficient memory management.
Solution Approach 2:
The system uses feedback from function call monitoring to control the loading process. The runtime system observes which functions are invoked during execution and uses this feedback information to determine when to load additional function groups. This feedback-driven approach simplifies the user interface while ensuring that memory resources are allocated based on actual usage patterns rather than static predictions.
Data Source
AI summary
Apparatuses, systems, and techniques to selectively load data required to use one or more functions. In at least one embodiment, selective load for one or more functions to be used is performed by one or more application programming interface for efficient use of memory on a system comprising a processor and a graphics processor.