Memory Management for Dedicated Processing Resources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems face challenges in managing memory resources for dedicated processing resources like GPUs, leading to out-of-memory issues during large-scale deep learning model training, which can result in model training failures and require costly hardware upgrades.
Innovation Solution
A method for dynamically managing memory across multiple dedicated processing resources by determining available memory spaces and transferring data to resources with sufficient capacity, optimizing data transmission rates, and using topology-aware allocation to balance storage loads and reduce memory constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored only on the memory of a single dedicated processing resource, then the memory access speed is fast, but the available memory space is limited
Solution Approach 1:
The patent combines memory resources from multiple dedicated processing resources into a unified storage system. When the first dedicated processing resource lacks sufficient memory space, data is automatically transferred to and stored on the memory of a second dedicated processing resource, effectively merging the memory capacities of both resources to solve the limitation of individual memory space.
Solution Approach 2:
The patent introduces a memory management mechanism that acts as an intermediary between multiple dedicated processing resources. This intermediary monitors memory availability across resources and coordinates data transfer between them, enabling seamless utilization of distributed memory space while maintaining the performance benefits of local memory access.
2Quantity of substance
If memory resources are shared across multiple dedicated processing resources, then the total storage capacity increases, but the system complexity increases
Solution Approach 1:
The patent implements a self-service memory management system where the dedicated processing resources automatically monitor their own memory status and initiate data transfers to other resources when needed. The system autonomously determines when and where to store data without requiring complex external management intervention, thereby reducing overall system complexity.
Solution Approach 2:
The patent employs feedback mechanisms where the memory management system continuously monitors memory usage levels across dedicated processing resources and automatically triggers data transfer operations when threshold values are exceeded. This feedback-driven approach simplifies management by using straightforward threshold-based decisions rather than complex allocation algorithms.
3Quantity of substance
If data is transferred between dedicated processing resources, then the memory space utilization improves, but the data transmission time increases
Solution Approach 1:
The patent performs preliminary actions by pre-transferring data to secondary memory resources before the primary resource actually needs it. The system monitors memory usage trends and proactively moves data to available space in advance, reducing the actual transmission time when data is needed and improving overall system responsiveness.
Solution Approach 2:
The patent applies partial action by transferring only the specific portions of data that are needed rather than moving entire datasets. The system intelligently identifies and transfers only the minimal necessary data portions to the second dedicated processing resource, reducing transmission time while still achieving the goal of resolving memory space constraints.
Data Source
AI summary
A method comprises determining, in a process of storing data for a computing task of a first dedicated processing resource of a set of dedicated processing resources to the first dedicated processing resource, a size of an available space of a memory of the first dedicated processing resource; in response to the size of the available space of the memory of the first dedicated processing resource being lower than a predetermined threshold value, determining a second dedicated processing resource of the set of dedicated processing resources, a size of an available space of a memory of the second dedicated processing resource is greater than the predetermined threshold value; and causing at least one portion of the data not stored on the memory of the first dedicated processing resource to be stored on the memory of the second dedicated processing resource.