Distributed System Memory Management via Unified Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GPUaaS-based distributed systems face performance issues due to the saturation of network transmission capabilities and memory capacity, particularly with the increasing data processing demands of deep learning and machine learning tasks, where the limited video memory of GPUs leads to insufficient memory or overflow problems.
Innovation Solution
A method for managing distributed systems that employs a unified memory access (UMA) scheme with fine-grain memory mapping between client and server memory regions, allowing for efficient data synchronization and processing by mirroring memory regions and transmitting only changed data blocks, thereby reducing network overhead and improving memory management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transmitted between client and server for processing, then computing tasks can be performed using remote GPU resources, but network transmission capability becomes saturated and memory capacity is insufficient
Solution Approach 1:
The patent segments memory into multiple memory regions (first memory region on client, second memory region on server) and divides data into data blocks that can be independently managed and transmitted. This segmentation allows selective synchronization of only changed data blocks rather than transmitting entire datasets, effectively addressing memory capacity limitations while maintaining computing productivity.
Solution Approach 2:
The patent introduces a new dimension of memory management by establishing a mapping relationship between memory regions across different spatial locations (client and server). This dimensional extension allows the system to access and manage memory resources beyond local boundaries, effectively increasing available memory capacity without being constrained by local memory limits.
2Reliability
If all data is synchronized between client and server, then data consistency is maintained, but network overhead increases and transmission efficiency decreases
Solution Approach 1:
The patent extracts and transmits only the changed data blocks from the memory regions rather than synchronizing all data. By identifying and separating only the modified portions (data blocks that have changed), the system maintains data consistency where needed while dramatically reducing network transmission overhead and energy consumption.
Solution Approach 2:
The patent applies different synchronization strategies to different data blocks based on their change status. Changed data blocks are transmitted and synchronized, while unchanged blocks remain local. This localized quality approach ensures data consistency for modified regions while avoiding unnecessary network operations for unchanged regions.
3Productivity
If fine-grain memory mapping is implemented, then memory management efficiency is improved and data transmission is reduced, but system complexity increases
Solution Approach 1:
The patent establishes memory region mappings and data block tracking mechanisms in advance before data processing operations begin. This preliminary setup creates a structured framework that automatically identifies changed data blocks during processing, reducing the need for complex real-time analysis and simplifying the overall system operation despite the initial configuration complexity.
Data Source
AI summary
A method of managing a distributed system comprises: in response to receiving data that changes in a first memory region of a client, updating data in a second memory region with the received data based on a mapping between the first memory region and the second memory region; in response to receiving a calling request for a computing node in a server from the client, causing the computing node to perform a processing step based on the updated data in the second memory region; determining data that changes after the processing step in the second memory region; and transmitting, to the client, the data that changes after the processing step, such that the client updates the data in the first memory region based on the mapping.