Fabric-Attached Memory for Zero-Copy Distributed Dataset Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory systems face challenges in efficiently processing large-scale datasets for machine learning workflows due to data duplication and replication across multiple nodes, leading to resource inefficiencies and performance degradation.
Innovation Solution
A distributed workflow system utilizing fabric-attached memory (FAM) enables direct access and zero-copy analysis, allowing selective data extraction and computation without copying data to local memory, leveraging CXL compliance and Apache Arrow format for high-speed interconnection and efficient data handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is replicated across multiple nodes in a distributed system, then data availability and access speed are improved, but memory resource consumption and processing overhead increase
Solution Approach 1:
The patent merges multiple copies of the same dataset stored across different nodes into a single shared memory space accessible by all nodes. This eliminates redundant data storage while maintaining fast access speeds through direct memory addressing, resolving the contradiction between data availability and memory resource consumption.
Solution Approach 2:
The shared memory system provides universal access to all distributed nodes, allowing any node to access any data without replication. This multi-functional memory architecture serves both as storage and as a shared resource for computation, improving data availability without proportionally increasing total memory requirements.
2Productivity
If datasets are copied to local memory for processing, then computation speed is improved, but memory bandwidth consumption and processing overhead increase
Solution Approach 1:
The patent extracts only the specific portions of data needed for computation from the shared memory, rather than copying entire datasets to local memory. This selective data extraction minimizes memory bandwidth consumption while maintaining computation speed by providing direct access to required data elements.
Solution Approach 2:
The shared memory acts as an intermediary between storage and local processing memory, allowing nodes to access data efficiently without full duplication. This intermediary layer reduces the burden on memory bandwidth by enabling direct access patterns that avoid unnecessary data movement.
3Adaptability or versatility
If data is stored in traditional formats for distributed processing, then compatibility with existing systems is maintained, but processing efficiency and scalability are reduced
Solution Approach 1:
The patent changes the organizational parameters of data storage by implementing columnar storage formats and structured data layouts optimized for vectorized operations. This reorganization of data parameters enables efficient parallel processing and scalability while maintaining compatibility through standardized interfaces and protocols.
Data Source
AI summary
In some implementations, a memory system may store a dataset in a portion of a fabric-attached memory, wherein the dataset is stored in a format that enables zero-copy analysis of the dataset by multiple host devices associated with a distributed workflow. The memory system may establish a respective direct access connection to the portion of the fabric-attached memory with each host device of the multiple host devices associated with the distributed workflow. The memory system may permit each host device, of the multiple host devices, to access the dataset via the respective direct access connection and by using a zero-copy access technique to extract a batch of data objects from the dataset for performing a computation associated with the distributed workflow.