AI Memory Architecture with Split Storage for Cost and Wear
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high demand for memory resources in AI computing, particularly for large-scale AI models, leads to increased costs and reduced lifespan of memory devices due to frequent read and write operations, necessitating effective resource sharing to improve performance and stability while extending the service life of computer hardware systems.
Innovation Solution
A computer hardware system utilizing a primary storage device for write-intensive data and cache, a secondary storage device for read-intensive data, and accelerator processors to execute layer computations, optimizing memory allocation based on the characteristics of different storage devices to enhance efficiency and durability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory resources are allocated to store parameter data and cache data for AI computing, then computing performance is improved, but device cost increases
Solution Approach 1:
The patent combines primary storage device (e.g., DDR4 memory) and secondary storage device (e.g., NAND flash) into a unified storage system that serves both parameter data storage and cache data storage functions. This merging allows the system to leverage the high-speed access of primary storage for parameter data while using the high-capacity secondary storage for cache data, achieving optimal performance without requiring separate dedicated storage devices for each function, thereby reducing overall device cost.
Solution Approach 2:
The patent makes the storage system multi-functional by enabling the same storage infrastructure to handle both parameter data (which requires fast access) and cache data (which requires large capacity). The storage manager dynamically allocates and manages these different types of data across the primary and secondary storage devices, allowing the storage system to serve multiple purposes simultaneously rather than requiring separate specialized storage devices for each function.
2Device complexity
If primary storage device is used to store both parameter data and cache data, then device complexity is reduced, but reliability deteriorates due to limited read/write cycles
Solution Approach 1:
The patent segments the storage system into two distinct parts: primary storage device for storing parameter data and secondary storage device for storing cache data. This segmentation allows each storage device to be optimized for its specific function - the primary storage device handles read/write operations for parameter data while the secondary storage device handles cache data with its own read/write cycle characteristics. By dividing the storage responsibilities, the system maintains reliability while managing complexity through structured organization.
Solution Approach 2:
The patent introduces a storage manager as an intermediary component that manages the allocation, access, and coordination between primary and secondary storage devices. This storage manager handles the complexity of interfacing with two different storage devices, translating high-level storage requests into appropriate device-specific operations. By placing this intermediary layer, the system can maintain simplicity at the application level while handling the complexity of multi-device coordination internally, and protect reliability by preventing direct conflict between the two storage devices.
3Productivity
If high-frequency read and write operations are performed on storage devices, then computing efficiency is improved, but service life is shortened
Solution Approach 1:
The patent applies local quality by assigning different storage devices to different data types based on their specific characteristics and requirements. The primary storage device (e.g., DDR4) is used for parameter data that requires high-speed read/write operations during AI model training and inference, while the secondary storage device (e.g., NAND flash) is used for cache data that benefits from large capacity and can tolerate its own read/write cycle limitations. This localized optimization ensures that each storage device operates within its performance envelope, maintaining computing efficiency for parameter data while preserving the service life of both devices by avoiding excessive wear on any single device.
Data Source
AI summary
A computer hardware system for executing an artificial intelligence (AI) computing process is provided. The computer hardware system includes a primary storage device, a primary processor, a secondary storage device, and at least one accelerator processor. The primary storage device is configured to store write-intensive data. The primary processor is connected to the primary storage device and is configured to execute a setup process in the artificial intelligence computing process. The secondary storage device is configured to store read-intensive-and-no-write-intensive data. The at least one accelerator processor is configured to load the read-intensive-and-no-write-intensive data stored in the secondary storage device and access the write-intensive data in the primary storage device through the primary processor.