Storage Controller Memory Switching for On-Device LLMs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The limited availability of high-performance memory resources in personal computer and mobile environments hinders the effective operation of on-device artificial intelligence services, particularly for large language models.
Innovation Solution
A storage device with a controller that manages a first and second auxiliary memory, allowing data loading and caching to optimize memory usage, providing access rights to host devices, and switching between operation modes to enhance memory availability for AI services.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is loaded into first auxiliary memory for AI operations, then memory access speed is improved, but memory resource availability for other operations deteriorates
Solution Approach 1:
The system dynamically switches between first and second operation modes based on host device needs. In first mode, data is loaded into first auxiliary memory for fast AI operations. In second mode, the same memory area is made available to the host device. This dynamic reconfiguration resolves the contradiction by allowing the memory resource to serve different purposes at different times rather than being statically allocated.
Solution Approach 2:
The memory allocation follows a periodic pattern where the system alternates between loading data for AI operations and providing memory access to the host device. This periodic switching ensures that high-speed memory access is available when needed for AI computations, while also ensuring memory resources are periodically made available for other operations, balancing both requirements.
2Quantity of substance
If storage device provides full memory access to host device, then memory availability is improved, but operational control and caching capability deteriorates
Solution Approach 1:
The memory management is segmented into different operational modes. The controller divides its functionality to either provide full memory access to the host device or to perform caching and data management operations. This segmentation allows the system to optimize for either memory availability or operational control depending on the current mode, resolving the contradiction by making the optimization goal explicit and switchable rather than attempting to achieve both simultaneously.
3Productivity
If data is cached in second auxiliary memory, then access efficiency is improved, but memory capacity for loading new data deteriorates
Solution Approach 1:
The second auxiliary memory is designed to serve multiple functions: it acts as a cache for frequently accessed data during AI operations, but can also be used to load new data when needed. This multi-functionality allows the same memory capacity to serve both caching and loading purposes at different times, resolving the contradiction between maintaining cache efficiency and having capacity for new data.
Data Source
AI summary
A storage device may process a command by loading mapping data into a first memory area of a first auxiliary memory included in the storage device in a first operation mode, and may process a command by providing the first memory area as a control buffer to a host device and loading mapping cache data corresponding to a part of the mapping data into a second auxiliary memory in a second operation mode, thereby providing a system capable of efficiently utilizing system resources and implementing artificial intelligence services while operating a large language model.