Multi-Channel LLM Storage Control for Type-Based Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory devices face challenges in storing high-capacity data for large language models due to increased costs and inefficiencies in data access, particularly when reading large amounts of specific-type data, which affects the performance of artificial intelligence model inference operations.
Innovation Solution
A storage device with a storage controller that manages a channel table to distribute data across multiple channels based on data types, optimizing data storage and access efficiency by using non-volatile memory devices connected through multiple channels, allowing for efficient distribution and retrieval of data for large language model inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored in a single channel or without channel distribution, then device complexity is reduced, but data access efficiency deteriorates when reading large amounts of specific-type data
Solution Approach 1:
The patent divides the storage system into multiple independent channels (first channel, second channel, etc.), each capable of handling different data types simultaneously. This segmentation allows parallel data access operations, improving overall data access efficiency while distributing the complexity across manageable channel units.
Solution Approach 2:
The channel table acts as an intermediary data structure that manages the mapping between data types and channels. It stores correspondence relationships between first data types and first channels, and second data types and second channels, enabling efficient routing decisions without complex direct management, thus resolving the contradiction between access efficiency and management complexity.
2Quantity of substance
If high-capacity buffer memory is implemented to store large language model data, then storage capacity is improved, but cost increases
Solution Approach 1:
The storage device implements multi-functionality by enabling a single storage system to efficiently handle multiple data types (first data type, second data type, etc.) through channel distribution. This allows the system to serve large language model inference requirements without requiring separate specialized storage systems for each data type, reducing overall implementation cost while maintaining high storage capacity.
Solution Approach 2:
The system changes the organizational parameter of data storage from a single-structure approach to a multi-channel distributed approach. By organizing data according to data type parameters and distributing them across channels based on the channel table, the system achieves high-capacity storage optimized for large language model operations without proportionally increasing costs.
3Loss of time
If data is distributed across multiple channels based on data types, then data retrieval efficiency is improved, but channel management complexity increases
Solution Approach 1:
The channel table is pre-configured with correspondence relationships between data types and channels before actual data operations occur. This preliminary organization of data type to channel mappings allows for rapid data retrieval without complex real-time decision-making, reducing data retrieval time while keeping management complexity manageable through advance planning.
Data Source
AI summary
Disclosed is an operation method of a storage controller which is connected to a plurality of non-volatile memory devices configured to store inference data of a large language model (LLM) through a plurality of channels. The method includes receiving a first request for writing a first key vector corresponding to a first data type, storing the first key vector in at least one first non-volatile memory device connected to one or more first channels among the plurality of channels, by referring to the first data type and a channel table, and updating the channel table such that a channel corresponding to the first data type is changed to a second channel among the plurality of channels.