Scatter-Gather DMA Rule Tables for Low-Memory AI Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional SG DMA operations for AI services, particularly for Large Language Models (LLM), result in increased memory space complexity due to the proportional increase in dimension and number of input variables, leading to inefficiencies in data transfer.
Innovation Solution
A data processing method using a rule table to derive source and destination addresses and data sizes through matrix operations, reducing space complexity by adaptively considering data transfer structures and minimizing host interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional SG DMA operation is used to transfer data for AI service, then data transfer can be performed, but memory space complexity increases in proportion to the increase in dimension and number of input variables
Solution Approach 1:
The patent uses a rule table that stores pre-defined mapping relationships between input variables and memory addresses. Instead of dynamically calculating addresses for each input variable, the system copies the rule table structure and uses it to directly determine memory access patterns, reducing the computational complexity from O(N) to O(1) where N is the number of input variables.
Solution Approach 2:
The patent transforms the address calculation parameters from variable-dependent (based on input dimension and number of variables) to constant (based on pre-defined rule table entries). By changing the parameters from dynamic calculations to static table lookups, the memory space complexity is reduced while maintaining data transfer functionality.
2Productivity
If conventional SG DMA operation is used, then data transfer is performed, but work efficiency decreases due to increased memory space requirements
Solution Approach 1:
The patent copies the essential mapping information into a compact rule table structure that can be quickly referenced. This copying approach eliminates the need for complex runtime address calculations, thereby improving work efficiency while reducing memory space requirements compared to storing full address mappings for all input variables.
Solution Approach 2:
The patent performs preliminary action by pre-defining all address mapping relationships in the rule table before the actual data transfer operation. This preliminary setup allows the SG DMA operation to proceed with simple table lookups rather than complex calculations during execution, improving work efficiency and reducing memory space complexity.
Data Source
AI summary
A data processing method using a scatter gather direct memory access (SG DMA), the method comprising: obtaining information for a rule table for specific subtasks of a SG DMA from a host, deriving the rule table for the specific subtasks based on the information, deriving source addresses, destination addresses and data sizes for the specific subtasks based on the rule table, and performing a SG DMA operation to transfer data of the data sizes located at the source addresses of a first memory to data spaces of the data sizes located at the destination addresses of a second memory.