Host-Controlled AI Weight Loading Across Memory Channels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for uploading weight data to computing devices in artificial intelligence systems are inefficient, as they do not account for the specific channel structure between memory devices and accelerators, leading to suboptimal data transmission speeds.
Innovation Solution
A host system that generates lightening weight data by applying techniques like pruning, quantization, and compression, and aligns this data into multiple files based on configuration information about the computing device's channels, allowing parallel access and loading into multiple memory devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If weight data is uploaded using existing methods without considering channel structure, then the process is simple, but data transmission speed is slow
Solution Approach 1:
The patent segments weight data into multiple channels based on the channel configuration of the computing device. Each channel receives a portion of the weight data in parallel, increasing data transmission speed. The host divides the weight data into channel-specific data streams and transmits them simultaneously through multiple channels to the computing device.
Solution Approach 2:
The patent introduces a channel dimension to the data transmission process. Instead of single-channel sequential transmission, the system utilizes multiple channels as an additional dimension for parallel data flow. This multi-dimensional approach allows simultaneous data transmission across different channels, significantly improving transmission speed.
2Productivity
If weight data is processed and aligned into multiple files based on channel configuration, then data transmission speed improves, but processing time increases
Solution Approach 1:
The patent performs preliminary alignment of weight data into channel-specific files before transmission. The host prepares the data by organizing it according to the computing device's channel configuration in advance, so that when transmission begins, the data is already optimized for parallel channel processing. This preliminary organization enables efficient parallel transmission without requiring complex real-time processing during data transfer.
Data Source
AI summary
Disclosed is an operation method of a host which controls a computing device performing an artificial intelligence computation. The operation method includes receiving configuration information from the computation device, generating a plurality of lightening weight data by performing lightening on weight data based on the configuration information, generating a plurality of files by performing an aligning operation on the plurality of lightening weight data based on the configuration information, and loading the plurality of files into a memory device of the computing device. The configuration information includes channel information about a plurality of channels between the memory device and an accelerator, which are included in the computing device, and the number of the plurality of files is equal to the number of the plurality of channels.