AI Weight Data Channel Alignment for Faster Accelerator Loading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for uploading weight data to computing devices in artificial intelligence systems are inefficient, as they do not account for the specific channel structure between memory devices and accelerators, leading to suboptimal data transmission speeds.
Innovation Solution
A host system that generates lightened weight data and aligns it into multiple files based on configuration information about the computing device's channels, allowing parallel access and loading into separate memory spaces, thereby improving data transmission efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If weight data is uploaded to computing device using existing methods, then data transmission is completed, but data transmission speed is suboptimal due to not accounting for channel structure
Solution Approach 1:
The patent segments the weight data into multiple files corresponding to different memory channels. Each file is loaded independently through its designated channel, enabling parallel data transmission and improving overall data loading speed while accounting for the multi-channel architecture of the computing device.
Solution Approach 2:
The host performs preliminary actions by pre-processing the weight data into channel-specific files before transmission. This includes organizing data according to channel configurations and preparing appropriate data formats in advance, which streamlines the subsequent parallel loading process and optimizes transmission efficiency.
2Productivity
If weight data is loaded into single memory space, then loading process is simple, but data access efficiency is reduced due to sequential access requirements
Solution Approach 1:
The patent divides the weight data into separate files that are loaded into different memory spaces or memory regions, each accessible through dedicated memory channels. This segmentation enables simultaneous parallel access to multiple data segments by different processing units, significantly improving data access efficiency for AI computations.
Solution Approach 2:
The patent utilizes the multi-dimensional memory channel architecture by mapping different data files to different channel dimensions. This allows the system to exploit spatial parallelism across multiple memory channels, transforming a potentially sequential single-channel access pattern into a parallel multi-dimensional access pattern.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Disclosed is an operation method of a host which controls a computing device performing an artificial intelligence computation. The operation method includes receiving configuration information from the computation device, generating a plurality of lightening weight data by performing lightening on weight data based on the configuration information, generating a plurality of files by performing an aligning operation on the plurality of lightening weight data based on the configuration information, and loading the plurality of files into a memory device of the computing device. The configuration information includes channel information about a plurality of channels between the memory device and an accelerator, which are included in the computing device, and the number of the plurality of files is equal to the number of the plurality of channels.