The invention discloses a
large model training
dynamic memory management method based on a multilayer hierarchical cache mechanism, and relates to the technical field of computer
system architecture and storage. The method is managed on the basis of a dynamic heterogeneous memory unloading
system, and comprises the following steps of: in an initial stage of a training task, constructing a
user state access path, establishing a TensorID-LBA mapping table, and performing hierarchical caching and threshold presetting; in the training task running stage, the access popularity of each
tensor is evaluated in real time; when occupation of the GPU
video memory cache layer exceeds a threshold value or cold data is detected, a dynamic migration strategy is triggered, and the low-frequency
tensor is unloaded to a CPU memory cache layer or an SSD storage layer; when the
tensor access popularity is improved, automatically migrating back to a high-performance layer through a prediction mechanism; in the process, tensor self-adaptive migration is executed by a user mode I / O
thread pool. According to the method, the high-performance I / O stack is constructed in the user mode, and the adaptive tensor unloading and hierarchical caching mechanism of the multi-layer heterogeneous memory is combined, so that the universal deployment of the
system is realized.