Block level cache prefetching optimization method and system based on deep learning

Abstract

The invention discloses a block level cache prefetching optimization method based on deep learning. The method comprises the steps of obtaining IO data with bytes as a unit from a test data set, converting the IO data into IO data with a block as a unit; judging whether the converted IO data is hit in a cache or not; if not, performing sequential prediction on the converted IO data to obtain a plurality of storage blocks, storing the converted IO data in an IO queue of a memory, judging whether the IO queue is full or not, if yes, inputting all the IO data in the IO queue of the memory into the trained Seq2Seq model to obtain predicted IO data, and obtaining a plurality of corresponding storage blocks according to the predicted IO data. According to the method, correlation of IO is mined through a deep learning method, prediction of an IO sequence is completed through the Seq2Seq model based on LSTM, and finally IO sequence prediction and sequential prediction are combined, so that cache prefetching is completed, and the hit rate of a cache is increased.

Description

technical field [0001] The invention belongs to the technical field of computer storage, and more particularly relates to a block-level cache prefetch optimization method and system based on deep learning. Background technique [0002] In the era of big data, the performance of the storage system is becoming more and more important. In order to further improve the performance of the storage system, it is necessary to prefetch the block-level access of the local disk. [0003] The current prefetching methods for block-level access mainly include fixed prefetching algorithms, sequential prefetching algorithms, prefetching algorithms based on application hints, and prefetching algorithms based on data mining. Among them, the implementation of the fixed prefetch algorithm is relatively simple, only need to prefetch n data behind the currently accessed data; the sequential prefetch algorithm is an improvement on the fixed prefetch algorithm, which can automatically adjust the pre...

AI Technical Summary

Problems solved by technology

[0004] However, the above prefetching methods for block-level access have some defects that cannot be ignored: both the fixed prefetching algorithm and the sequential prefetching algorithm only use the locality principle of data access, and the prefetching efficiency and prefetching accuracy are relatively low ; The prefetch algorithm based on application hint needs to provide an interface for conveying prefetch instructions to the upper application, which cannot be supported in many systems; the prefetch algorithm based on data mining has limited ability to mine data access characteristics, In different access patterns, the performance of prefetch accuracy varies greatly

Images