The present invention discloses a multi-level cache and file batch download method,
system and storage medium, which belongs to the field of computer storage technology. The method includes: S1: the
object storage gateway kernel loads the file
list, and assembles and deduplicates the file
list; S2: setting the file upload threshold, grouping
small files, and sharding large files; S3: using the number of small file grouping threads plus the number of large file sharding threads to calculate the number of threads that need to be opened by the
cloud storage gateway; batch opening multiple threads for file download; S4: after the
cloud storage service end obtains the file, it caches it according to the first heat threshold. It has stronger reliability and adopts the NDN+IP-based C /
S system structure. Even if the central
server fails, the customer business
system will not be affected at all when the edge
network connection is normal. It has stronger security and the files are uploaded using block-by-block deduplication and
encryption. Compared with traditional
object storage, it greatly reduces the risk of data leakage.