Cloud storage framework and construction method for secondary cache prefetching of small files
A secondary cache and construction method technology, applied in the field of cloud computing, can solve problems such as low cache hit rate, weakened time locality, and reduced metadata cache hit rate, so as to improve efficiency and save network overhead
Active Publication Date: 2019-09-03
HARBIN INST OF TECH AT WEIHAI +1
View PDF8 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
In the case of random access, its cache hit rate is low
And because there are often CDN servers outside the cloud storage system, which have a caching function, file data based on time locality is often read from the cache of the external CDN server, and the time locality is requested through the access of the storage node. is further weakened, and its metadata cache hit rate is further reduced
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0050] Exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The description of the exemplary embodiments is for the purpose of illustration only, and in no way limits the invention and its application or usage.
[0051] figure 1 A flowchart of a method for constructing a cloud storage framework for secondary cache prefetching of small files according to the present invention is shown. In step S1, all objects in the object storage and the access records of each object are traversed first, the correlation between different objects is calculated, and the objects with high correlation are combined into a data set. For example, using Probabilistic Latent Semantic Analysis (PLSA) in the topic model, whether the access time is close or whether the subject of the object is close is used as consideration factors to calculate the object relevance. A person skilled in the art may select an index representing objec...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
A cloud storage framework and construction method for secondary cache prefetching of small files are disclosed, wherein the cloud storage framework includes: an object storage, an associative computing module, a data merging module, a storage node cache module and an agent node cache module. The present invention firstly calculates the correlation between different objects, and combines the objects with high correlation into one data set, and achieves the purpose of data prefetching by caching these data sets. On the one hand, random disk access can be transformed into continuous disk access to improve the efficiency of the storage system; on the other hand, by transferring the object collection to the proxy node cache module for pre-reading at one time, network overhead can be saved; File merging also reduces the amount of metadata and reduces the metadata operation time when objects are accessed.
Description
technical field [0001] The invention relates to the technical field of cloud computing, in particular to a cloud storage framework and construction method for secondary cache prefetching of small files. Background technique [0002] The technical background related to the present invention will be described below, but these descriptions do not necessarily constitute the prior art of the present invention. [0003] Swift (Openstack Object Storage) is developed by Rackspace, the second largest data center in the world. Partial composition. Swift is built on relatively cheap standard hardware devices. Due to cost considerations, it does not use RAID (Redundant Array of Disks), and introduces consistent hashing and data disaster recovery technologies through the software level. Swift supports multi-tenancy, containers, and object read and write operations. In order to obtain higher availability and scalability, it sacrifices certain data consistency, which is suitable for sol...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F12/0862
Inventor 朱东杰张凯王琦李从生陈金昌杜海文赵奇隆王子健董爽爽王希光
Owner HARBIN INST OF TECH AT WEIHAI