Data storage optimization method and system based on cuckoo hash calculation

Abstract

The invention discloses a Cuckoo hash calculation-based data storage optimization method and system. The method comprises the following steps of: considering each barrel in an index table as a subgraph node, considering each element stored in the table as an edge, and pointing from practical storage positions to candidate positions of the elements so as to form the whole index table into a diagraph comprising a plurality of connected subgraphs; recognizing one or two subgraphs of each element through hash calculation before the element is practically inserted; predicting an insertion result according to states of the subgraphs; and finally executing an insertion operation according to the prediction result or directly storing the prediction result into a temporary space. According to the method and system, mass data is flat hashed to the whole index table by utilizing a Cuckoo hash mechanism, so that the hash conflict of data in sets is reasonably solved, the load is balanced under the condition of ensuring the query efficiency, the utilization rate of the index table is effectively improved and the data insertion result is predicted in advance; and through predicting the result before data storage, the ineffective kicking overhead is avoided and the data storage efficiency is improved.

Description

technical field [0001] The invention belongs to the technical field of computer storage, and more specifically relates to a data storage optimization method and system based on Cuckoo hash calculation. Background technique [0002] With the rapid development of the Internet, cloud computing, Internet of Things, social media, and other information technologies in recent years, the data accumulated in various industries has shown an explosive growth trend. For example, Facebook processes more than 500TB of data every day, Alibaba has more than 100PB of data, Sina Weibo has more than 500 million users, and more than 100 million Weibos are generated every day. Many commercial companies usually have to process TB-level or even Petabytes of data. According to the 2014 report of International Data Corporation (IDC), the total amount of data generated globally doubles every two years and will reach 44ZB in 2020. [0003] The advent of the big data era has brought new challenges an...

AI Technical Summary

Problems solved by technology

[0007] For the above defects or improvement needs of the prior art, the object of the present invention is to provide a data storage optimization method and system based on Cuckoo hash calculation, thereby solving the problem of constructing an index table based on Cuckoo hash in the existing cloud storage system The technical problem of kicking out the infinite loop of the path caused by element insertion

Images