Construction and query method of scalable storage index structure in cloud environment
A technology for expanding storage and indexing structures, applied in database indexing, structured data retrieval, digital data information retrieval, etc., can solve problems such as high network overhead, complex P2P network maintenance, affecting query performance of cloud storage systems, etc., to improve efficiency , the effect of good scalability
Inactive Publication Date: 2019-05-03
YUNNAN UNIV
View PDF4 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
However, most of these solutions use an overlay network based on the P2P protocol to achieve parallel query in the global index, but the maintenance of the P2P network itself is relatively complicated, and the network overhead during query is also relatively large, which will affect the query performance of the cloud storage system
At the same time, since the existing cloud storage systems are generally in the master-slave structure, rebuilding a P2P network on these nodes will have a certain negative impact on the original storage system
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment
[0030] figure 1 It is a schematic diagram of the architecture of the scalable storage index in the present invention. Such as figure 1 As shown, the scalable storage index in the present invention adopts a KD tree structure, and each node corresponds to a range information, which is used to divide the data range corresponding to the left and right child nodes of the node. The depth D of the tree structure can be customized to control the size of the data set corresponding to each leaf node. For example, when the size of the complete data set is 100 million records, if the depth of the KD tree is set to 11, then the KD tree has 1024 leaf nodes, and each leaf node corresponds to about 10,000 records; if the depth of the KD tree is set to If the depth is 21, then the KD tree has 1,048,576 leaf nodes, and each leaf node corresponds to about 100 records. The internal nodes in the KD tree only play the role of routing, so they can be called router-nodes. A leaf node corresponds ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a construction and query method of an extensible storage index structure in the cloud environment. The method comprises the steps that firstly a KD tree index structure is created, when a KD tree is created, data of each index dimension is sequentially used as a classification standard of a layer node, the index data of each leaf node dataset in the created KD tree is stored in an HBase, and a Bloom Filter structure is created for the whole dataset and stored; in a single key value query, firstly it is detected that weather the data exits or not through the Bloom Filter structure, and then a precise query is conducted according to the KD tree; in a range query, a subtree corresponding to the query range is determined, and then a precise query is conducted according to the leaf nodes under the subtree. By means of the KD tree data structure and the HBase, the extensible storage index structure is created in the cloud environment in a targeted mode, data subsets of all dimensionalities in a certain range are mapped together by means of the KD tree, and the query is achieved in a multi-dimensional range.
Description
technical field [0001] The invention belongs to the technical field of cloud storage, and more specifically relates to a method for constructing and querying an expandable storage index structure in a cloud environment. Background technique [0002] With the development of computer and network technology, cloud computing technology, as a high-performance, low-cost practical distributed computing technology, has been widely used in various network applications represented by big data processing. Cloud storage systems with high scalability and reliability have gradually become one of the preferred solutions for big data processing. Existing excellent cloud storage systems include: Google's GFS, MapReduce and its open source implementation Hadoop, Amazon's Dynamo, and Facebook's Cassandra, etc. . Compared with traditional data storage systems, cloud storage systems are more widely distributed and support more data, which means that the auxiliary index system in the era of clou...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/22G06F16/245
CPCG06F16/2246G06F16/2264G06F16/245
Inventor 周维刘建坤罗静姚绍文张浩
Owner YUNNAN UNIV