Unlock instant, AI-driven research and patent intelligence for your innovation.

Indexing method and system for large-scale data

A large-scale data and indexing technology, applied in the database field, can solve problems such as slow query of multi-dimensional data

Active Publication Date: 2014-06-18
EAST CHINA NORMAL UNIV
View PDF2 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The present invention overcomes the existing defect of slow multi-dimensional data query in big data, and proposes an indexing method and system for large-scale data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Indexing method and system for large-scale data
  • Indexing method and system for large-scale data
  • Indexing method and system for large-scale data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] The present invention will be further described in detail in conjunction with the following specific embodiments and accompanying drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the content specifically mentioned below, are common knowledge and common knowledge in this field, and the present invention has no special limitation content.

[0046] Such as Figure 1 to Figure 9 , 1-original data storage unit, 2-leaf layer storage unit, 3-intermediate layer storage unit, 4-leaf layer construction unit, 5-intermediate layer construction unit, 6-query unit.

[0047] The large-scale data-oriented indexing method of the present invention is based on a tree data structure, including an index structure construction stage and a query stage. Figure 8 Shown is the overall flowchart of the indexing method, wherein the index structure constructed in the stage of constructing the index structure includes a leaf layer ind...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an indexing method for large-scale data. The method comprises an indexing structure constructing step and a query step, wherein the indexing structure constructing step comprises generating an indexing structure according to original data, and the query step comprises obtaining corresponding original data according to the indexing structure. The original data comprise at least one data block composed of data element groups; the indexing structure comprises a leaf-layer index and a root-layer index, wherein the leaf-layer index comprises a layer of leaf-layer indexing files which comprise at least one leaf-layer data block, the root-layer index comprises a middle-layer index, and the middle-layer index comprises at least one layer of middle-layer indexing files which comprise at least one middle-layer data block. The indexing method for the large-scale data solves the problem of low response speed during large-scale data query through a MapReduce framework and improves the query performance by introducing an indexing mechanism. The invention also discloses an indexing system for the large-scale data.

Description

technical field [0001] The invention belongs to the technical field of databases, in particular to an indexing method and system for large-scale data. Background technique [0002] Big data exists in many applications, such as web logs, sensor networks, social networks, astronomical monitoring, etc. For example, the Large Synoptic Survey Telescope (LSST), the product of a multinational collaboration, is expected to be completed in 2014. After completion, it will provide humans with unprecedented starry sky observation capabilities, generating 30TB of data per night. Big data has the following three characteristics: 1. Massiveness: in many applications, the amount of data becomes very large; 2. High speed: because data is generated very fast, it will continuously enter the system like a data stream; 3. Diversity: There are many types of data, both structured and unstructured. [0003] The existing centralized processing technology cannot effectively manage big data. Some r...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/134G06F16/182
Inventor 李春生金澈清周傲英
Owner EAST CHINA NORMAL UNIV