Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

HBase secondary-index storage and query system and query method thereof

A secondary index and query system technology, applied in the field of data processing, can solve the problems of unsearchable index tables, low efficiency, large data redundancy, etc., to reduce full table scans, improve query efficiency, and fast data read and write speed Effect

Inactive Publication Date: 2015-12-30
FENGHUO COMM SCI & TECH CO LTD
View PDF4 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Existing secondary index storage and query schemes require redundant filter columns in each index table of the data table, resulting in large data redundancy; when querying, if the index table does not contain all the filter conditions in the query conditions, the The index table cannot be searched. If all the index tables are not searchable, it is necessary to use the query condition to construct a filter and perform a full table scan on the data table, which is extremely inefficient

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • HBase secondary-index storage and query system and query method thereof
  • HBase secondary-index storage and query system and query method thereof
  • HBase secondary-index storage and query system and query method thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] like figure 1 As shown, the present invention discloses an HBase secondary index storage and query system, including a client 1, a query processing module 2, a query execution engine module 3 and an HBase storage module 4, and the query processing module 2 receives the query sent by the client 1 Request, the query processing module 2 obtains the query condition from the query request, the query processing module 2 sends the legal query condition to the query execution engine module 3, the query execution engine module 3 matches the query condition, and finds out the query condition from the HBase storage module 4 that meets the requirements. For data, the HBase storage module 4 stores the index table 41 and the data table 42 respectively, the index table 41 is stored on the SATA hard disk, and the data table 42 is stored on the SSD solid state disk. If it is necessary to perform a full table scan on the data table 42, storing the data table 42 in the SSD solid state di...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of data processing, in particular to an HBase secondary-index storage and query system and a query method thereof. The HBase secondary-index storage and query system comprises a client-side, a query processing module, a query execution engine module and an HBase storage module. The HBase secondary-index storage and query system stores a data table in an SSD and is higher in data read-write speed compared with a traditional SATA hard disk, and the query efficiency is greatly improved. In addition, the HBase secondary-index storage and query system optimizes matching logic, can use a filter column condition included in an index table as a filter to conduct scan on the index table so as to obtain a rowkey of the data table, then uses a query condition as a filter to conduct accurate get on the data table by using the rowkey so as to obtain a query result, omits whole-table scanning of the data table and greatly improves the query efficiency.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to an HBase secondary index storage and query system and a query method thereof. Background technique [0002] With the development and application of big data technology, HBase has gradually become a NoSQL distributed storage system widely used in the industry. It is highly reliable, column-oriented, and open source, and has been successfully used in production systems by companies such as Facebook and Alibaba. How to efficiently store and query secondary indexes on HBase is a research hotspot in the industry. At present, the widely used scheme architecture is as follows: image 3 As shown, the solution mainly includes three modules: HBase storage module, query processing module, and query execution engine. The HBase storage module is responsible for the storage of original data and index data. It has the characteristics of distributed, large capacity, and fast response. Si...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
CPCG06F16/24553G06F16/2282G06F16/2453
Inventor 王勇强赵智峰周帅锋曹俊亮李佳宁韦蓉刘宇
Owner FENGHUO COMM SCI & TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com