Hbase-based internet log data inquiring method and device

A data query device and data query technology are applied in network data retrieval, network data browsing optimization, electronic digital data processing and other directions, which can solve problems such as low query efficiency, achieve convenient and direct concurrent query of online list data, optimize storage format, The effect of improving query performance

Active Publication Date: 2014-12-31
FUJIAN NEWLAND SOFTWARE ENGINEERING CO LTD
View PDF4 Cites 24 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The technical problem to be solved by the present invention is to provide an hbase-based online log data query method, which can solve the problems of large data volume analysis and low query efficiency in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Hbase-based internet log data inquiring method and device
  • Hbase-based internet log data inquiring method and device
  • Hbase-based internet log data inquiring method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0017] In order to describe in detail the technical content, the achieved objectives and effects of the present invention, the following description will be given in conjunction with the embodiments and the accompanying drawings.

[0018] hbase-HadoopDatabase is a highly reliable, high-performance, column-oriented, and scalable distributed storage system. Using hbase technology, a large-scale structured storage cluster can be built on a cheap PCServer.

[0019] Hbase is built on hdfs to provide a database system with high reliability, high performance, column storage, scalability, and real-time reading and writing. It is between nosql and RDBMS, it can only retrieve data through the primary key (RowKey) and the range of the primary key, and only supports single-row transactions (complex operations such as multi-table joins can be implemented through hive support). Mainly used to store unstructured and semi-structured loose data. Like hadoop, hbase's goal mainly relies on horizonta...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an Hbase-based internet log data inquiring method. The Hbase-based internet log data inquiring method includes inputting RowKey field values through a client, and sending an internet log inquiring request to the Hbase; subjecting the Hbase to acquiring a RowKey range meeting conditions according to the input RowKey field values, to be specific, the length of the RowKey is smaller than a preset length threshold value; according to the RowKey field values, subjecting the Hbase to finding out the corresponding column family range within the RowKey range and returning the column values of the found column families to the client, to be specific, all the internet log fields are saved in a column of a column family; subjecting the client to receiving and analyzing the column values. The invention further discloses an Hbase-based internet log data inquiring device. The Hbase-based internet log data inquiring method and device has the advantages that inquiry performance of the internet log data is improved, memory space occupation is decreased, and direct inquiry of the internet list data is facilitated for the external interfaces.

Description

Technical field [0001] The invention relates to the field of unstructured data storage and query, in particular to a method and device for querying online log data based on hbase. Background technique [0002] The data that is inconvenient to use the two-dimensional logical table of the database is called unstructured data, including office documents in all formats, text, pictures, XML, HTML, various reports, images, and audio / video information, etc. [0003] In the context of the continuous development of the mobile Internet and the ever-expanding amount of information, the large-scale systems of various mobile communication companies have recorded a large number of users' online logs. Many companies still use traditional ETL tools for data collection, analysis, and query. They are responsible for extracting data from dispersed and heterogeneous data sources, such as relational data, flat data files, etc., to the temporary intermediate layer for cleaning, conversion, and integrati...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/3331G06F16/957
Inventor 朱爱军陈威叶潇席先广
Owner FUJIAN NEWLAND SOFTWARE ENGINEERING CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap