A hbase-based online log data query method and device
A data query device and data query technology are applied in network data retrieval, network data browsing optimization, electronic digital data processing and other directions, which can solve problems such as low query efficiency, achieve convenient and direct concurrent query of online list data, optimize storage format, The effect of reducing storage space usage
Active Publication Date: 2017-12-08
FUJIAN NEWLAND SOFTWARE ENGINEERING CO LTD
View PDF4 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0006] The technical problem to be solved by the present invention is to provide an hbase-based online log data query method, which can solve the problems of large data volume analysis and low query efficiency in the prior art
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0017] In order to describe the technical content, achieved goals and effects of the present invention in detail, the following descriptions will be made in conjunction with the embodiments and accompanying drawings.
[0018] hbase-HadoopDatabase is a high-reliability, high-performance, column-oriented, and scalable distributed storage system. Using hbase technology, a large-scale structured storage cluster can be built on a cheap PCServer.
[0019] Based on hdfs, hbase provides a database system with high reliability, high performance, column storage, scalability, and real-time read and write. It is between nosql and RDBMS, and can only retrieve data through the primary key (RowKey) and the range of the primary key, and only supports single-row transactions (complicated operations such as multi-table join can be realized through hive support). It is mainly used to store unstructured and semi-structured loose data. Like Hadoop, HBase mainly relies on horizontal expansion to i...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses an Hbase-based internet log data inquiring method. The Hbase-based internet log data inquiring method includes inputting RowKey field values through a client, and sending an internet log inquiring request to the Hbase; subjecting the Hbase to acquiring a RowKey range meeting conditions according to the input RowKey field values, to be specific, the length of the RowKey is smaller than a preset length threshold value; according to the RowKey field values, subjecting the Hbase to finding out the corresponding column family range within the RowKey range and returning the column values of the found column families to the client, to be specific, all the internet log fields are saved in a column of a column family; subjecting the client to receiving and analyzing the column values. The invention further discloses an Hbase-based internet log data inquiring device. The Hbase-based internet log data inquiring method and device has the advantages that inquiry performance of the internet log data is improved, memory space occupation is decreased, and direct inquiry of the internet list data is facilitated for the external interfaces.
Description
technical field [0001] The invention relates to the field of unstructured data storage and query, in particular to an HBase-based online log data query method and device. Background technique [0002] Data that is inconvenient to be represented by two-dimensional logical tables of the database is called unstructured data, including office documents in all formats, text, pictures, XML, HTML, various reports, images and audio / video information, etc. [0003] Under the background of the continuous development of the mobile Internet and the continuous expansion of the amount of information, the large-scale systems of various mobile communication companies have recorded a large number of users' online logs. Many companies still use traditional ETL tools for data collection, analysis, and query. They are responsible for extracting data from scattered and heterogeneous data sources, such as relational data and flat data files, to the temporary middle layer for cleaning, conversion,...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
CPCG06F16/3331G06F16/957
Inventor 朱爱军陈威叶潇席先广
Owner FUJIAN NEWLAND SOFTWARE ENGINEERING CO LTD