Unlock instant, AI-driven research and patent intelligence for your innovation.

Method for batch merging of hbase table regions

A batch and table name technology, applied in the field of big data, can solve the problems that HBase is difficult to perform manually, and achieve the effect of avoiding low application efficiency, avoiding system crashes, and improving system performance

Inactive Publication Date: 2014-12-10
INSPUR GROUP CO LTD
View PDF2 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the merge command provided by HBase can only merge two regions at a time, which is difficult for HBase with existing terabytes of data.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for batch merging of hbase table regions

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0016] The advantages and design content of a method for merging hbase table regions in batches according to the present invention will be described in detail below through an embodiment.

[0017] The method for merging hbase table regions in batches described in this embodiment first obtains the region name list of the hbase table, then modifies the hbase script, and finally executes the script merge.sh for merging regions according to the region name list, and performs the script merge for merging regions. sh to complete batch merging of regions of hbase tables.

[0018] In the method for merging hbase table regions in batches described in this embodiment, the region name list for obtaining the hbase table includes: starting HBase, opening the hbase monitoring page http: / / master:60010 / in a browser, and opening the table to be merged , enter the corresponding frame http: / / master:60010 / table.jsp?name=dangdang. Create a new excel file, select "From Website" from the "Data" me...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method for batch merging of hbase table regions and relates to the technical field of big data. By obtaining a regionname list of hbase tables, name lists of the regions of the hbase tables are guided out and are stored as text files with the same names as list names; and by modifying an hbase script, a configuration file path is added; by executing a script merge.sh, the script merge.sh carries out polling on the name lists of the regions, two adjacent regions are merged, and the purpose of batch merging is achieved. According to the method, on the basis of a big data platform, the large number of regions of the hbase tables are subjected to batch merging, the problems that a presorting rule is not proper, or initial regionsize is too small, so that an empty region, a small region and split happen too frequently, and the number of opened files is too large are solved, system stability is improved, and the efficiency of an application program is improved.

Description

technical field [0001] The invention relates to the technical field of big data, in particular to a method for merging HBase table regions in batches. Background technique [0002] HBase is a distributed, column-oriented open source database. It is the most common database for Hadoop clusters and is suitable for unstructured and massive data storage. HBase can easily expand column families and columns to increase storage information; it can also easily expand nodes horizontally to increase computing and storage capabilities. This convenience also brings some problems. The most common ones are that the initial region size is too small, or the region pre-segmentation rules for a certain table are not appropriate, empty regions, small regions, splits are too frequent, and the number of open files exceeds the system Problems such as the upper limit will affect the efficiency of the process, and in serious cases will cause the system to crash. In order to solve this writing pro...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06F9/44
CPCG06F9/44505G06F16/313
Inventor 范莹于治楼梁华勇
Owner INSPUR GROUP CO LTD