Region allocation method and system of hbase table

A distribution method and distribution system technology, applied in the transmission system, electrical components, etc., can solve problems such as uneven distribution of Regions and prone to hot spots, and achieve the effects of avoiding system paralysis, reducing hot spots, and reducing request time

Active Publication Date: 2018-05-08
CTRIP COMP TECH SHANGHAI
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The technical problem to be solved by the present invention is to overcome the defect that hotspots are prone to occur due to uneven distribution of Regions with different Hash column values ​​in the prior art, and provide a HBase table that can evenly distribute Regions with different Hash column values. Region allocation method and system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Region allocation method and system of hbase table
  • Region allocation method and system of hbase table

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0050] see figure 1 , a Region allocation method for an HBase table, comprising the following steps:

[0051] Step 101, count the Hash column value of each Region in the HBase table of the cluster.

[0052] Step 102: Calculate the average distribution number of each Hash column value in the cluster, and the average distribution number of each Hash column value is equal to the total number of Regions corresponding to the Hash column value divided by the total number of RegionServers in the cluster.

[0053] Step 103. Select a target RegionServer from the RegionServers of the cluster.

[0054] Step 104, counting the distribution of Hash column values ​​of the target RegionServer. The Hash column value distribution includes the Hash column value of each Region of the target RegionServer.

[0055] Step 105, using each Hash column value on the target RegionServer as the Hash column value to be processed respectively, and judging whether the number of Regions corresponding to the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a Region allocation method and system of an HBase table, wherein the Region allocation method includes: counting the Hash column value of each Region; calculating the average distribution number of each Hash column value; selecting a target RegionServer; counting the target RegionServer Hash column value distribution; each Hash column value is used as a pending Hash column value, and whether the number of Regions corresponding to the pending Hash column value is greater than the average distribution number of the pending Hash column value, if greater, from the pending Hash column value Select the N first Regions from the Regions corresponding to the Hash column values ​​and remove them. The present invention can make up for the disadvantages in the prior art that the Regions with different Hash column values ​​are unevenly distributed in the cluster, resulting in easy occurrence of hotspots, and evenly distribute the Regions with different Hash column values.

Description

technical field [0001] The invention relates to the field of Internet big data, in particular to a Region allocation method of an HBase table (a distributed, column-oriented open source database). Background technique [0002] The HBase table includes multiple Regions, and certain data is stored in each Region. A RegionServer in a cluster may be assigned to one or more Regions. [0003] Since the schema of some HBase tables was not originally designed with Hash (hash), resulting in uneven data writing and reading. Now, on the basis of the original schema, a Hash column is added, and the result set of Hash is used for pre-split (a method of using HBase). Because the original Region lacks the Hash column, the Hash column value will be zero, and the new Region will be equally divided into different Hash column values. After the new schema goes online, the number of regions with non-zero Hash column values ​​is relatively small, and the balancer (a function built into HBase) ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/08
CPCH04L67/1097H04L67/1001
Inventor 李亚锋刘钰帆
Owner CTRIP COMP TECH SHANGHAI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap